Eric CurtinWhile everyone's talking about Apple's Containerization framework announcement at WWDC, just a few days ago #krunkit quietly hit a major milestone: GPU passthrough in VMs, with up to 80% native LLM performance.
#krunkit and #podman are still the best hypervisor and container combination on macOS. #libkrun is also a great option for AI microVMs. We are thankful to be able to take advantage of these features in #RamaLama #AI
https://github.com/containers/ramalama
But I'm still looking forward to where Apple's variants go.
GitHub - containers/ramalama: RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of ...