Mastodawn

Running Qwen 3.6 27B locally on a 24GB GPU with Podman and llama.cpp.
Covers NVIDIA CDI GPU passthrough, KV cache presets, and working configs for coding, reasoning, and vision.

https://scavazzon.com/posts/run-qwen-3.6-27b-locally-on-a-24gb-gpu-with-podman-and-llama.cpp/

#localLLM #llamacpp #podman

Run Qwen 3.6 27B locally on a 24GB GPU with Podman and llama.cpp

Run Qwen 3.6 27B locally on a 24GB GPU with Podman and llama.cpp. NVIDIA CDI passthrough, KV cache presets, and working configuration examples.

Marco Scavazzon