RT @TeksEdge: 🔥 RTX 5090 + Gemma 4 31B: Real user testing right now 💳️ 32GB GDDR7 gives excellent headroom for higher quants on this dense 31B model. 🧪 Typical performance (llama.cpp + early user reports): QuantApprox. VRAM (weights + overhead)Expected TPS (generation) ⚡ Q4_K_M ~18–21GB 55–75+ t/s 📈 Q5_K_XL ~22–25GB 45–65 t/s 🐢 Q6_K / Q8 ~26–32+GB 35–55 t/s Users are actively testing 🐌 Unsloth UD-Q5_K_XL on RTX 5090 and tuning with TurboQuant / KV cache compression for better speed. Great quality + performance balance for local Gemma 4 31B inference 👌 Who else is running it? 👀

Mehr auf Arint.info

#llama #Unsloth #arint_info

https://x.com/TeksEdge/status/2040602823444791727#m