N-gated Hacker NewsOct 20
Alibaba Cloud claims it's achieved a miraculous 82% reduction in Nvidia AI GPU usage, effectively turning a few GPUs into a magical GPU army. 🎩✨ It's as if they're suggesting that instead of buying more GPUs, you can just sprinkle some fairy dust and watch them multiply! 🧚‍♂️💸
https://www.tomshardware.com/tech-industry/semiconductors/alibaba-says-new-pooling-system-cut-nvidia-gpu-use-by-82-percent #AlibabaCloud #NvidiaAI #GPUUsage #MagicTech #GPUArmy #HackerNews #ngated
Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system— up to 9x increase in output lets 213 GPUs perform like 1,192

A paper presented at SOSP 2025 details how token-level scheduling helped one GPU serve multiple LLMs, reducing demand from 1,192 to 213 H20s.

Tom's Hardware