hexualFeb 7, 2024

Smaug-72B-v0.1: The New Open-Source LLM Roaring to the Top of the Leaderboard

https://lemmy.world/post/11686094

Smaug-72B-v0.1: The New Open-Source LLM Roaring to the Top of the Leaderboard - Lemmy.World

Abacus.ai [http://Abacus.ai]: > We recently released Smaug-72B-v0.1 which has taken first place on the Open LLM Leaderboard by HuggingFace. It is the first open-source model to have an average score more than 80.

Show threadsimpleFeb 7, 2024
I’m afraid to even ask for the minimum specs on this thing, open source models have gotten so big lately
Show threadTheChurnFeb 7, 2024

Every billion parameters needs about 2 GB of VRAM - if using bfloat16 representation. 16 bits per parameter, 8 bits per byte -> 2 bytes per parameter.

1 billion parameters ~ 2 Billion bytes ~ 2 GB.

From the name, this model has 72 Billion parameters, so ~144 GB of VRAM

Show threadrs137
Llama 2 70B with 8b quantization takes around 80GB VRAM if I remember correctly. I’ve tested it a while ago.
Feb 8, 2024 at 6:23pmWeb