Mastodawn

Llama 3.1 AI Models Have Officially Released

Big day for people who use AI locally. According to benchmarks this is a big step forward to free, small LLMs.

128k token context is pretty sweet. Mistral nemo also just launched with a similar context. Good times.

How does the Nemo 12B compare to the Llama 3.1 8B?

At long context, Nemo is way better than llama 8B in my testing.

Turns out they are both very sensitive to quantization though.

The loud minority is really loud.