@cesar_s

1 Followers
2 Following
2 Posts
In the intersection of AI, Ethics and Philosophy
Show threadcesar_sNov 5, 2024
@martin thanks for the clarification! it’s hard indeed to use on-device LLMs for now. Even quantized versions are slow at inference on regular devices. Again, awesome keynote!
cesar_sOct 26, 2024

Great keynote from @martin during moodlemoot global 2024. He said something interesting that right now it’s hard to embed open sourced AI in mobile devices. However, just a couple of days ago, Meta released lightweight llama models that can run in mobile devices, the speed at which AI is getting developed is amazing.

https://ai.meta.com/blog/meta-llama-quantized-lightweight-models/

#moodlemootglobal24 #ai

Introducing quantized Llama models with increased speed and a reduced memory footprint

As our first quantized models in this Llama category, these instruction-tuned models retain the quality and safety of the original 1B and 3B models, while achieving 2-4x speedup.

Meta AI