behnamoh

Run LLMs on Apple Neural Engine (ANE)

https://github.com/Anemll/Anemll

GitHub - Anemll/Anemll: Artificial Neural Engine Machine Learning Library

Artificial Neural Engine Machine Learning Library. Contribute to Anemll/Anemll development by creating an account on GitHub.

GitHub
May 3, 2025 at 3:29pmWeb
Show threadantirezMay 3, 2025
The README lacks the most important thing: how many more tokens/sec at the same quantization, compared to llama.cpp / MLX? It is worth to switch default platforms only if there is a major improvement.