Miguel de Icaza ᯅ🍉Yesterday they reverse engineered Apple’s neural engine: https://github.com/maderix/ANE
And today people are running the amazing Qwen3.5 accelerated on an m1, this is insane:
Tom Armstrong@Migueldeicaza Excuse the potentially naive question, I’m not all that familiar with the context; the GitHub link implies that the breakthrough is training using the ANE, but the video shows inference, right?
I had thought it was already possible to run the Qwen models on Apple hardware. What does the ANE reverse engineering effort do to aid the inference shown in th video?
@tom_armstrong before this stuff, you only could use CoreML to use the ANE, which is too limited to run LLMs. Now that they cracked open the ANE, they can. They give additional details on the link and blogs linked.