Gus
Miguel de Icaza ᯅ🍉Yesterday they reverse engineered Apple’s neural engine: https://github.com/maderix/ANE
And today people are running the amazing Qwen3.5 accelerated on an m1, this is insane:
Eric Vitiello@Migueldeicaza yeah now I definitely need to upgrade my M1 iMac.
B O@Migueldeicaza why wasn’t it open in the first place? Oh right, it’s Apple… pay obscene amount of money for hardware you can’t even fully utilize
@piyuv internal APIs are done so you can change the implementation without breaking people - that’s why people do it. Every time you expose things it ties your hands.
Joseph Humfrey@Migueldeicaza I’m confused, isn’t a model that’s converted to MLX effectively always running on the neural engine? Or it just on the GPU?
@Migueldeicaza I think Apple's AI strategy should really be: don't make their own model but make a software strategy that can beat cuda and leverage apple silicon memory model to beat NVidia (i.e. have OpenAI lining up to buy mac minis)
Arcticulate@tylercheung @Migueldeicaza No, I’m an M1 Mac Mini owner (and 2012 i7 Mac Mini before that). What if they always end up being sold out at times when consumers are interested to buy one? 😅.
@Arcticulate @Migueldeicaza that's true but I have more faith in Tim Cook making chips than Jensen making chips, judging how atrocious the GeForce4000 and 5000 series roll outs have been…
@tylercheung @Migueldeicaza I don’t use gaming PCs and feel kinda lost in the gaming GPU world of things. I only know about the budget vs mid-range vs premium setup, from my ownership of GeForce 2 GTS way back in the yesteryears of using a PC with Linux for many years.
The only non-retro PC I have at home these days is my WFH laptop, aka my employer’s HP Elitebook with Windows 11 Pro. That one is ”beefy for M365”, but not much else.
@tylercheung @Migueldeicaza However, I am impressed with Tim Cook’s logistics guru skills.
सुरेन्द्र • Surendrajat@Migueldeicaza wasn't it already reversed by @AsahiLinux people here https://github.com/eiln/ane ? What's the difference exactly?
@surendrajat This is one of those situations where clicking on the link and reading it would answer the questions.
Tom Armstrong@Migueldeicaza Excuse the potentially naive question, I’m not all that familiar with the context; the GitHub link implies that the breakthrough is training using the ANE, but the video shows inference, right?
I had thought it was already possible to run the Qwen models on Apple hardware. What does the ANE reverse engineering effort do to aid the inference shown in th video?
@tom_armstrong before this stuff, you only could use CoreML to use the ANE, which is too limited to run LLMs. Now that they cracked open the ANE, they can. They give additional details on the link and blogs linked.