One year ago the #Framework13 AMD Ryzen 300 AI series got released. It has a neural processing unit (NPU).
Unless you compile your own kernel and a bunch of modules and toolchains the NPU hardware is still unusable on any Linux distribution out there.
One year later and there are zero end users on that NPU on Linux. It's dead hardware.
When (if at all) it's integrated smoothly, the NPU hardware will be outdated hardware.
#AMD 🤝 #Framework that's what Linux-first looks like? 💩🏆
@djh doesn't lemonade work now?
https://github.com/lemonade-sdk/lemonade/releases/tag/v10.0.0
@crepererum Exactly!
What do you think how long will it take for any end user app to even consider implementing support for it? My bet is on never.
At that point you gotta ask yourself:
Is it really worth the trouble of a custom ex-Xilinx built hardware chip that will not see any usage from 99% of end users.
Or can you just put e.g. AVX512 in there and have it in end user hands very very quickly.
(Disclaimer: The Framework 13 AMD Ryzen AI does have AVX512 which I'm really happy about!)
@djh TBH with that Hardware I would rather bet on Vulkan compute, also see
https://fosdem.org/2026/schedule/event/CZSPSC-llama-cpp-vulkan/
On my dedicated AMD GPU I DO get better llama.cpp perf using Vulkan compared to ROCM. And it's A LOT faster than CPU.
Daniel
crepererum
