DanielMar 14

One year ago the #Framework13 AMD Ryzen 300 AI series got released. It has a neural processing unit (NPU).

Unless you compile your own kernel and a bunch of modules and toolchains the NPU hardware is still unusable on any Linux distribution out there.

One year later and there are zero end users on that NPU on Linux. It's dead hardware.

When (if at all) it's integrated smoothly, the NPU hardware will be outdated hardware.

#AMD 🤝 #Framework that's what Linux-first looks like? 💩🏆

Show threadcrepererumMar 14

@djh doesn't lemonade work now?

https://github.com/lemonade-sdk/lemonade/releases/tag/v10.0.0

Release v10.0.0 · lemonade-sdk/lemonade

Headline Linux NPU support is available for LLMs and Whisper via FastFlowLM Native integration with Claude Code: lemonade-server launch claude A Fedora .rpm installer is now published in the relea...

GitHub
Show threadcrepererumMar 14
@djh ah I see, you still need the XRT kernel drivers, right? That's stupid, esp. Because other NPUs (like Rockchip) work using a unified kernel interface and Mesa.
Show threadCaleb MaclennanMar 16
@crepererum @djh Am I missing something? Even with the xrt tooling and xdna kernel modules installed I am unable to get the NPU to work via FastFlowLM in Lemonade. The Lemonade site asserts that only the AI Max 300 chips are supported, not the AI 300 that is in the FW13 laptops.
Show threadcrepererum
@alerque @djh that's probably my fault for not reading the whole thing, sorry.
Mar 16 at 8:50amWeb
Show threadCaleb MaclennanMar 16

@crepererum @djh It's a little confusing because the mobile chips seem to fall through the cracks on the supported hardware table here:

https://lemonade-server.ai/flm_npu_linux.html

If I'm not mistaken the current generation of FW13 has an AI300 series chip which in newer than the 7000/8000 series chips with XDNA1 which is listed as unsupported but not actually the Strix Halo in the Max 300. Hence I'm not sure whether it is / will ever be supported.

LLMs on Linux with FastFlowLM

LLMs on Linux are now supported with FastFlowLM. This guide helps you set up the FLM stack on supported AMD XDNA 2 NPUs and troubleshoot common validation issues.

Show threadCaleb MaclennanMar 16

@crepererum @djh

ABORT ABORT ABORT. Maybe I should get my eyes checked. The second line of that table does list the 300 series separately from the Max 300 series. That puts it firmly in the realm of documented as supported, so I'll go back and poke at my system and see what is still missing.

Either way to the point of this thread it definitely isn't "works out of the box" realm yet, but when Linux 7 gets out of RC status it's looking hopeful that it might get to that status.