its funny, this little doodad supposedly has an 'npu' in it, and it was marketed towards ai bro types to be some kinda at-home llm-biased thing.
its an asrock-industrial nuks box-255h. its basically an off-brand intel nuc, and its *BEEFY*. its physically smaller than the phobos nucs (we have a fleet of em we ship to customers to do onsite stuff if they prefer that over someone in person) and its easily 10x the power.
but .. how do you use the npu at all?
ollama doesnt support it
@DaveMWilburn cool so you hafta custom compile stuff with all those build args, then after that you get access to..
nine models
whee!
I wonder if you could also figure out Vulkan support. It would also likely require custom compiling, but the advantage there is that Vulkan is likely better supported across a broader range of hardware and software. I've occasionally thought about that for my old RX 580, which lost support under rocm a long time ago.
Viss
Dave Wilburn 