Hacker News Reader

It has been like 8 months since I got Ryzen 8700G with NPU just for the purpose of inferencing NN, and so far only acceleration I'm getting is through vulkan on iGPU, not NPU (I'm using Linux only). On the bright side, with 64GB of RAM had no isues with trying models over 32GB. Kudos to llama.cpp for supporting vulkan backend!

View on HN

You should have ROCm/HIP support on the iGPU as well, be sure to compile llama.cpp w/ the LLAMA_HIP_UMA=1 flag. If you take a look at https://github.com/amd/RyzenAI-SW you can see there's a fair amount of software to play with on the NPU now, but Phoenix is only 16 TOPS, so I've never bothered testing it.

Reply on news.ycombinator.com