Hacker News Reader

If you’re on AMD, use llamacpp with the vulkan backend. For the 35B I get around 115 tps without mtp and 150-170tps with it. Also the pcie slot you use affects it

View on HN

I have issues with the vulkan backend. It seems to cause over all GUI sluggishness, but maybe that was actually Firefox. I'll give it another go.

Reply on news.ycombinator.com