Hacker News Reader

Or attach a 12 or 16 GB GPU to a single Pi 5 directly, and get 20+ tokens/s on an even larger model :D

https://github.com/geerlingguy/ollama-benchmark?tab=readme-o...

Reading the beginning of your comment I was like “ah yes I saw Jeff Geerling do that on a video”.

Then I saw you github link and your HN handle and I was like “Wait, it is Jeff Geerling!”. :D

Haha I had nearly the same thing happen. First I was like "that sounds like something Jeff Geerling would do". Then I saw the github link and was like "ah yeah Jeff Geerling did do it" and then I saw the username and was like "oh it's Jeff Geerling!"

Thanks for sharing. Pi5 + cheap AMD GPU = convenient modest LLM api server? ...if you find the right magic rocm incantations I guess

Double thanks for the 3rd party mac mini SSD tip - eagerly awaiting delivery!

llama.cpp runs great with Vulkan, so no ROCm magic required!

Reply on news.ycombinator.com