Hacker News Reader

Seriously though, what are the low cost chips that can usefully run LLMs? Is a Mac Mini the lowest we can go? Are there iGPUs on mini-itx that can do it, or are there dedicated AI chips that one could turn into a pi HAT?

View on HN

Depends on what you consider to "usefully run LLMs".

Earlier this year, I bought a mini pc from Aliexpress, specs are roughly Ryzen H255, 24GB LPDDR5, 1TB SSD. This was around 350€ including VAT, customs, shipping etc. I would personally consider this somewhat of a lowest class of useful LLM box. It can run 8B models well, up to somewhere around 24B. I currently run Gemma 4 26B A4B Q5 on it, with MTP, and it is quite slow, but smaller models would run okay on it.

Let's teleport back 10 years and what you have is a magic box that could make you billions.

An Orange Pi 5 Max does this job for real. It's an RK3588 board — $75 for 4GB, $95 for 8GB on AliExpress. A community test got Qwen2.5-0.5B at about 12 tok/s on the CPU via llama.cpp. The chip also has a 6 INT8 TOPS NPU if you'd rather go the RKNN route. Won't beat a Mac Mini, but it's an actual computer for under a hundred bucks.

I regret to inform you that prices have long departed the lower atmosphere. Even on AliExpress, you're looking at a few hundred bucks for those. Still less than a Mac Mini, but less less.

Reply on news.ycombinator.com