Hacker News Reader

I have 4x r9700s as my coding daily drivers running qwen 3.8 27b at 80tps each. Zero complaints. Especially at $1200 each.

View on HN

No way you can get that much each without extremely quants

For a single r9700 you have 637 GB/s and for qwen 3.8 27b q4_k_xl the maximum tg/s is 33 before mtp

Now if you meant 4xr9700 tensor parallelism with mtp, 80 tg/s starts to make sense

You can get those numbers with https://codeberg.org/ggz14/radiance-vllm-mxfp4. I also can get it on a single R9700 but the 75 ~ 80 t/s is only peak acceptance of very predictable tokens like coding or json, and averages lower for prose. It's still much faster than regular llama.cpp.

Can confirm. I've a single R9700 and have maxed out at 45 tokens/second on llama.cpp with Q4 Qwen 3.6 27B (with MTP)

Q4 isn't an extreme quant, and I average 75 toks/s on code, 45 tok/s on prose with MTP.

Please share what operating system and model runtime you use? I have two and don't get close to that with AMD's own Lemonade. Thanks!

Lemonade is one of the worst performing options. Run any modern Linux distro and ask your current LLM to setup llama.cpp with dflash2 for you as an unprivileged container running from a systemd user unit.

Obviously only on a system you do not trust at all.

Reply on news.ycombinator.com