Hacker News Reader

These token numbers look off for a 5090. For comparison, an rtx 5000 Blackwell SFF with just 470gb/s bandwidth gets me 30 tok/s on gemma4-31B-QAT. Almost 60 tok/s with MTP enabled. A 5090 should get you much more than that!

View on HN

Thanks for flagging, this is on my local rig and it's driving my display too. I'm curious now, will take a closer look. These are the tok/s as reported by LMStudio.

EDIT: Updating llama.ccp gets me 58 tok/s on Gemma 31b

The Qwen-35B-A3B numbers are even weirder, did you drop a digit? I get half of that speed on a AMD Radeon RX 5500 XT (RADV NAVI14) (8192 MiB) (unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Q6_K with q8_0 context).

No, it's what I get repeatedly. Recall that it's QB_0 though. I'll get back to you when the Q4_K_M quant finishes downloading.

EDIT: I get 259 tok/s with the Q4_K_M quant

I assume you mean Q8_0, not QB_0? (was confused, as I've not obviously seen a QB_0 before and can't find one now).

bahaha, LMStudio's badge looks a lot like a B but the tooltip says it is indeed an 8 (I just assumed it was B for "byte", but that's a poor initialism considering "bit").

https://dl.jszym.com/share/boards/pictures/Screenshot_202607...

Agree - I got 9.7 tok/s on an M1 Max with unsloth's gemma-4-31B-it-qat-UD-Q4_K_XL.

Reply on news.ycombinator.com