Hacker News Reader

Comment by warkdarrior | Hacker News Reader

What's a better model to run on one GPU?

QWQ 32B q4 OR deepseek-r1-qwen-32b for reasoning, Qwen2.5 Coder 32B q4 (pair with QWQ 32B) for coding

QwQ-32

If I ask qwq32 anything that is even slightly complicated it will ramble until it exceeds the context window, then forget my question. Q4k which is all that fits (with context) on a 3090.

Gemma3 27B gives me a rapid 1shot response, and actually works really well for the type of rubber duck brainstorming partner I often need.

I’ve had this same exact experience.

Try giving it a river crossing puzzle with substitutions. QwQ can take a lot of time but it will solve it. Gemma will just confidently give you a wrong answer, and will keep giving you wrong answers if you point out the mistakes.

Now, yes, QwQ will take a lot of tokens to get there (in one case it took it over 5 minutes running on Mac Studio M1 Ultra). Nevertheless, at least it can solve it.

Yeah, but how many river crossing puzzles and murder mystery games was it trained on, and how many times do I actually need to solve a river crossing puzzle?

It's unequivocally not. What usecase do you have that QwQ-32 is outperforming Gemma3? In real world uses I didn't even prefer it to Gemma 2.

Anything that requires reasoning rather than regurgitating. For a simple example, try the classic river crossing puzzle with non-trivial substitutions. Gemma can't solve it even if you keep pointing out where it fucks up. To be fair, this also goes for all non-CoT 70B models, so it's not surprising. But QwQ can solve it through sheer persistence because it can actually fairly reliably detect when it's wrong, and it just keeps hammering until it gets it done.

Yes ok but when does this come up in the real world?

It's an example of a problem that requires actual reasoning to solve, and also an example of a "looks similar therefore must use similar solution" trap that LLMs are so prone to.

Translating this to code, for example, it means that Gemma is that much more likely to pretend to solve a more complicated problem that you give it by "simplifying" it to something it already knows how to solve.

Reply on news.ycombinator.com