Hacker News Reader

Comment by KronisLV | Hacker News Reader

I hope really badly that we'll get a new 35B A3B or similar MoE model!

I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model <100B would be the sweet spot (when you have the VRAM but not the TDP or compute power). Heck, I'd gladly take A5B or A8B or even A10B as a sort of middle ground.

Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

View on HN

I'm hoping too that they'll put out some MoE variants.

Qwen3.5:122b:a10b can run about twice as fast as this 27b dense model.

Edit: Like its predecessors, 3.8 seems really inclined to overthinking, and on a 27b dense model that's kind of painful. I think I'm going to stick with gemma4:26b-a3b as my go-to because it runs about 4x as fast and tends to only need a fraction of the tokens in its 'thinking' stage to get the same or similar answer.

Did you try the claude reasoning traces finetune for qwen3.6? I find that it works muuch better. I assume the same 3.8 finetune will be released at some pointas well.

Edit: link - https://huggingface.co/rico03/Qwen3.6-27B-Claude-Opus-Reason...

Reduce or turn down thinking:

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Yeah, that's probably the answer given that it apparently defaults to 'xhigh'.

Probably helps it score a little bit better in benchmarks :) `medium` seems like a nice balance so far; along with some light steering to vary think effort as needed for task and being pragmatic.

Interestingly 'medium' is the closest thing the _model itself_ has to a default thinking level. The chat template injects directions [1] at the very start of the system message when the reasoning effort is 'xhigh' or 'low' but 'medium' implicitly just means no added reasoning-level instructions.

[1] The specific directions are "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer." and "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."

"inclined to overthinking," holy cats you're not kidding! On a Mac mini M4 Pro 64GB I prompted it with "svg owl" and it thought for 17m12s, outputting 36.3KiB of thinking chatter. It did end up producing a 20.2KiB HTML+JS+SVG file with a very nice owl, including cursor-tracking animation, but it ran for more than a half hour!

The MoE models are stupendously faster.

I think that's called test-time scaling i.e using more tokens at infer time to squeeze out higher model performance. That's must be part of the explanation for good benchmark results.

Yep.. it's pretty obnoxious for real-world use with the default 'xhigh' thinking. Ridiculous amount of "Wait, actually.." which might help for complex coding tasks but makes it unbearable for general purpose use.

that seems to be how most Chinese models achieve increased benchmark scores. GLM and Kimi models are "thinkslop" models that reason over their own thinking, which increases cost and decreases speed significantly. That's why GPT is in a different tier altogether - faster, smarter, and sometimes cheaper.

what harness did you use?

No agentic harness; I just prompted literally "svg owl" into LM Studio 0.4.21+2. macOS Tahoe 26.6.1, Mac mini M4 Pro (14 core, 20 graphics cores) 64GB RAM. (I put the machine config so the 17m12s of thinking has some context.) It got about 12.75 tok/s and used 21,769 tokens.

For comparison's sake, Qwen 3.6 35B A3B MLX, same prompt, same machine, thought for 1.59 seconds, used 2,398 tokens, and hit 80.83 tok/s. It actually output just a straight SVG file of an owl (not a bad owl either!), whereas Qwen 3.8 27B spent a zillion times longer gold-plating everything and output HTML+JS+SVG with interactive animation, sound, etc., etc.

Same here! Qwen3.6-35B-A3B is the only local model I've found that runs reasonably on my iGPU. Looks like me and and my noisily-wheezing laptop will be sitting out this upgrade.

Mind sharing your laptops specs? Just interested to see what is needed to locally run Qwen3.6-35B-A3B

Don't mind! It's 64 GiB dual-channel DDR5-6400, i.e. roughly 100 GiB/s of bandwidth. (AMD 7840U (Zen 4))

I'm using a Q4 quantization from unsloth (Qwen3.6-35B-A3B-UD-Q4_K_XL). It gets up to ~20 tokens/second in generation. I don't know precisely how much KV cache I can safely use, but it's in between 140k–256k. (I.e., 140k reliably works, 256k kernel-crashes from OOM. Don't feel like bisecting).

Inference is llama.cpp with the Vulkan GPU backend on Linux. (I.e., -DGGML_VULKAN=1 on the llama.cpp build, and --gpu-layers all on llama-cli or llama-server. (And for my specific setup, two kernel parameters specific to amdgpu: ttm.pages_limit and ttm.page_pool_size. A driver VRAM limiter. Look it up if you're on amdgpu!)).

Thanks! I don't have any knowledge of running models locally.

I assume it would not be able to handle an unquantized Qwen3.6-35B or is it irrelevant as you almost always would want to run a quantized version of the model on consumer hardware?

not parent, but 4-bit quantization is generally consider a good trade off for speed/performance, so you might use it even when you aren't on consumer hardware, but definitely when you are on consumer hardware.

yeah, that's the A3B part; going up to A5B would probably also feel comfortable.

on the 395+ AI MAX w/128GB, the A10B qwen 3.5 can do a lot of long running work if you don't need to baby sit it. deer-flow works well like that.

[self-reply because comment edit window ended]: I now have a head-to-head benchmark. On my wheezy laptop (specs in sibling comment), this new dense model, Qwen 3.8 27B, gets ~4 tokens/second on generation. The older mixture-of-experts model, Qwen 3.6-35B-A3B, gets ~20 tokens/s.

MoE is literally 5x times faster (on CPU) than comparable dense Qwen!

Tested:

Qwen3.6-35B-A3B-UD-Q4_K_XL

Qwen3.8-27B-UD-Q4_K_XL

35A3 might be more comparable to 10 dense.

27 dense is far more capable than 35A3.

So you happen to know how to demonstrate this higher capacity, what do we mean by that? Can understand bigger codebases? And how do we explain it, what part of the model does it?

If you are not already using MTP, you should be able to get ~2x decode tokens/s with Qwen 3.8 27B.

Got Qwen3.8 to run on my Framework 12 Intel Core 13 Gen Raptor Lake i5-1334U small laptop with 48G RAM stick:

llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL

but it failed my basic prompt to compose a vim regex to match CamelCaseWords

downgraded a bit with Q4_K_M from ollama run qwen3.8:27b

and /set nothink and at least 1 regex matched FooBar

prompt eval at 2.8 t/s eval at 0.94t/s

I particularly enjoyed this usage of the regex: /%\1\%/ ... onward for 700+ characters of \%\/ :-)

AgentWorld is a pretty recent MoE release from the Qwen folks with quite a bit better performance than 3.6, released around the same time as 3.7 Plus/Max, I guess. Not sure why it didn't get more attention, as it is definitely better than 3.6 35B A3B on all dimensions, but especially for agentic use. Still nowhere near the dense models (even 3.6 27B), but clearly an upgrade in the small Qwen MoE line.

https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B

KAT Coder is another Qwen 3.6 MoE fine-tune that also improves on 3.6 by a measurable amount.

But, I do hope for a bigger MoE, in the 70B to 120B range, something in the Coder Next lineage. I've got a Strix Halo that isn't getting used to its best ability because the best models all run fine on my faster desktop dual 32GB GPU setup, and they run too slow for comfort on the Strix Halo.

HUH? AgentWorld is a simulation of the world (e.g. tools and programs) for use by an agent!

And, yet it outperforms Qwen 3.6 35B A3B on Terminal Bench and SWE, etc. I dunno.

Edit: I guess you're right; apparently it's for simulation. I didn't look into it beyond the benchmarks. But, it does work in an agentic context, regardless. It'll write code, and drive an agent.

Me too. 35B A3B runs really fast on my MacBook Pro (M4 Max) and is suitable for real-time tasks like dictation post-processing. The dense model is not.

I'm still confused about Qwen 3.6 35B A3B. Everything I read said that the 27B model performs better at coding tasks, so what's the purpose of the 35B model?

Runs faster on bad hardware.

Or even on fairly high end (by general public standards, not gaming community standards) consumer hardware. Fantastic fit for M-series Macs with 32-96GB RAM.

It's better for VRAM poor people. I get 4-5 t/s with 27B and 20-30 t/s with 35B A3B.

Also radically better on an M1 Max. I get well up into the 60s t/s with the A3B, stuck at 9.5 or so with this new 27B, though perhaps an MLX build will help.

The "a3b" refers to its active parameters -- unlike 27b it is a mixture of experts model, so it runs much faster, about as fast as a 3b model, but needs as much memory as a 35b model! So good for unified memory systems like macs :)

anecdotes: 35B-A3B does want more memory, bigger model. But if you get it running it will be faster and more enjoyable to use -- text will fly by -- due to only 3B params being active, in my experience at least.

MoE models have less active parameters in play at any particular moment, so they perform much faster on lower bandwidth memories (like Strix Halo or DGX Spark) and also use less memory generally.

Dedicated GPU memory tends to be much faster (either DDR6 or HBM). So if you can fit a whole dense model in it, you're probably better off with that.

35Ba3b is usable in plain cpu inference on a fast server, the dense model is MUCH slower. On GPU the 35ba3b is still around 2x the tok/s single threaded, which can be a good tradeoff for some applications.

> I'd gladly take A5B or A8B or even A10B as a sort of middle ground.

Whats up with focusing on the active param count? Do yall fiddle with the weights or something?

You can run these on CPUs at a somewhat reasonable speed.

Or (somewhat) low TDP GPUs for that matter, like workstation ones, that might have enough total VRAM but not the best bandwidth/compute.

Total param count decides how much vram you need to run it. Active param count decides how fast it runs. My 10 year old GPU can load quantized 35B or 27B, but it can’t process 27B parameters per token faster than 2-4tok/s, while it can do A3B at >40tok/s

Thank you Kenny

Reply on news.ycombinator.com