Hacker News Reader

Comment by cyanydeez | Hacker News Reader

for comparison, Qwen3.8-Flash-Next only requires 6B parameters for computation, but stores 125B, 51B of those can be comfortably offloaded as they're not actively used in decode but a single token look up.

The Quant iQ4 of this model loads, then, in ~60GB of vram, and on disk it's 85GB.

So if you could etch it, you'd need a ~25GB ssd chip and 60GB of vram.

The vram costs likely contributed to these things being out of reach of the current economic cycle.

View on HN

You don't want to use a sparse model for a Taalas-like design. Something like a Qwen 3.8 27B makes much more sense.

I'm not smart enough to know why; I do know that 27B is greater for short/interactive on blackwell, but the intellgence leap of the MoE in Qwen3.8-Flash-Next is quite remarkable.

I'm pretty convinced the pathway to local models will be MoE, especially if they can find a way to keep tweasing out things like PLE into the slow bandwidth lanes.

The Taalas architecture makes loading weights free, but they basically has to pay the same silicon for every weight, whether it's used or not. MoE models are more efficient than dense models per weight you load, but less efficient per weight you have to store.

MoE models are the path to local models with traditional system architectures, but they are antithetical to what Taalas was doing. If you spent all the money to etch 125B weights into silicon, you'd want to activate them all for each token, instead of only touching 6B. You cannot match the 125B sparse model with a 27B dense one, but you might be able to match it with a 60B or so one.

Perhaps, but every time i conceptualize dense models, it seems like its overfitting, and the prime attention is never going to be that dense.

Reply on news.ycombinator.com