Hacker News Reader

I agree; I don't think there's any reason to assume Jevons paradox won't apply.

> If we have to switch to something like burning the model weights into silicon to continue to make gains

I think that's already being considered semi-seriously [0][1]

[1] https://ir.amd.com/news-events/press-releases/detail/1296/am...

What's really interesting is that if you scale it to higher densities (eg 3nm and stacked die) with ComputeInMemory for fp8 you can reasonably start to fit 30B-70B models. With MoE and multiple stacked die, just like HBM, you could fit an open weight near frontier 1T model (like GLM5.2) at similarly much lower power <10kW and high token rates >2ktps. For running a bunch of agents where fill rate and speed/latency are important it may not matter that you're 6-18mo behind on weights. The process for the chip design could be largely automated, and new silicon pumped out as new weights are available (with a 3-6mo delay).

Reply on news.ycombinator.com