Hacker News Reader

Comment by nchmy | Hacker News Reader

I've been working with the chinese open models for 4 months. They are more than capable for a tiny fraction of the cost of the frontier ones. And yet they also continue to get significantly better and (Deepseek's recent price increase aside) cheaper. Its hard to fathom how the truly frontier stuff will be able to compete long-term.

View on HN

And why won’t the frontier models continue to become better? The open models are getting better but so are the frontier models. The frontier models might remain in a constant race to remain ahead

They are running out of novel, clean training data and compute. There is probably a limit to how much improvement can be squeezed out of LLMs. Recent improvements have been more about orchestration and "reasoning" loops (i.e. iteratively feeding context back through the model).

For base models they really must be running out of new training materials. It's more about size and architecture. But they still seem to be getting big strides out of improving the post training. They keep dropping point upgrades in under 8 weeks lately, which is an insane pace for product release. At some point it's going to slow down but we're not close yet imo.

Diminishing returns on both intelligence and training, mostly

I don't think it's a matter of "staying ahead"; the proprietary frontier models are better, but the trouble for them is that open weight models are good enough in increasingly many cases. This raises the floor on the frontier companies and cuts their total addressable market by commodifying the easier LLM tasks. This is really the central argument of tfa :)

The big question is if there are larger economic gains to be made from ever greater intelligence. It seems to me like that may be the case for only a select few hard problems, while the vast majority of tasks approach their economic ceiling asymptotically with intelligence.

You suggest that the market segment for frontier models is necessarily shrinking. But what if it's just a failure of imagination on the part of users ?

I suspect due to model distillation, their need to stay on top for IPO value is more important than absolutely crushing their competition, who will simply harvest their output and distill it for training data on a ~3 month delay. Also they are probably hitting a wall on increase in intelligence vs training time.

Reply on news.ycombinator.com