Yes it seems like the thread is discounting that frontier providers are likely already baking new, stronger models. I agree that GLM and its ilk are quite good, but having used them I’m not convinced they’re on par with eg Opus in terms of things like tool calling. And they’re fast but less capable so I spend about the same amount of time with them, just with more hand holding. Maybe this is a harness limitation. I know on paper they seem comparable but anecdotally and qualitatively they’re not as useful as the frontiers’, so maybe there’s some truth to benchmaxing claims. For some workloads the distilled models may be good enough, and I suspect at some point there will be diminishing returns to spending a premium on frontier models, but I don’t think we’re there yet. That said I’m continuing to try them.

The question is whether this steals enough marketshare from frontier providers that they don’t have the capital to train the next model iteration. The open models are going to push down the unit price of an intelligence-token, but there will still be a market for a smarter bot. And as intelligence gets cheaper, the demand for it will rise (see Hank Green’s Jevons Paradox video). Not to mention there’s all kinds of other directions to go at the frontier (world models, robotics, video gen, etc).

Another thing, and this is pure speculation, but if the Chinese model providers already discovered the decrypting COT trick and leveraged it to do RL training, and assuming frontiers plug that hole, then maybe future distillation will be harder.