The other models are literally distilling OpenAI’s models into theirs.
I'm not even sure what is being alleged there—o1's reasoning tokens are kept secret precisely to avoid the kind of distillation that's being alleged. How can you distill a reasoning process given only the final output?
The first model has a much harder problem. It must figure out a way to iron out the differences between all the different models (i.e. multiple humans, or noise in the dataset) it is trained on, to come up with its single viewpoint of relationships. It must then be tuned to optimally fit its purpose, instead of simply operating like a (composite) rando human.
The second model then has these advantages:
1. Trained on a single model, whose behavior comes out of consistent learned relationships, as apposed to many inconsistent models with noise (lots of different humans operating differently at any given time for reasons beyond what the data can show).
2. Trained on post-tuned model behavior directly. One-step training is a considerably simpler problem than two-step. It doesn't learning anything it will need to unlearn or modify.
3. These simplifications both reduce the necessary size of the second model.
And with these combined advantages (a simpler more consistent single-system to model, smaller size, and fewer training steps), it can be trained much faster.
There, of course, can be disadvantages. The distilled model may not be as versatile, if the data used to prompt the original model isn't as diverse as the first models data. So prompt data for training still matters.
OpenAI was just first out of the gates, there'll always be some company that's first, essence is how they handle their leadership, and they've sadly been absolutely terrible and scummy.
Actually i think Google was a pretty good example of the exact opposite, decades of "actually not being evil", while openAI switched up 1 second after launch.
What is your point? OpenAI wasn’t the first out of that gate as your own argument cites Google prior. All these companies are predatory, who is arguing against that? OP said OpenAi was irrelevant. That’s just dumb. They are not. Feel free to advance an argument in favor of that narrative if you wish as I was just trying to provide a single example that shows that some of these lightweight models are building directly off the backs of giants spending the big money. I find nothing wrong with distillation and am excited about companies like DeepSeek.
Google's overwhelming victory in search had ~ nothing to do with marketing.
There was a time when Google was thought of as a respectable, high-quality, smart and nimble company. That has faded as the marketing grew.
Companies are 100% using these big players to generate synthetic data. Distillation is extremely powerful. How is this even in question?