And that's before factoring in what other models they are cooking that they're in no hurry to release.
If ChatGPT is so easy to outcompete, where's the competition?
Only few use cases need the best models. Most of them work ok on mini models and LLaMAs. The problem is that it costs very much to develop this shrinking segment of high end. With each new release in open / smaller models, the island of superiority shrinks for OpenAI. But people's task don't get any more difficult on average. So eventually your phone model will suffice for 99% of use cases, and be free and private. What will OpenAI do with 1% of the market?
These "catch all" use cases sound cool, but in practice are nearly worthless. Because you'll spend more time verifying than if you just did it yourself. The compelling use cases are the human-less use cases. Imagine ridding your entire accounting department because it can be handled by highly specific LLMs and calculators.
It takes like 10 seconds.
You’re just using it to point you in a direction, not solve the entire problem.
That's just not a compelling use case. It makes far more sense to just hire junior engineers. Because then they're actual engineers, and their knowledge base will grow and transfer over time.
If you have proprietary systems, then it may make sense to train an LLM on those. But I would consider that a more "focused" use case. I just don't see the main value of AI being a glorified, generalized google search.
There's potential here to automate tasks with MUCH less friction. Now business folk and people who understand processes (but not algorithms) can potentially automate business processes.
Yesterday, I implemented a recursive PostgreSQL function to crunch some pretty complex tables to produce a new output. As a senior engineer, I knew the output I wanted and understood the trade offs, but I didn’t want to spend 3 whole days trying to figure out how to build the recursive function myself.
Claude did it correctly in about 5 minutes, and then I spent another hour or two just tweaking and exploring alternate options.
I think it also goes without saying that I could not handle this to a junior engineer. We’re quite clearly past the “GPT is only as good as a junior engineer” phase.
It’s so different from anything else that’s ever existed. You could not do what I did yesterday without spending a LOT more time. Google would not have solved that problem, nor would a junior engineer.
This hasn't been the case in my experience.
All that feedback has made their product better and they're at a very different scale than everyone else.
The more interesting thing is when the user tests the AI ideas. Like running some code or applying advice for your hobbies. The user returns again and again communicating the outcomes in order to get new advice. The model can collect all chats across many days by topic, and judge them in retrospective. Which answers were good or bad?
The user base is so large, 200M users means lots of perspectives, lots of ideas, chatGPT puts trillions of tokens in human brains per month. They influence on a grand scale. After a while some article will describe the outcomes, and that data percolates back in the next training set. A feedback loop. And an indirect agent, working through people. An experience flywheel, collecting from hundreds of millions and serving back.