Does this mean we’ve reached the next AI winter? This is as good as it gets for quite a long time? Honest question :) perhaps this will postpone everyone’s fears about the singularity…
Another good example is in the coding landscape, which feels closer to existing. Ingest all of a company's code into a model like this, then start thinking about what you can do with it. A chatbot is one thing, the most obvious thing, but there's higher order product use-cases that could be interesting (e.g. you get an error in Sentry, stack trace points Sentry to where the error happened, language model automatically PRs a fix, stuff like that).
This shit excites me WAY WAY more than GPT-5. We've unlocked like 0.002% of the value that GPT-3/llama/etc could be capable of delivering. Given the context of broad concern about cost of training, accidentally inventing an AGI, intentionally inventing an AGI; If I were the BDFL of the world, I think we've got at least a decade of latent value just to capture out of GPT-3/4 (and other models). Let's hit pause. Let's actually build on these things. Let's find a level of efficiency that is still valuable without spending $5B in a dick measuring contest [1] to suss out another 50 points on the SAT. Let's work on making edge/local inference more possible. Most of all, let's work on safety, education, and privacy.
[1] https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-...
This is analogous to what’s happening with AI models. Sam Altman is saying we have reached the point where spending $100M+ trying to “beat” GPT-4 at everything isn’t the future. The next step is to chunk off a piece of it and turn it into something a particular industry would pay for. We already see small sprouts of those being launched. I think we will see some truly large companies form with this model in the next 5-10 years.
To answer your question, yes, this may be as good as it gets now for monolithic language models. But it is just the beginning of what these models can achieve.
I don't disagree, but it does align pretty well with the OpenAI business model, no? "No need to develop your own base model, just buy our own"
If anything successes in ChatGPT etc will be motivation for continued efforts.