The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.
edit: GP giving more downvotes than proofs
You just change the prelude and use it for something new.
But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.
I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).
However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.
(https://www.reddit.com/r/SubSimulatorGPT2/ but for HN.)
For a more extensive rebuttal, I wrote one here. https://news.ycombinator.com/item?id=23346972 Though that was more a rebut of GPT in general as a path to AGI than metalearning in particular for generating memes.
That being said, I'm not sure I understand why you can't use GPT-3 to make an HN simulator.
The counterpoint is that it would be significantly cheaper AND have better performance to fine-tune models to each customer's use case than it is to just run GPT-3 at inference.
GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers.
"We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can do the job their ML engineers were doing.
I think everyone understood that.
One thing GPT-3 is not able to do for example, is replacing 2 ML engineers to build a GPT-3 like model. But OpenAI can do that.