A CS lecturer of mine told us that when he was a student he had a lecturer who advised him to be sceptical of AI revolutions. That was nearly 20 years ago. I've no doubt we'll see further steps but I'm not going to hold my breath for something transformative.
Personally, my guess is that it’s actually just plagiarizing the training set in a way that most researchers will come to view as a kind of cheating. What I mean by that is, if you take some plagiarism detection software and run it on GPT-3’s output, it will ring like crazy.
I say this both because I believe it and because if it’s not the case, if we really have a proto-AGI on our hands, then being wrong won’t matter. I sincerely hope that we are a thousand years away from that, because otherwise we are plainly doomed.
We're doomed regardless. We don't have a thousand years. Maybe not even 100.
Somebody should try this. I ran a few paragraphs from AIDungeon through https://plagiarismdetector.net/ and got zero or low plagiarism percentages, but I'd imagine there are much better detectors that aren't publicly available.
(the critique is: GPT-3 can in fact do all the things Marcus said it couldn't)
Q: what comes next in the series: 0,3,6,9, A: 12
Increases in reasoning power should allow for much smaller usable models.
The fun bit is generalization. Create a pattern that hasn't been read before. Hard with GTP-3 because it's been given everything to read...