Indeed, see Gary Marcus' critique from last year: https://thegradient.pub/gpt2-and-the-nature-of-intelligence/
(the critique is: GPT-3 can in fact do all the things Marcus said it couldn't)
Q: what comes next in the series: 0,3,6,9, A: 12
Increases in reasoning power should allow for much smaller usable models.
The fun bit is generalization. Create a pattern that hasn't been read before. Hard with GTP-3 because it's been given everything to read...