I can't play with GPT-3 but when I play with GPT-2 I can easily trick it with counting games. It does well with 0,1,2,3,.... but things like 0,1,3,6,10, get poor responses. Is GPT-3 good at that?
Q: what comes next in the series: 0,3,6,9, A: 12
Increases in reasoning power should allow for much smaller usable models.
The fun bit is generalization. Create a pattern that hasn't been read before. Hard with GTP-3 because it's been given everything to read...