GPT-3 stays incredibly in-topic. You can give it logic problems it won't be able to solve, but GPT-3 is capable of things that, as a GOFAI supporter, I never thought brute-force connexionist approach could do. I am now very cautious before saying a task can't be done by DL because it requires "understanding".

These approaches discover concepts and the relationship between them, and use that in their tasks. It is not far-fetched to say that there is some kind of understanding there.

For now we have trained it to generate fake text and basically made a master bullshitter, but I have no doubt that it can easily extract meaning and intent from text.