What is the "hard line" that makes something AGI or not AGI? Because IMO it looks like GPT4 is somewhat AGI, but also the older models possibly all the way down to even Markov chains: it's just that this AGI is nowhere near human-level.
However, I think we can certainly prove when it doesn't. For instance, the fact we need an external plugin to get the model itself to return text claiming 1+1=2, tells me that GPT4 cannot reason about numbers in the abstract, and therefore lacks abstraction ability.
I think we're so strongly biased against a deeply uncomfortable reality that there may not be a hard line, we don't even want to consider the alternative.
But to make matters worse, or more muddied, 1+1=1 can be a valid mathematical statement. It simply depends upon the set and group you have, or if you're doing modular arithmetic, etc. Sometimes you're given a unital magma. So, there's still a heavy dependency on context for the problem setup, but the underlying discrete and deterministic rules that are applied to the context is less malleable than other context switches in NLP LLMs do well in (such as language styling).
An example about ping pong players. The pace of their movements is too fast for conscious thinking, so it's all trained reflexes with some overall strategic planning trying to keep up with events. There is no time to think about anything. Is intelligence suspended there, at least the general one? Then the same person stops playing and gives an interview about the game and full general intelligence turns on again.
Humans certainly have context windows. Try asking your CEO about some lines of code in your work. Humans have a fairly large one, I give you that and it is fuzzy.
Human level performance on general tasks (you can't be just good in 90%, you should avoid being terrible in the remaining 10%).
Though LLMs don't need to be AGI, to have a catastrophic economic impact.