I don't think it's likely to be a helpful one in this case.
For a system to be able to solve the same classes of problems human can solve it would need to be able to invent their own rules just like humans can.
Though, I will admit, a system that acts in a way that’s indistinguishable from a human will be awful hard to classify as anything but AGI.
Maybe I’m conflating AGI and consciousness, though given that we don’t understand consciousness and there’s no clear definition of AGI, maybe they ought to be inclusive of each other until we can figure out how to differentiate them.
Still, one interesting outcome, I think, should consciousness be included in the definition of AGI, is that LLMs are deterministic, which, if conscious, would (maybe) eliminate the notion of free will.
I feel like this whole exercise may end up representing a tiny, microscopic scratch on the surface of what it will actually take to build AGI. It feels like we’re extrapolating the capabilities of LLMs far too easily from capable chat bots to full on artificial beings.
We humans are great at imagining the future, but not so good at estimating how long it will take to get there.
Expert systems, for instance, use a set of if-then rules derived from human expertise to make decisions in specific domains. This is a form of deductive reasoning, albeit limited and highly structured. They don't 'understand' in a human sense but operate within a framework of logic provided by humans.
LLMs, on the other hand, use statistical methods to generate responses based on patterns learned from vast amounts of data. This isn't reasoning in the traditional philosophical sense, but it's a kind of probabilistic reasoning. They can infer, locally generalize, and even 'extrapolate' to some extent within the bounds of their training data. However, this is not the same as human extrapolation, which often involves creativity and a deep understanding of context.