How to Test-Drive an AGI

> I remain sceptical about any form of combination of reinforcement learning and LLMs.

I'm inclined to agree with this. And if we treat language learning as a paradigmatic example of a non-trivial problem, it certainly requires extensive environmental interaction (regardless of the stance taken on the degree to which language is "modular" - hopefully the basic statement that "real-world language learning is hard and situational" is non-controversial).

The intention of the linked article is to start from a good-faith position on claims of plan-making AGI, and then to say, "Okay, we've got this thing that outputs language that is supposed to reflect deeper knowledge of the ways of the world. If we don't have access to some underlying structure that convinces us of its capabilities, what enquiries can we make of the way it deals with somewhat known aspects of language itself that might at least begin to point to some of the necessary conditions for concept-formation and plan-making?"

The post outlines what are intended to be some fairly concrete tests that could be run in a systematic empirical way (or also preliminarily prodded in a more ad hoc way by anyone with access to an LLM). Like you, I'm pretty sceptical of a strong positive outcome for a model trained on nothing other than a very large sample of distributional data about linguistic symbols. But hopefully this is an interesting starting point for giving an AGI a chance to prove itself.

I remain sceptical about any form of combination of reinforcement learning and LLMs.

Acting successfully in the world when faced with complex issues requires learning useful ad-hoc concepts from the specific situation you find yourself in. It's plausible an AI can learn template tactics from large datasets, but I don't think that's enough.

There are various fields, like creativity research and design thinking, where it's understood that non-trivial problems need interaction with the environment to frame a problem in a way that allows an approach to a solution. This is because of the uniqueness and novelty in the situation itself.

It might be my lack of imagination, but I don't see how a deep learning on a large data set will get there.