It will be AGI once it can update its own weights. It can't be "general" intelligence if its weights are frozen and requires to be updated manually.
An AGI test should be black-box; we shouldn't impose require requirements on internal components. As long as the overall AI is capable of learning and remembering things, it shouldn't matter if there's a stateless LLM internally.
Granted I have more memory than can fit in currently-practical LLM context windows, but RAG mostly solves that. When an AI is thinking about math, it can have relevant math memories in context without needing all the other stuff.