The question the paper claims to address isn't whether current LLMs are AGI, the question is whether LLMs can reason. And solving a problem that seems to require reasoning some of the time is still evidence of reasoning - especially if the question is what we can expect this technology to do in the future.