The relevant question is whether Humans + LLMs are much more likely to get to AGI than humans without LLMS. And the answer is pretty obviously yes. I don't think anyone was arguing that we would get to AGI by just training on more data with exactly the same models. Practically every advance in the last few years has been building additional functionality on top of LLMs, not just scaling up the same architecture to more data.
But zooming out, LLMs are universal approximators, so it's trivially true that they can approximate any function that describes AGI. It's also true that logic (from logos or "word") is about reasoning constrained by language and conversations. So an LLM is the right sort of device you'd expect to achieve general intelligence.
There are arguably non-linguistic forms of intelligence, such as visual intelligence. But those also can operate on written symbols (e.g. the stream of bits from an image file).
The other relevant question is why does Gary Macus always seem so angry? It's draining reading one of his posts.