Thought experiment to prove this: if you trained an LLM on every utterance of human language before the 5th century BC would you get any idea we would recognize as modern?
Longer, if they keep diluting the term and selling shit labeled as AI.
What you really want to do is compare how good the AI is at that thing compared to the _average person_ at that thing, and I would guess that generative AI outclasses the average human at almost every task that it's even moderately competent at.
People like to point out how it can't pass the bar exam or answer simple math questions or whatever, and how that _proves_ that it's not intelligent or can't use reasoning when _most people_ would also fail at the same tasks.
Almost all the Gen AI models already have super human competency if you judge it across _everything it can do_.
AI research is worse than all the social sciences combined when it comes to provocative titles/abstracts that are not supported by the actual data.