b) Next-token training doesn’t magically grant inner long-horizon planners..
c) Long context ≠ robust at any length. Degradation with scale remains.
Not moving goalposts, just keeping terms precise.
It's not just "long context" - you demand "infinite context" and "any length" now. Even humans don't have that. "No tools" is no longer enough - what, do you demand "no prompts" now too? Having LLMs decompose tasks and prompt each other the way humans do is suddenly a no-no?
In that sense, Yann was right.
We've had LLMs for 5 years now, and billions were put into pushing them to the limits. We are yet to discover any fundamental limitations that would prevent them from going all the way to AGI. And every time someone pops up with "LLMs can never do X", it's followed up by an example of LLMs doing X.
Not that it stops the coping. There is no amount of evidence that can't be countered by increasing the copium intake.