For one, it needs multiple orders of magnitude more tokens (millions? billions? more?) to start following grammar rules: watch any kid pick it up with exposure to maybe 10,000 words by the age of 2 (look how they apply the rule to exceptions and while it does not work, it shows their internalization of the rule).

Next, "how" LLMs work is evident when it barfs in gotcha questions like "if yesterday was Tuesday, what day was yesterday" (I believe all major players have overcome this one in the last 12 months, but you can always find these LLM-twisters: yes, you can trick humans too by distracting them, but these are computers which "don't pay attention"?)

And it can frequently undo an improvement it added earlier, or mess something new up that you did not call out specifically.