I think this is a good place to ask: are there any tasks left that LLMs still fail significantly more often than average humans? Not "sometimes it writes buggy code", or "it refuses to discuss ethics", but getting something wrong in a way that people on the street wouldn't.

So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that mentions its own word count"). But those feel unsatisfactory, like asking a human to decode ASCII.

I've personally seen some impressive feats from simple LLMs, like writing sensible code in a new language based only on a few surrounding lines. That shows a level of expertise beyond the common line "it's just mixing sentences from the training data".