I do wish the writer would stop justifying the relevance of their experiment by saying “a human would conclude that their time was being wasted long before the LLM”.

This is a fallacy.

A better analogy would be a human who has been forced to answer a series of questions at gunpoint.

At this point it becomes more obvious that the LLM is not “falling short” in some way.