The last part of that is the problem and why a paper like this is critical.
These systems are being pushed onto people who don't understand how they work. CEO's and other business leaders are being pushed to use AI. Average users are being shown it in Google search results. Etc etc.
People are being told it can do far more than it really is.
Meanwhile these HN comments are split between:
* Lots of people confirming what the paper itself notes (but doesn't highlight), that the most advanced models actually can solve this problem at least a significant portion of the time. (A proportion which one can pretty easily project is only likely to increase with future models.)
* Lots of people saying "this confirms LLMs can't do reasoning".
Questions I'd ask you to consider:
* Is "LLMs can't do reasoning" actually more accurate than the typical hype?
* Is a "critical understanding of how [LLMs] work" that would predict they simply cannot solve this problem actually a good understanding?
LLMs do not reason. They appear to reason by repeating the structure of reasoning in their training data. This is indistinguishable in many cases.
We arent dealing with a technology that is 99.9% right in our most common use cases, so that we need to engineer some incredibly complex problem to expose the flaw. Rather, in most cases there is some obvious flaw. It's a system that requires typically significant "prompt engineering" to provide the reasoning the system otherwise lacks.
I guess that offers an explanation: people aren't aware that via their own prompt engineering they are repairing the deficiencies of the process by manipulating its inputs to include the structured reasoning it lacks. So there's a sort of hot-reading effect at work.
Right -- we are a long way from "this is a very nuanced error" being the dominant failure.