To me, these are hallmarks of reason, and not available in LLMs, in fact probably impossible just with pattern recognition.
can the average person?
For example: You have a goat, a wolf, a cabbage and you want to cross a river...
So the supervisor algorithm will do the tree search if needed.
If I ask it a leading question that intentionally relies on a wrong solution, will it recognize that?
In the canonical example it also remarks "This is a classic river crossing puzzle" before delivering the solution.
EDIT: I tried some variations with "two wolves and a rabbit" and "three wolves and a rabbit". ChatGPT started bullshitting about its solution that supposedly was "a bit more complex". It started with taking one wolf to the other side. After I pointed out the rabbit would be eaten by the remaining two wolves it apologized for the mistake but just kept going hallucinating "correct" solutions.
It would have to think out loud though.
All I want from AGI is to demonstrate that it can solve a straightforward logic problems (puzzles, if you will), that it provably didn't see before. Or at least recognize it is being indirectly given such task. So far, evidence suggests it is not capable of that.
That's what the experiments have shown - once the unknown instance gets large enough, the reasoning of LLM breaks down. This is not the case with humans, who can, as noted elsewhere, do a tree search, form hypotheses, etc.