Of course it’s important to remember that the ability of an LLM to answer an obscure riddle like that has nothing to do with its reasoning abilities, but rather depends on whether the answer was included in its training dataset.
I would like to reiterate that both Claude and GPT answered correctly. It was just bizarre how Claude got a initial, minor detail incorrect, but reasoned enough to get the more difficult answer correct.
Whereas, a human of average intelligence, in possession of a dictionary or perhaps just a list of animals, could reason their way through getting the correct answer in finite time. They could probably get it without the list if they really wanted to.
Is it time to update the laws? [1]