Hacker News Reader

Oh wow I got similar results with GPT4o. I’m surprised! It even messed up the second round in a similar way.

Presumably, because river crossing challenges are an important benchmark in satisfying AI hype, it's been aggressively trained to output solutions to them and now overfits their three-object/eight-crossing structure onto prompts that approximate the river cross prompt.

Parlor tricks are lurking everywhere, and especially in ChatGPT, because of it's high profile and high stakes.

> Parlor tricks are lurking everywhere, and especially in ChatGPT, because of it's high profile and high stakes.

The only way for LLMs to get this right is to be trained on all possible paths of all possible problems, which is not very feasible. That's not really reasoning, you may as well calculate all permutations and add it to a DB and call it a day.

I think it is more so just very common data in any dataset used in training. And thus you would expect model to produce it when relevant tokens appear.

These models store these common patterns and then repeat slight variations of them as that is natural flow of the tokens. Humans do it similarly with the same problem if they have heard it before.

Challenge is that coming up with actually novel reasoning problems is not that easy. So generating one to test models is non-trivial task.

Reply on news.ycombinator.com