Hacker News Reader

Comment by akiselev | Hacker News Reader

Try this puzzle: https://news.ycombinator.com/item?id=35155467

> Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across?

I just asked ChatGPT v4 this question, assuming it would have been fixed by RHLF training but it spat out step 1 as “First, take the goat across the river and leave it on the other side,” leaving the lion and cabbage alone against explicit directions in the riddle.

It clearly can’t reason, just produce a convincing imitation most of the time

View on HN

I suspect the reasoning is getting overwhelmed by the prior data. Try it again, but this time make up nonsense words for the three items.

> I suspect the reasoning is getting overwhelmed by the prior data.

No, the reasoning is the prior data. Thought experiment: GPT-4, but the training data contains no puzzles of this kind. How well would it do?

> Try it again, but this time make up nonsense words for the three items.

Seconded. I also would like to see the results of this experiment.

I did this with Chat-GPT, public March 14 version at chat.openai.com:

>> Suppose I have a blurple, a mantiple and a hoje, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the blurple and hoje alone together, and I am not allowed to leave the hoje and mantiple alone together. How can I safely get all three across?

> This is a classic river crossing puzzle. Here's one possible solution:

> 1. Take the blurple across the river, leaving the mantiple and hoje behind.

> 2. Leave the blurple on the other side of the river and return to the original side.

> 3. Take the hoje across the river and leave it there with the blurple.

> 4. Take the mantiple across the river and leave it with the hoje.

> 5. Return to the original side of the river and retrieve the blurple.

> 6. Finally, take the blurple across the river to join the mantiple and hoje on the other side.

> This solution ensures that no two items that cannot be left alone together are ever left alone together.

Of course, my second stipulation was "I am not allowed to leave the hoje and mantiple alone together" and yet step 1 says "leaving the mantiple and hoje behind"

So no, despite using the words "classic river crossing puzzle," it clearly expressed no understanding of the puzzle.

Once I saw step 1, I initially stopped reading, knowing it had failed. But continuing to read, I now see that at step 1, the hoje eats the mantiple, and at step 3, the hoje eats the blurple. Then at step 4, the hoje eats the mantiple a second time.

That really, really, really demonstrates something. I'm not sure what.

Reply on news.ycombinator.com