Hacker News Reader

Comment by blueorange8 | Hacker News Reader

That Goat wolf cabbage problem gpt-4 can solve already

Yes, but does it because it read somewhere? Can it adapt the existing solution to a new variation? Can it solve a similar problem with different things? This is what humans do all the time.

It can solve the variation "You have a rabbit, a wolf, a haypile and you want to cross a chasm". What kinds of variations do you have in mind?

If I add a small condition that makes the solution impossible, will it recognize that? Will it recognize for your example that it's a variation? Will it still be able to solve it when it is just a subtask of a bigger input?

If I ask it a leading question that intentionally relies on a wrong solution, will it recognize that?

I can't answer your questions (yet) but I have an observation. In the left sidebar where it summarizes the conversations in its "own words" the canonical example gets tagged with "River Crossing Puzzle" while my variation with the chasm gets tagged as "Transporting animals across chasm".

In the canonical example it also remarks "This is a classic river crossing puzzle" before delivering the solution.

EDIT: I tried some variations with "two wolves and a rabbit" and "three wolves and a rabbit". ChatGPT started bullshitting about its solution that supposedly was "a bit more complex". It started with taking one wolf to the other side. After I pointed out the rabbit would be eaten by the remaining two wolves it apologized for the mistake but just kept going hallucinating "correct" solutions.

Reply on news.ycombinator.com