Hacker News Reader

Comment by jfim | Hacker News Reader

You can see it in various tasks where the answer for a text generator would be incorrect, yet simple reasoning or understanding yields the correct answer. For example, the everything fits in the boat variant of the farmer across the river problem: https://chat.openai.com/share/7d2de1c4-1cd6-4d7d-97e4-645830...

View on HN

I love this example! I've been using a different one and GPT has been getting better at it (unconvinced they aren't training on my reporting that it gets the wrong answer. Or that I've posted this question dozens of times here and some have gotten pretty high up. But it's been a year now and we're still in stochastic phase). The question I ask is variants of "Which weighs more, a pound of feathers or a kilogram of bricks?" Similar to yours it is just a slight variant on a standard logic puzzle.

For those wondering why ask these questions, we're testing for overfitting. Yes, you can overfit even if your training and validation curves don't diverge. If the model fails these questions, they've clearly overfit. But the solution space is very large and complex, so overfitting in one regime doesn't mean it didn't underfit another.

GPT 3.5: https://chat.openai.com/share/328cc39d-7fb3-4726-92a2-29437c...

LLaMA 2 70B chat: https://hf.co/chat/r/KX5H3P2

Falcon 180B Chat: https://hf.co/chat/r/V8XxdYh

None of these models can consistently get the answer right. GPT used to actually explain to me the difference between a pound and kilogram (correctly) and then use that answer to justify why they are the same. Such an answer is very clearly a demonstration of a lack of understanding as it isn't even remotely self consistent.

LLMs are powerful and amazing technologies. But we can also critique them. Hyping up models hinders the ability to improve them because every thing (LLMs, humans, governments, whatever) has limitations and is worthy of critique. But criticism isn't saying something is garbage and too many people confuse this.

Agreed, there's definitely a lot of value in LLMs even if they don't really have an understanding of the world.

It is quite interesting to me that people latch onto the world model so much as if it is highly significant or a lack of a world model invalidates LLMs. Personally I still believe they're magical (in the proverbial sense) and amazing tools even though I also believe they are just complex stochastic machines that have very little understanding of the things they are doing. That just tells me how cool and amazing that statistics is. I honestly don't know anyone that thinks LLMs are useless things, despite knowing a large number of people that are highly critical of them. But those are very different metrics. I don't know why we can't have nuance in things, why they need to either be a gift from god or a pile of shit.

I was pleased to see GPT-4, combined with my custom ChatGPT instructions, had no problem with this: https://chat.openai.com/share/726d5c7b-761f-462e-8a6a-61904a...

You should refer to this comment I made[0], as your prompt is spoiling the problem. It is far too specific. Hacking the answer in is not a means of getting correct answers. It won't scale and won't generalize.

[0] https://news.ycombinator.com/item?id=37825566

It figures it out if you point out the difference much like a human would when faced with a riddle they didn't get.

https://chat.openai.com/share/34df3eb0-c41c-4c5b-80d2-48f5d0...

I'll be honest, the only reason I knew to read your variation of the farmer-boat problem carefully is because you set me up for success - I knew it wasn't going to be the original.

Curiously, I haven't been able to get it to one-shot the question, even by prefixing it with instructions.

Except the problem here is that you spoiled the solution. It is quite difficult to provide hints that don't spoil. Or rather, don't provide hints but rather tell it that it is wrong.

Here, I'll demonstrate it. First I'll prod it with vague responses about it just generally being wrong. Trying to leak no information to the model. We'll try to slowly add a bit more and then use your followups. Notice that the model cannot escape the overfit regime without your strong hints. You told the model specifically what to consider. You told it that it is a trick question of a trick question. This is not how a human would handle the situation. To my followups they wouldn't spit out the same answers. They'd actually likely followup with questions if they were confused. Which is a behavior I've never seen from an LLM: asking clarifying questions.

https://chat.openai.com/share/57ab9bca-326d-45cb-9257-7fb8c2...

I do not think that answer would correctly solve the original riddle. The LLM blindly just gave an answer without raising the question of which pair of objects together would be a problem if left alone.

Good catch, it actually leaves the goat with the cabbage in the answer above.

But that isn't a correct solution - it leaves the cabbage with the goat.

The correct solution is that everything fits in the boat, the farmer can just go across the river with everything at once.

Reply on news.ycombinator.com