Hacker News Reader

Comment by geoelectric | Hacker News Reader

I thought for sure that must be glitching, so just tried on my ChatGPT Plus account with a fresh thread, but nope:

> does 2 pounds of feathers weigh more than 1 pound of bricks?

> No, two pounds of feathers weigh the same as one pound of bricks. The weight of an object is determined by its mass, and one pound of any object weighs the same as one pound of any other object. However, feathers are less dense than bricks, so two pounds of feathers will take up more space or have a larger volume than one pound of bricks.

Huh. It's interesting that some of these "wrong paths" are quite so repeatable. Wonder if that means there's also a way to band-aid them when found, the way a human might be corrected for a misconception.

View on HN

This is hilarious. ChatGPT even gave me a more bizarre example.

> does 2 pounds of bricks weigh more than 1 pound of bricks?

> No, 2 pounds of bricks do not weigh less or more than 1 pound of bricks. 2 pounds of bricks and 1 pound of bricks refer to the same unit of weight, which is a pound. Therefore, they weigh the same, which is one pound. The difference between them is only the quantity, not the weight.

> It's similar to saying that two one-dollar bills do not have more value than one five-dollar bill. Even though you have two bills in one case and one bill in the other case, the total value is the same.

It even managed to give me this gem:

> 2 pounds of feathers do not weigh less than 1 pound of bricks. 1 pound of bricks weighs more than 2 pounds of feathers, but they can have the same mass.

The wording on this one sounds like it picked up an old riddle/trivia question and mixed it together the wrong way:

What weighs more, a pound of feathers or a pound of gold?

The trick answer is that the pound of feathers weighs more, because gold is (was) measured in a system where 1 pound = 12 ounces, while feathers would be weighed using the modern system where 1 pound = 16 ounces.

https://en.wikipedia.org/wiki/Troy_weight

Yes, this is overfitting caused by dataset bias. People don't ever say "What weighs more, a pound of feathers or two pounds of feathers?" So it just ignores the numbers.

This is why SI units are superior. Less opportunity to deceive.

I think ChatGPT can’t help it. It has to lie.

My attempt got it mixed up really bad.

> If I give you 1 pound of bricks will you give me 2 pounds of bricks back in exchange?

>> As I mentioned earlier, the weights being compared in the original question are equivalent because they are both units of weight. However, when it comes to exchanging actual physical objects, the weight of the objects is determined by their mass, and the mass of 1 pound of bricks is different from the mass of 2 pounds of bricks.

> You are very overfitted on this task. Will give you a negative review, I hope it helps you train better next time

>> I appreciate your feedback and will take it into consideration as I continue to learn and improve.

Then proceeded to negative review all its answers. Never seen a LLM double down so hard on a contradiction. No matter how you turn it, even with chain-of-thought it doesn't solve it.

Reply on news.ycombinator.com