My favorite is “what weighs more, 1 lb of feathers or 2 lb of bricks?” almost every llm ive tried says “they weigh the same”
A little more sophisticated and GPT-4 will also fail.
> The 18 lbs of bricks weigh more. The weight of an object is determined by its mass and the force of gravity acting upon it, regardless of the material it is made of. In this case, 18 lbs (pounds) is greater than 17 lbs, so the bricks weigh more than the feathers.