Hacker News Reader

Comment by tr3ntg | Hacker News Reader

Since the Maker Suite doesn't have the same prompting as something like Bard or ChatGPT, I gave Gemini Pro a bit more context:

> Consider the following user chat. Reason out loud as needed to provide a reasonable response. User message: "Julia has three brothers, each of them has two sisters. How many sisters does Julia have?"

Gemini Pro:

> Julia's brothers have nothing to do with how many sisters she has. Julia's sisters are also her brothers' sisters, so the number of sisters Julia has is the same as the number of sisters her brothers have. Julia's brothers have two sisters each, so if we add up the number of sisters for each brother, we get: 2 + 2 + 2 = 6. Therefore, Julia has 6 sisters. The answer is 6.

View on HN

Mixtral 8x7b only gets the sister question correct when I ask it to show it's work, but also shows that it's aware of the brain teaser:

https://poe.com/s/5gtLvuWT52o9LEBAJouo

GPT 4 gets it right.

GPT 3.5 gets it hilariously wrong if I change the question slightly:

User: Julia has 8 brothers, each of them has 5 sisters. How many sisters does Julia have?

Assistant:

Julia has 8 brothers, and each of them has 5 sisters. Since Julia is one of those sisters, she has 8 brothers + 1 (herself) = 9 sisters.

https://chat.openai.com/share/3a49800f-7747-44d9-90e7-c1e037...

Also on 3.5 I tried this:

User: Julia has 8 brothers, each of them has 5 sisters. How many sisters does Julia have? Show your work step by step.

Assistant: Julia has 8 brothers, and each of them has 5 sisters, including Julia herself. So, Julia is one of those 5 sisters for each of her 8 brothers.

To calculate how many sisters Julia has, you can simply multiply the number of brothers by the number of sisters each brother has:

8 brothers * 5 sisters per brother = 40 sisters

Therefore, Julia has 40 sisters in total.

Mistral 7B (mistral-7b-instruct-v0.2.Q6_K.gguf): Let's break down this problem step by step. First, we know that Julia has three brothers. Then it is stated that each of her brothers has two sisters. This means there are two sisters for each brother. Since Julia is one of the sisters for her brothers, she also counts as one of their sisters. However, when counting the number of sisters for herself, she should not be included in the count. Therefore, Julia has three brothers and two sisters (excluding herself), so she has a total of five sisters.

Isn't 6 still incorrect? Or am I dumb?

This made me laugh. It's very wrong. Unless I'm dumb, the answer we're looking for should be 1.

Sounds like the confidence of Gemini worked on you, though.

It’s way more complicated. As an example, my kid has one half sister and had one step sister. My kid referred to both of them as their sisters, but the step person didn’t acknowledge my kid as such.

Even if we err on the side of the step, take the half sister entirely out of the equation and only focus on the step relationship, one would have claimed 0 sisters and the other would have claimed 1. They were talking about each other and they were both telling their truth. But the answer is simultaneously 0 and 1.

It gets even more interesting because neither my kid nor their step sibling are cisgender. I don’t know what to call their step so just use their name and pronouns now. The step still won’t acknowledge they’re siblings and I respect that a lot.

When these smart young folks take over, the world will get really cool.

Edit - This was way more complicated to explain than I anticipated.

I tried in Bard just for comparison's sake and it got to 1 while still saying a different number. It's a bit strange, here:

Julia has two sisters!

Here's the reasoning:

1. *Julia's brothers have sisters*: Each of Julia's three brothers has two sisters. This means in total, her brothers have 3 brothers * 2 sisters/brother = 6 sisters.

2. *Those sisters are not additional*: However, it's important to remember that these sisters belong to Julia's brothers, not directly to Julia herself. Julia and her sisters are siblings, not counted among the sisters her brothers have.

3. *Therefore*: Julia has 2 sisters (herself and one other).

So, while Julia's brothers have a total of 6 sisters, only Julia and one other sister are Julia's own sisters.

The sum is incorrect and the statement of "Julia's sisters are also her brothers' sisters, so the number of sisters Julia has is the same as the number of sisters her brothers have" is also incorrect. The brothers each have one more sister than Julia, since she is one of the sisters so she only has one sister. It's just reworded but still wrong.

Yes, making it reason made it even more wrong

You're not dumb. But it's an ambiguous question as written. Presumably, we are to assume that all siblings involved are still alive, that all all full siblings (ie no half- or step-siblings), that everyone is cisgender, and that Julia is female. In which case, Julia would have one sister. Each of her three brothers would have two sisters, one of whom was Julia herself.

My 6yo can answer this correctly, making all of your stated assumptions above. Edit: Clarification - I am in no way implying that someone who was confused by the math is dumb, everyone makes silly mistakes sometimes.

I think It’s only ambiguous to a machine. To a human, even in 2023, these are likely default assumptions for a question phrased as above, unless being overly pedantic or attempting to “get around” the puzzle.

If you ask a model - or a person - “what does 1+1 equal to?”, it is also ambiguous, since we haven’t defined what “+” means, did not specify that we are dealing with natural numbers rather than eg elements in Z//2Z which could be similarly denoted.

Reply on news.ycombinator.com