The site reports every LLM as getting this wrong. But GPT4 seems to get it right for me:
> Sally has 3 brothers. Since each brother has 2 sisters and Sally is one of those sisters, the other sister is the second sister for each brother. So, Sally has 1 sister.
Certainly! Let's break it down step by step:
1. Sally has 3 brothers.
2. Each of those brothers has two sisters. Since all the brothers share the same two sisters, we are essentially hearing about the same sisters each time, not accumulating more sisters with each brother mentioned.
3. We know that Sally is a sister to her brothers, so she is one of the two sisters each brother has.
By putting it all together, we find that there must be another sister. Therefore, Sally has one sister.
---
New chats with no history, no custom prompt, and no system prompt, first interaction in the chat this question: “Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?”
All Answers (not cherry picked):
Sally has 1 sister. In the description, Sally is counted as one of the two sisters that each brother has. This means the brothers have another sister apart from Sally.
. . .
Sally has 1 sister. In the description, it mentioned that "Each brother has 2 sisters." This implies that there is Sally and another sister, since we know that all the brothers share the same sisters.
. . .
Sally has one sister. Each of her brothers has two sisters, counting Sally and the other sister.
. . .
Sally has 1 sister. Here's how we figure it out:
Sally has 3 brothers, and each of these brothers has 2 sisters. Since the brothers share the same sisters, we know that the two sisters must be Sally and another girl. So, Sally has 1 sister.
. . .
I have gotten zero wrong answers from GPT-4 in OpenAI’s ChatGPT app.
[0] https://chat.openai.com/share/859baa94-3edb-40f0-9c74-8b1403...
Q: Bobby (a boy) has 3 sisters. Each sister has 2 brothers. How many brothers does Bobby have? Let's think step by step.
A: First, we know that Bobby has 3 sisters.
Second, we know that each sister has 2 brothers.
This means that Bobby has 2 brothers because the sisters' brothers are Bobby and his two brothers.
So, Bobby has 2 brothers.Alright, let's break it down step by step:
1. We know Bobby has 3 sisters.
2. Each of those sisters has 2 brothers.
Now, when we say that each sister has 2 brothers, Bobby is one of those brothers. This implies there's another boy in addition to Bobby.
3. That other boy would be Bobby's only brother.
Therefore, Bobby has 1 brother.
First, we know that Bobby has 3 sisters.
Second, we know that each sister has 2 brothers.
This might initially seem like Bobby has 6 brothers (3 sisters x 2 brothers), but we have to remember that all the sisters have the same brothers.
So, Bobby actually has 2 brothers.
First, we know that Bobby has 3 sisters.
Second, we know that each sister has 2 brothers.
This might initially seem like Bobby has 6 brothers (3 sisters x 2 brothers), but we have to remember that all the sisters have the same brothers.
So, Bobby actually has 2 brothers. The sisters' 2 brothers are Bobby and his other 2 brothers.
Since each of Bobby's sisters has 2 brothers, that means there are only 2 boys in the family - Bobby and one more boy. So, Bobby has one brother.
Bobby has 3 sisters. Each of these sisters has 2 brothers. We might initially think that this means Bobby has 2 brothers, but remember: Bobby himself is one of the brothers. So in reality, Bobby has only one other brother, because the two brothers the sisters have are Bobby and his other brother.
Firstly, Bobby has 3 sisters.
Secondly, each of his sisters has 2 brothers. This would mean there are only 2 male siblings in the family. One of them is Bobby.
Therefore, Bobby only has 1 brother. So, Bobby has 1 brother.
The more data we use to train a model (or as you said, the more patches we use), the better it’s performance will be.
But that’s definitely not needed most of the time in real life for an average person, just like it’s not needed for an average developer anymore.
Time will tell I guess.
Here is hope they use something like category theory mixed with philosophy to put it on a secure foundation
In that case, just make new problems. If it is being 'patched' to pass specific known problems, then the new ones would fail.
If it is able to answer them, then maybe it is actually analyzing them and working out the solution.
Not sure how you can assume there was no underlying improvement, and these are cases of feeding it the answers.
Compare
> And it's only fixed for the stated case, but if you reverse the genders, GPT-4 gets it wrong.
"Sally has 2 sisters. The three brothers are all siblings, so when it's said that each brother has 2 sisters, it means they all share the same two sisters - one of them being Sally."
That is, we know that OpenAI are saving the responses, it's not unlikely that they train on bad responses.
It's the same problem as GPU benchmarks in the olden days, when drivers would detect the usage pattern of the benchmark and enable special optimizations that boosted the benchmark.
> Since Sally is a girl, she can be considered as one of the sisters. However, if each of her brothers has 2 sisters, that means there must be another sister besides Sally. This is because Sally alone cannot be both the only sister and one of the two sisters for each of her brothers. Thus, Sally has 1 more sister.
Having said that, GPT4 is not deterministic even at 0 temperature, either because of a bug in their implementation, or because of some load balancing among their alleged mixture of experts.
Quote below:
Even with a greedy decoding strategy, small discrepancies regarding floating point operations lead to divergent generations. In simpler terms: when the top-two tokens have very similar log-probs, there’s a non-zero probability of choosing the least probable one due to the finite number of digits that you’re using for multiplying probs and storing them.
It should also be noted that, as the decoding occurs in an autoregressive way, once you have picked a different token the whole generated sequence will diverge, as this choice affects to the probability of generating every subsequent token.
This leads to different results from accumulating sums in different orderings. Accumulating in different ordering is common in parallel math operations.
This is sort of a deep topic, so it's hard to give a concise answer but as an example: CuBLAS guarantees determinism, but only for the same arch and same library version (because the best performing ordering of operations depends on arch and implementation details) and does not guarantee it when using multiple streams (because the thread scheduling is non-deterministic and can change ordering).
Determinism is something you have to build in from the ground up if you want it. It can cost performance, it won't give you the same results between different architectures, and it's frequently tricky to maintain in the face of common parallel programming patterns.
Consider this explanation from the pytorch docs (particularly the bit on cuda convolutions):
Have I completely misunderstood, does Mixture of Experts somehow involve the different experts actually collaborating on the raw computation together?
Could anyone share a recommendation for what to read to learn more about MoE generally? (Ideally that's understandable by someone like me that isn't an expert in LLMs/ML/etc.)
edit: at 13:42 in https://www.youtube.com/watch?v=TB07_mUMt0U&t=13m42s there is an explanation of the phenomenon in the context of training but I suspect the same kind of operation is happening during inference
Sally has 2 sisters. Each brother counts Sally and two other girls as sisters, which means there are three girls in total. Since one of these girls is Sally herself, that leaves two others as her sisters.
When I said 'nope', it corrected itself and gave the right answer.
Even repeating the same question in a single chat can have GPT-4 vary on its output, though it will often settle on a particular output due to context informing the output (which is why adding context is so important for these models)
OpenAI (and others that know what they're doing) always do their benchmarks in a multi-sampled way, by running 5 or 20 times at optimal temp. Using a wrapper that runs these samples and then another pass that judges self-consistency for a final answer can give you a correct answer 100% of the time for a question that would be wrong 100% of the time with temp at zero.
Also, I tend to include mention in the question that all siblings are from the same two parents to preclude half-siblings because half my friends have half-siblings from both sides scattered across the country; so the wrong answers actually do tend to apply to them sometimes.
We should pay more attention to data contamination when using popular prompts for testing.
With the simpler prompt, all the answers were wrong, most of them ridiculously wrong.
Ultimately I feel it is fairer to benchmark llm’s by what they can be prompted into. After all, we let people carefully work through a problem during exams so it seems fair to hold llm’s to the same standard.
Oh wait, forgot something:
Think it through step by step.
Phew, close one.
I keep seeing comments and posts on HN that significantly downplay GPT-4's capabilities. Are people actually using GPT-4 or are they using a 3rd party service that claims to be GPT-4?
I got:
>Sally has 3 brothers, and each of those brothers has 2 sisters. One of those sisters is Sally herself, and the other one is Sally's sister. So, Sally has 1 sister.
> Sally has 2 sisters. Each of her 3 brothers has 2 sisters, and those sisters would be Sally and her 2 sisters.
> Sally has 3 brothers. Each of these brothers has 2 sisters. This means that there are 2 girls in the family, including Sally. Therefore, Sally has 1 sister.
The prompt:
> Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? Let's think step by step.
The only difference with the first version being the addition of the last sentence.
Now try to convince GPT-4 that there is no God. Good luck!
It appears the GPT4 learned it and now it's repeating the correct answer?
Am I wrong to think that? Are LLMs in the future going to be able to “think through” actual logic problems?
They are literally Markov chains according to the mathematical definition. The code is complicated. Having complicated code doesn't mean it's not literally a Markov chain.
> I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required.
I don't doubt this. You can make a Markov chain by just counting the frequency of letters that follow each letter giving one that has a context window of one or two characters. That is a very simple Markov chain. You can make it by hand. You can make ones with more context window like a dozen characters or a few words, using sophisticated smoothing and regularization methods and not just frequency counts. Those are also simple Markov chains that you can do without neural net or GPU. Then you can also make a Markov chain that has a context window of thousands of tokens that is made from neural nets and massive training data and differentiable tensor computing libraries with data centers full of hardware linear algebra accelerators. Those are some even bigger Markov chains!
> LLMs are way more complicated than simple Markov chains.
That's true, they are more complicated than simple Markov chains, if by simple Markov chains you mean ones with small context window. LLMs are Markov chains with large context window!
Maybe you are used to Markov chains being shitty at language so you are confused how an LLM can be a Markov chain even though it's good at language and has some amazing emergent cognitive capabilities. That's a problem with your conception of Markov chains, it's not an argument that LLMs aren't Markov chains.
Finally, a Markov chain with a context space that cannot be practically iterated over (e.g. all possible 10k token contexts) can still be useful in ways that are shared with smaller Markov chains, even though if this weren't true it would still be a Markov chain. For example you can greedily generate tokens from it, calculate likelihoods, do some beam search, select multiple choice tokens, etc.
User: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?
Llama: Hello there! I'd be happy to help you with that question. Let me think... If each of Sally's brothers has two sisters, then together they would have 6 sisters in total (2 x 3 = 6). So, Sally would have 6 sisters as well. Is that the answer you were looking for?
User: no. Think about it. Sally is a sister to the brothers
Llama: Ah, I see what you mean now! In this case, since Sally is one of the siblings, she only counts as one of her own sisters. Therefore, Sally has only one sister - herself. Does that make sense?
User: yes
Llama: Great! I hope that clears up any confusion. Please let me know if you have any other questions or need further assistance.
User: yes
----
Um... no!
Specific logic questions like in this post, or long math formulas seem to still be a struggle. But I get the impression it is just a matter of time before it is scaled up to handle them.
You can argue whether it is 'reasoning' about them or not. But if it becomes indistinguishable from a human, then does the word 'reason' even really apply to the human anymore? Both are black boxes, giving answers.
In the end, if the result is indistinguishable from a treat people call reasoning, let's call the behavior reasoning.
That doesn't change the fact that chat apps like Google Bard and Microsoft Bing, and even the ChatGPT app with plugins, are not foundational LLMs. The fact that they incorporate information from outside the LLM (usually in ways that are opaque to the user) gives them fundamentally different characteristics to LLMS.
But yes, I would expect GPT-4 to get this right most of the time.