When I asked the normal "How many 'r' in strawberry" question, it gets the right answer and argues with itself until it convinces itself that its (2). It counts properly, and then says to it self continuously, that can't be right.
https://gist.github.com/IAmStoxe/1a1e010649d514a45bb86284b98...
Skynet sends Terminator to eradicate humanity, the Terminator uses this as its internal reasoning engine... "instructions unclear, dick caught in ceiling fan"
For example, IMMEDIATELY, upon it's first section of reasoning where it starts counting the letters:
> R – wait, is there another one? Let me check again. After the first R, it goes A, W, B, E, then R again, and then Y. Oh, so after E comes R, making that the second 'R', and then another R before Y? Wait, no, let me count correctly.
1. During its counting process, it repeatedly finds 3 "r"s (at positions 3, 8, and 9)
2. However, its intrinsic knowledge that "strawberry" has "two Rs" keeps overriding this direct evidence
3. This suggests there's an inherent weight given to the LLM's intrinsic knowledge that takes precedence over what it discovers through step-by-step reasoning
To me that suggests an inherent weight (unintended pun) given to its "intrinsic" knowledge, as opposed to what is presented during the reasoning.
In which of the following Incertae sedis families does the letter `a` appear the most number of times?
``` Alphasatellitidae Ampullaviridae Anelloviridae Avsunviroidae Bartogtaviriformidae Bicaudaviridae Brachygtaviriformidae Clavaviridae Fuselloviridae Globuloviridae Guttaviridae Halspiviridae Itzamnaviridae Ovaliviridae Plasmaviridae Polydnaviriformidae Portogloboviridae Pospiviroidae Rhodogtaviriformidae Spiraviridae Thaspiviridae Tolecusatellitidae ```
Please respond with the name of the family in which the letter `a` occurs most frequently
https://pastebin.com/raw/cSRBE2Zy
I used temp 0.2, top_k 20, min_p 0.07
They speak a different language that captures the same meaning, but has different units.
Somehow they need to learn that their unit of thought is not the same as our speech. So that these questions need to map to a different alphabet.
That's my two cents.
It's very easy to write a paper in the style of "it is impossible for a bee to fly" for LLMs and spelling. The incompleteness of our understanding of these systems is astonishing.
I read an explanation about why it makes sense to change doors. But no, my gut tells me there's a 50/50 chance. I scroll down, repeat...
These probabilities don't change just because you subsequently open any of the doors.
So, Monty now opens one of the other 2 doors and car isn't there, but there is still a 2/3 chance that it's behind ONE of those 2 other doors, and having eliminated one of them this means there's a 2/3 chance it's behind the other one!!
So, do you stick with your initial 1/3 chance of being right, or go with the other closed door that you NOW know (new information!) has a 2/3 chance of being right ?!
Let's call the door you initially pick A.
car initial monty stick swap
A A B A C -- or Monty picks C, and you swap to B
B A C A B
C A B A C
So, if you stick, get it right 1/3, but swap get it right 2/3.
if you get to pick one and he opens 98 of the remaining ones, obviously you would switch to the remaining one you didnt pick, since 99/100 times the winning door will be in his set.
Remember, the host always knows which is the correct door, and if you selected incorrectly on the initial choice they will ALWAYS select the correct door for the second choice.
I think the easiest way to demonstrate that this is true is to play the same game with two doors, except the host doesn't open the other door if it has the prize behind it. This makes it obvious that the act of opening the door changes the probability of winning, because if the host opens the other door, you now have 100% chance of winning if you don't switch. Similarly, if they don't open the other door, you have a 0% chance of winning, and should switch. It's the fact that the host knows and chooses that is important.
It's only once you get over that initial hurdle that the 100 door game becomes "obvious". You know from the two door example that the answer isn't 50/50, and so the only answer that makes sense is that the probability mass gets concentrated in the other door.
Maybe we need a dozen LLMs with different biases. Let them try to convince the main reasoning LLM that it’s wrong in various ways.
Or just have an LLM that is trained on some kind of critical thinking dataset where instead of focusing on facts it focuses on identifying assumptions.
I sometimes put the 4 biggest models like this to converge on an optimal solution
To me the problem is that it is posed as a one-shot question. If you were in this actual situation, how do you know that Monty is not deliberately trying to make you lose? He could, for example, have just let you open the first door you picked, revealing the goat. But he chose to ask you to switch, then maybe that is a big hint that you picked the right door the first time?
If the game is just "you will pick a door, he will reveal another door, and then you can choose to switch" then clearly the "usual" answer is correct; always switch because the only way you lost is if you guessed correctly the first time (1/3).
But if the game is "try to find the car while the host tries to make you lose" then you should never switch. His ideal behavior is that if you pick the door with the goat then he gives you the goat; if you pick the door with the car then he tries to get you to switch.
It is very likely “just flip a coin to turn it back to 50/50” but may be something statistically sophisticated.
If his objective is more subtle -- increasing suspense or entertainment value or getting a kick out of people making a self-destructive choice or just deciding whether he likes a contestant -- then I'm not sure what the metrics are or what an optimal strategy would be in those cases.
Given that his motives are opaque and given no history of games upon which to even inductively reason, I don't think you can reach any conclusion about whether switching is preferable. Given the spread of possibilities I would tend to default to 50/50 for switch/no-switch, but I don't have a formal justification for this.
It strikes me that it's both so far from getting it correct and also so close- I'm not an expert but it feels like it could be just an iteration away from being able to reason through a problem like this. Which if true is an amazing step forward.
https://gist.github.com/gsuuon/c8746333820696a35a52f2f9ee6a7...
(I doubt it has, but there ARE already cases where models know they are LLMs, and therefore make the plausible but wrong assumption that they are ChatGPT.)
"Alice has N brothers and she also has M sisters. How many sisters does Alice's brother have?"
The 7b one messed it up first try:
>Each of Alice's brothers has \(\boxed{M-1}\) sisters.
Trying again:
>Each of Alice's brothers has \(\boxed{M}\) sisters.
Also wrong. Again:
>\[ >\boxed{M + 1} >\]
Finally a right answer, took a few attempts though.
Written out here: https://news.ycombinator.com/item?id=42773282
I feel like one round of RL could potentially fix "short circuits" like these. It seems to be convinced that a particular rule isn't "allowed," when it's totally fine. Wouldn't that mean that you just have to fine tune it a bit more on its reasoning path?
If I asked you, "hey. How many Rs in strawberry?". You're going to tell me 2, because the likelihood is I am asking about the ending Rs. That's at least how I'd interpret the question without the "llm test" clouding my vision.
Same for if I asked how many gullible. I'd say "it's a double L after the u".
It's my guess this has muddled the training data.
It could be the quantized version failing?
There is now research in Large Concept Models to tackle this but I'm not literate enough to understand what that actually means...