And i think it would. I think a lot of people would ask the invigilator to see if something is wrong with the test, or maybe answer both questions, or write a short answer on the cat question too or get confused and give up.
That is the kind of question where if it were put to a test I would expect kids to start squirming, looking at each other and the teacher, right as they reach that one.
I’m not sure how big this effect is, but it would be very surprising if there is no effect and unsuspecting, and unwarned people perform the same on the “normal” and the “distractions” test. Especially if the information is phrased as a question like in your example.
I heard it from teachers that students get distracted if they add irrelevant details to word problems. This is obviously anecdotal, but the teachers who I chatted about this thought it is because people are trained through their whole education that all elements of world problems must be used. So when they add extra bits people’s minds desperately try to use it.
But the point is not that i’m right. Maybe i’m totaly wrong. The point is that if the paper want to state as a fact one way or an other they should have performed an experiment. Or cite prior research. Or avoided stating an unsubstantiated opinion about human behaviour and stick to describing the AI.
If they want to estabilish this as a fact there is a trivialy easy experiment they can conduct.
“Someone on hacker news strongly feels it is true, and is willing to argue the case with witty comments.” is not how scientific knowledge is estabilished. We either have done the experiments and have the data, or we don’t.
Humans are not reliable. For every "no human would make this kind of mistake", you can find dozens to hundreds of thousands of instances of humans making this kind of mistake.
Humans are pretty good at not making mistakes in high-reasoning scenarios. The problem is that humans make mistakes in everything pretty constantly. Like, even saying a word - people say the wrong word all the time.
So when we look at really easy tasks that can be trivially automated, like say adding 2 + 2, we say "humans are so stupid! Computer is smart!".
Because humans get 2 + 2 wrong 1% of the time, but computers always get it right.
But, as we know, this isn't how it works. Actually, humans are much smarter than computers, and it's not even close. Because intelligence is multi-dimensional. The thing is, that failure rate for humans stays pretty constant as the complexity of the task increases, to a degree. Whereas computers start failing more and more, and quickly. It's a very, VERY sharp cliff for algorithms.
LLMs take the cliff further, but they do not eliminate it.
Many students clear try to answer exams by pattern matching, and I've seen a lot of exams of students "matching" on a pattern based on one word on a question and doing something totally wrong.
For example, customer service reps tend to often vaguely match your request with a possibly or only vaguely applicable templated response.
Technically savvy customers who tend to try explain problems in detail are probably more likely to get an actually non-applicable canned response as the CS rep gets frustrated with the amount of information and will latch onto the first phrase which relates to a templated response without really considering context.
My reply’s getting a little tangential now, but I feel this is good life advice, I’ve found I’m more likely to get decent customer service if I keep my requests as short as possible.
The first sentence needs to essentially state the issue I need help with. In some cases a bulleted list of things I’ve tried helps and then I’m sure to include essential info like an account number, e.g.
I’m getting error 13508 when I try log into my account. I’ve already tried the following solutions with no success:
- Clearing my browser cache and cookies.
- Restarting my computer.
- Running all software updates.
My account number: xxx
What is the next step here?
The next step will be to walk you through clearing your browser cache and cookies.
Because the CS rep has no idea who you are, and your protestations of competency fall on deaf ears because they've dealt with 23325424 people in the last year that claimed to know what they're doing but actually didn't at all.
Their goal is to get through the script, because getting through the script is the only way to be sure that it's all been done the way it needs to be done. And if they don't run through the script, and refer you to the next level of support, and it turns out that you hadn't actually cleared your browser cache and cookies, then that's their fault and they get dinged for it.
I always approach these situations with this understanding; that the quickest way to get my problem solved is to help them work through their script. And every now and then, just occasionally, working through the script has shown up something simple and obvious that I'd totally missed despite my decades of experience.
Obviously I don't do business with that company anymore.
However, I still think any irrelevant facts would upset a number of exam takers, and claiming it "clearly" wouldn't is far too strong a claim to make without evidence.
It reminds me of Kahneman's "system 1" (fast) and "system 2" (slow) thinking. LLMs are system 1 - fast, intuitive, instinctual. Humans often think that way. But we can also break out system 2 when we choose to, and apply logic, reason, etc.
But in general I do not think these models are claiming at being good at replicating the performance of a distracted or otherwise low performing pupil. I think they should be evaluated against humans who are capable of completing word problems containing context that is not inherently necessary to the math question. The reason those tests I mentioned use these word problems is that it's a way to evaluate someone's ability to think in abstract mathematical terms about everyday situations, which obviously involve lots of unimportant information the person must choose to consider or not.
tl;dr: I think a reasonably competent high school student could answer the apple and cat question, which is absolutely a reasonable bar for an LLM to clear. If university students are failing these questions, then they have not been taught test taking skills, which should be considered a mathematical failure just as unacceptable as that of the LLM, not a mitigating similarity for the latter.
We can easily cherry pick our humans to fit any hypothesis about humans, because there are dumb humans.
The issue is that AI models which, on the surface, appear to be similar to the smarter quantile of humans in solving certain problems, become confused in ways that humans in that problem-solving class would not be.
That's obviously because the language model is not generally intelligent it's just retrieving tokens from a high-dimensional statistically fit function. The extra info injects noise into the calculation which confounds it.
Nah. You would take a large number of humans, make half of them take the test with distractions and half without distracting statements and then you would compare their results statistically. Yes there would be some dumb ones, but as long as you test on enough people they would show up in both samples rougly at the same rate.
> become confused in ways that humans in that problem-solving class would not be.
You just state the same thing others are disputing. Do you think it will suddenly become convincing if you write it down a few more times?
We don't know how.
While there may be activity going on in the brain interpretable as high-dimensional functions mapping inputs to outputs, you are not doing everything with just one fixed function evaluating static weights from a feed-forward network.
If it is like neural nets, it might be something like numerous models of different types, dynamically evolving and interacting.
For example in this response: > the brain can wield language without having to scan anywhere near hundreds of terabytes of text.
The amount of text we need to train an LLM only goes down, even 2 years ago it was showed you need less than a few millions words: https://tallinzen.net/media/papers/mueller_linzen_2023_acl.p... , in order to "acquire" english.
Such a function is not inherently mysterious due to the size alone. For instance, if we fit a billion numeric points to a polynomial curve having a billion coefficients, we would not be mystified as to how the polynomial interpolates between the points.
Be that as it may, the trained neural network function does have mysterious properties, that is true.
But that doesn't mean we don't know how it works. We invented it and produced it by training.
To say that we completely don't understand it is like saying we don't understand thermodynamic because the laws of thermodynamic don't allow us to predict the path of a particle of gas in, and so we must remain mystified as to how the gas can take on the shape of the container.
Say we train a neural network to recognize digit characters. Of course we know why it produces the answer 3 when given any one of our training images of 3: we iterated on bumping the weights until it did that. When we give it a an image of 3 not in our training set and it produces some answer (either correctly 3 or something disappointing) we are less sure. We don't exactly know the exact properties of the multi-dimensional function which encode the "threeness" of the image.
Sure; so what? It's a heck of a lot more than we know about how a person recognizes a 3, where we had no design input, and don't even know the complete details of the architecture. We don't have a complete model of just one neuron, whereas we do have a complete model of a floating-point number.
Gas in a container is a kind of brain which figures out how to mimic the shape of the container using a function of a vast number of parameters governing the motion of particles. Should we be mystified and declare that we don't understand the thermodynamic laws we came up with because they don't track the path taken by a particle of gas, and don't explain how every particle "knows" where it is supposed to be so that the gas takes on the shape of the cylinder, and has equal pressure everywhere?
We would not be surprised - we wouldnt know how the model resolve the problem. We wouldn't know if it is approximating, calculating the correct value, or memorising result. We would only know how it was built. We would be mystified in how it solved the problem.
> But that doesn't mean we don't know how it works. We invented it and produced it by training.
It is not because it was invented that we know how it works. The fallacy in your reasoning is thinking that Emergent Behavior or Properties can be trivialy explained, by knowing it's building block.
I don't know, do we even know how the brain works? Like, definitively? Because I'm pretty sure we don't.
I think the question that adds a random cat factoid at the end is going to trip up a lot fewer humans than you think. At the very least, they could attempt to tell you after the fact why they thought it was relevant.
And ignoring that, obviously we should be holding these LLMs to a higher standard than “human with extraordinary intelligence and encyclopedic knowledge that can get tripped up by a few irrelevant words in a prompt.” Like, that should _never_ happen if these tools are what they’re claimed to be.
A human would probably note it as a trick in their reply.
The way LLMs work it could bias their replies in weird ways by changing their replies in unexpected ways beyond seeing it as a trick.
Maybe I'm totally wrong about that, but they really should have tested humans too, without that context this result seems lacking.