Hacker News Reader

Comment by FiberBundle | Hacker News Reader

I found the science exams results interesting and skimmed the paper [1]. They report an accuracy of >90% on the questions. What I found puzzling was that they have a section in the experimental results part where they test the robustness of the results using adverserial answer options, more specifically they used some simple heuristic to choose 4 additional answer options from the set of other questions which maximized 'confusion' for the model. This resulted in a drop of more than 40 percentage points in the accuracy of the model. I find this extremely puzzling, what do these models actually learn? Clearly they don't actually learn any scientific principles.

[1] https://arxiv.org/pdf/1909.01958.pdf

View on HN

I would be interested in hearing the results from humans presented with adversarial answer options. You may say that a machine learning correlations between words isn’t really learning science, but I wonder how many human students aren’t either, just pretty much learning correlations between words to pass tests...

They do give an example of a question, in which the model chose an incorrect answer in the adversarial setting:

"The condition of the air outdoors at a certain time ofday is known as (A) friction (B) light (C) force (D)weather[correct](Q) joule (R) gradient[selected](S)trench (T) add heat"

I assume this might be characteristic for other questions as well, although I don't know anything about the Regents Science Exam and whether there are multiple questions about closely related topics.

That’s a terribly worded question anyway. Of the original answers, ‘weather’ is the least worst but it’s still vague.

It is a well-worded question for its purpose. The whole point is that, of all the options given, only one is justifiable (and it does not require a tendentious stretch to justify it, either.) Even “light” (which was not chosen) only applies half the time, on average. This is a valid test of natural language understanding.

Remember when IBM went on Jeopardy? There was a question about which Egyptian pharaoh. A human with some knowledge of history might mix up Ramses and Seti, or whatever, or just not know the answer, but know that they didn't know. Watson answered "What are trousers?" with supreme confidence.

Jeopardy is fun and games and it was great for the blooper reel, but they're trying to sell this stuff to diagnose cancer and guide police efforts. Failure modes are kind of important.

Most multiple choice math problems can be completely circumvented by simply finding the digital root of the expression in the problem. I was surprised to find this to be true, even on college entrance exams.

Seriously? Can you explain the mechanism? 'cos that sounds like - well, numerology, to be honest.

The digital root of the problem expression will be the digital root of the answer, because the answer IS the problem expression. They're the same number, just with all the pieces jumbled up. Comparing against digital roots of answer choices will usually eliminate most of the wrong answers. Any remaining wrong answers tend to be obviously wrong. 2 + 2 != 13 obviously.

Can you give an example? I am intrigued, although suspect you are making a joke!

You can check the properties of digital roots on wikipedia:

https://en.wikipedia.org/wiki/Digital_root#Properties

Example from ASVAB practice math test:

(x+4)(x+4) =

A. x^2+16x+8

B. x^2+16x+16

C. x^2+8x+16

D. x^2+8x+8

Since we don't have to solve for X in this problem, we can just assume x is 1, which would make the digital root of the problem expression 7. Assuming x = 1, the digital roots of A, B, C and D are 7, 6, 7 and 8 respectively. C is the answer because its last term is the square of 4.

Perhaps I am being dumb, but I don't see what the digital root brings to this.

You can do the same thing without the digital root. substitute 1 into all the possible answers gives: A: 25 B: 33 C: 25 D: 17

Substitute 1 into the question gives 25. Therefore only A and C are possible answers and it must be C because 4*4 = 16

This is the same process as your answer but without the digital root!

Agreed. The real trick here is knowing that, since we don't need to find x, we can sub in an arbitrary x and do the comparison numerically instead of algebraically.

Of course there's a tiny chance that one of the other equations would be equal at your arbitrary x value. If so and you get two 'right' answers just try again with a different x.

It is the DeMillo-Lipton-Schwartz–Zippel lemma

If two polynomials evaluate to equal numbers with random variables values, the polynomials are almost certainly equal themselves

Ofc, x = 0 and x = 1 are hardly random

I suppose such is possible without digital roots. Let's try the same problem, but this time we'll multiply our constants by 3,571:

(x + 14284) (x + 14284)=

A: x^2 + 204032656x + 28568

B: x^2 + 204032656x + 204032656

C: x^2 + 28568x + 204032656

D: x^2 + 204032656x + 204032656

The digital root of the problem expression is 4. The digital roots of A, B, C and D are 4, 6, 4 and 3 respectively. The digital root of the square of 14284 is 1, and the digital roots of the last term of A and C are 2 and 1, respectively. The answer is C.

Of course, it is quicker just to multiply and add the last digit of the constant to find the only possible answer (or count digits to get an order of magnitude estimate, which will also show C as the only possible answer).

When a problem of this form uses big enough numbers to warrant a shortcut, digital root is almost always a suboptimal shortcut.

Yes you are the heuristics master!

Maybe is case of large numbers you can compute digital root easier than the answer.

For example, 25 * 26 = 650, digital root 2. Or take digital root of 25 and 26 first, get 7 * 8 = 56, same digital root 2.

I can't decide if I've just seen magic, or conjuring. I'm gonna have to go away and think about this.

This is true, but in many cases amounts to reducing the problem to a more time consuming problem, though one that requires less knowledge. On timed tests, this can be a big negative.

Adversiarial training means that they specifically search for answers that the network would misunderstand. If this only leads to a 40 percent loss (the network still answers correctly 50% of the time) I still consider that remarkable.

Choosing the best from 8 answers where some of them were adversarially derived should be equivalent to choosing the best from all possible answers, of which there could be tens of thousands. How would a human do in that situation?

Although the kinds of mistakes the network makes seem like a mistake you would never do (i.e. you wouldn't call the condition of air outdoors gradient), the opposite could also be true, that it would easily answer questions you would have a problem with.

I admire the way that you have reinterpreted the issue, but I don’t buy it.

For one thing, I can imagine a knowledgeable human working through ten thousand alternatives and picking the best, (if, objectively, there is one) - it would just take a long time. On the other hand, It does not seem obvious to me that current NLP systems would do better than humans on such a task, and nor is it obvious to me that one can conclude that from the fact that the machine sometimes ignores adversarial examples (if the assumption is that most of the ten thousand would be de-facto adversarial, that is a lot to not make one mistake on; that’s not so much a problem for a human with an understanding of the issue, and who might well come up with the right answer unprompted.)

This is probably all moot, however, as the most obvious way to compare the machine results to those of humans is to give both the actual tests in question, rather than substitute a dubious “equivalent” test.

That’s the thing. Machine Learning is a misnomer, the models don’t “learn” anything about the domain they operate in. It’s just statistical inference.

A dog can learn to turn left or to turn right for treats. But they don’t understand the concept of “direction”, their brain isn’t wired that way.

Machine learning models perform tricks for treats. The tricks they do get more impressive by the day. But don’t be deceived, they aren’t wired to gain knowledge.

My god if I hear this argument one more time I'm going to pop. What on earth gives you the idea that you are something more than statistical inference?

Yes, it’s widely believed that statistical inference is a part of how the brain operates. But we have barely scratched the surface in our understanding how the human brain works.

Do you honestly believe statistical inference completely explains a human’s ability to learn?

Of course.

I think that the reason your mind pops is that you are not trying to understand what the person saying that is actually meaning.

Stating that we are no more than statistical inference is no different to saying that we are no more math - really, you can build the deepest mathematical truths on statistical inference.

The thing is that the math/s.i. that ML models do is too trivial in comparison to the math that would be required to describe a human identity. It can detect faces very well, it can calculate fifths roots very quickly, it can solve discrete problems quickly, yes, but its achievements are, in comparison to actual human understanding (and purpose), more closer to those of another tool, like a hammer, than those of humans.

edit: typos/grammar

I hear what they are saying and I reject it as trivially incorrect. Of course all we are is math. What else do you think there is?

"The thing is that the math/s.i. that ML models do is too trivial in comparison to the math that would be required to describe a human identity" why do you think this? How do you know the expressive power of neural networks? How have you guaged the expressive power of biological neural networks. How have you made any of these conclusions?

I am exasperated.

You are exasperated because when I wrote "that ML do is too trivial..." you did not realize that for ML I meant "the current state of the art of ML"...

I never assumed we are more than math. Just that there is a lot to still learn about the structure in which those buildings blocks are organized in order to fully tame what is understood as human intelligence. If we could simulate all the neurons of a real adult living brain accurately enough it would be intelligent from a human perspective, and still, both the model and us would be equally far, very far, from understanding how to arrive to that point of neural organization to achieve that kind of intelligence.

For a moment I had myself convinced that your response was written by GPT-3. Part of me hopes that is the actual fact.

Because statistical inference does not create complex operating models designed to fit a subset of reality from a larger encompassing model of reality which humans base our daily decisions and life philosophies. Just because we do not deep dive into the complexities of existential questions every waking moment does not say we're not operating scaffolds which depend upon prior existential examinations. We're complex to say the least, and we require reasons - and that very idea: reason is what is not present in statistical inference. Without a prior body of human conversation, statements, and larger expressions of philosophy based upon human reason, statistical inference goes nowhere.

>reason is what is not present in statistical inference.

But why think reason isn't a kind of statistical inference? GPT-3 demonstrates a rudimentary capacity for reasoning and its "merely" a statistical inference engine.

Because it is possible to invent or imagine new things without resorting to endless random walks or blind trial and error.

Induction isn’t probabilistic. It’s the origin of all discovery and based in sussing out what’s important in patterns, selectively proposing causal mechanisms and testing those hypotheses by following priors and implications — an essential basis for original and creative thought that no purely probabilistic engine can employ.

You think a probabilistic engine can't model a human doing induction? At a certain level of capability, mere pattern imitation closes upwards.

Doesn't your brain derive knowledge through statistical inference? How else? I can imagine the AGI of the future (having perfected knowledge gathering through statistical inference) some day saying: "Humans perform tricks for treats. The tricks they do are impressive. But don’t be deceived, they aren’t wired to gain knowledge."

I didn’t claim that the way brains work didn’t include statistical inference. That’s why the dog metaphor works. Both the dog and the machine learning model use statistical inference to perform tricks.

Dogs haven’t been to the moon, however. There’s more to the brain we don’t understand.

I'm curious why you think a "difference in kind" rather than simply "difference in degree" must exist between us and dogs.

Adversarial examples probably exist for humans too, they are just less easy to find since we can't easily backprop through human judgements like we can a bunch of array multiplications and dot products.

Doesn't mean that they are not "actually" learning any more than we are.

These are "adversarial answers" - additional answers to a multi-choice test. If you know the correct answer to the test, adding more possible answers shouldn't make you change your selection.

Reply on news.ycombinator.com