The King of Human Error
vanityfair.com
vanityfair.com
"A team of psychologists performed personality tests on 100 professionals, of which 30 were engineers and 70 were lawyers. Brief descriptions were written for each subject. The following is a sample of one of the resulting descriptions:
Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies, which include home carpentry, sailing, and mathematics.
What is the probability that Jack is one of the 30 engineers?"
The answer given is that the probability is exactly 30%, since there are 30 engineers out of 100. But this ignores all of the information available in the description! Imagine if the description said "Jack's hobbies include writing compilers, hacking assembly code, World of Warcraft, designing new programming languages and LARP." Would we still conclude that there's only a 30% chance of him being an engineer?
A correct answer to this question should apply Bayes' rule. With E representing 'Jack is an engineer', L representing 'Jack is a lawyer' and D representing the description, Bayes rule tells us
P(E|D) = P(E) P(D|E) / P(D)
P(L|D) = P(L) P(D|L) / P(D)
where P(A|B) is 'the probability of A given B' and P(A) is 'the probability of A without taking other information into account'. We can ignore the common factor of 1/P(D) and we know that P(E)=0.3 and P(L)=0.7, so the relative probabilities of Jack being an engineer or a lawyer, given the description of him, are
P(E|D) ~ 0.3 * P(D|E)
P(L|D) ~ 0.7 * P(D|L)
so the a priori probabilities of Jack being an engineer or a lawyer need to be weighted by the probability of seeing the description that we did, given that Jack is an engineer or a lawyer! If you claim that there is a 70% chance of seeing the description if Jack is an engineer but only 30% if Jack is a lawyer, then taking all the evidence into account, you should ascribe an equal probability to the two possibilities.
Basic probability fail by the editors of Vanity Fair, I think.
[1] http://www.vanityfair.com/business/features/2011/12/kahneman...
"In one experimental condition, subjects were told that the group from which the descriptions had been drawn consisted of 70 engineers and 30 lawyers. In another condition subjects were told that the group consisted of 30 engineers and 70 lawyerss. [..] In sharp violation of Bayes' rule, the subjects in the two conditions produced essentially the same probability judgements."
There is also a blatant error in the OP article: Answer No. 2 (Linda is a bank teller and is active in the feminist movement.) is by no means "logically impossible".
EDIT: wording
They meant that it's logically impossible for the probability of A AND B to be higher than the probability of A, since A AND B is a subset of A. She could be feminist, but the probability of being feminist is lower than the 100% probability that she's either a feminist or not.
Overall, I find it rather ironic to find such gross errors in an article that sets out to educate the reader about "human error".
Then they went around asking people the same question:
Which alternative is more probable?
(1) Linda is a bank teller.
(2) Linda is a bank teller and is active in the feminist movement.
The vast majority—roughly 85 percent—of the people they asked opted for No. 2, even though No. 2 is logically impossible. (If No. 2 is true, so is No. 1.)
They say: "If No. 2 is true, so is No. 1.", but they omit that the opposite does not hold. Perhaps they thought it was obvious enough.
I think they get it and just left that second part out. They may have worded it poorly (better would be "it is logically impossible that #2 could be the correct answer"), but it was very clear to me when reading the article that the editors meant exactly what you describe.
They didn't give conditional probabilities but of course we can, and should, use what we know about the world to make guesses at P(E|'likes math'), or at least whether that's greater than P(L|'likes math'). It surely is and you can do something similar with carpentry and an apolitical nature.
It's true that even with the assumptions I've given above you can't come up with a precise answer - none of the choices are any good - you can certainly say that "...the correct response being precisely 30 percent" is thoroughly wrong.
Now that I think about this one, I think maybe the probability really is 30%. They don't say how they selected the lawyers and engineers. Maybe they selected patent lawyers (required to have engineering degrees) and all 70 members of the Society of Women Locomotive Engineers. Maybe they selected people at random. We don't know. Since we don't know the priors, we can't apply any reasoning about any characteristics of Jack.
In fact, they didn't say how they selected Jack either. Maybe they picked him out just to humiliate quiz takers who don't know the PI is a sadist.
This is a classic case of abusing the student by proving trick questions without essential information.
Engineers try to make rational decisions but probably we don't realize the extent to which it's not really human nature to do so.
Teaser:
Amos [Tversky] and I once rigged a wheel of fortune. It was marked from 0 to 100, but we had it built so that it would stop only at 10 or 65 One of us would stand in front of a small group, spin the wheel, and ask them to write down the number on which the wheel stopped, which of course was either 10 or 65. We then asked them two questions:
Is the percentage of African nations among UN members larger or smaller than the number you just wrote?
What is your best guess of the percentage of African nations in the UN?
The spin of a wheel of fortune had nothing to do with the question and should have had no influence over the answer, but it did. “The average estimate of those who saw 10 and 65 were 25% and 45 respectively.”
You are told that the test administrator is thinking of a number between 1 and 100, and that the number is odd. What is more likely?
A. The number is greater than 50. B. The number is greater than 50 and ends with a 3, 5, 7 or 9
Answer A and you have a 50% chance of being correct. Answer B and you only have a 40% chance of being correct.
(1) Linda is a bank teller.
(2) Linda is a bank teller and is active in the feminist movement.
The vast majority—roughly 85 percent—of the people they asked opted for No. 2, even though No. 2 is logically impossible. (If No. 2 is true, so is No. 1.) The human mind is so wedded to stereotypes and so distracted by vivid descriptions that it will seize upon them, even when they defy logic, rather than upon truly relevant facts."
Umm.. no. The reason 85% of people chose option #2 is because the answers were so poorly worded. They give the impression of an unspoken assumption for answer #1 that Linda is NOT active in the feminist movement.
Rather, bank teller becomes an irrelevant side-fact, and the focus of discussion becomes whether Linda is more likely to be part of the feminist movement given her being deeply concerned with issues of discrimination and social justice.
Human language is imprecise, and comes with assumptions based on context and culture. The people who came up with this question succumbed to the false assumption that the majority of people would interpret the wording of the question the same way they did.
Because of the flawed nature of the question an the incorrect assumptions of the test givers, this question cannot be relied upon to give any measure of the "conjunction fallacy".
Notice that in the jazz playing accountant example, 92% of people answered A, E, C.
Notice now that in the gambling situation, only 65% of subjects answered 2.
92% for the "story" question, and 65% for the "hard data" question. I still look at the jazz question and read it through the cultural interpretation of A > E > C, despite it being incorrect in a strictly logical sense. That does not make my interpretation a conjunction fallacy, regardless of what the question creator intended.
This only goes to show that the conjunction fallacy has no hope of being measured accurately unless the question is completely unambiguous in subtext (meaning that it doesn't tread upon cultural aspects of language, where people fill in the blanks and read between the lines).
And even then, the dice rolling question, though devoid of cultural baggage, still might not be the best way to measure the conjunction fallacy. I know a great number of people who couldn't figure out probability to save themselves. This could have skewed the results higher or even lower, depending on what proportion of the test subjects could properly interpret probabilities. Now if this question were asked to statisticians, the results would be compelling.
I ended up not choosing an answer and reading on, as it seemed like a 'trick' type question, and indeed, I was thinking about what it had alluded to, not what the actual question was.
God I love these guys.
“Most people after they win the Nobel Prize just want to go play golf,” said Eldar Shafir, a professor of psychology at Princeton and a disciple of Amos Tversky’s. “Danny’s busy trying to disprove his own theories that led to the prize. It’s beautiful, really.”
It shows amazing intellectual honesty.
If Linda is a randomly chosen person, then clearly 1 is more probable.
However, there is another sense in which 2 is more likely to be true. For example, in the radiology department where I work, doctors ordering studies need to enter in a reason for the study. Frequently, they put something generic like "pain" to get the study approved, which may or may not be accurate. On the other hand, if I saw a history that said "1 day history of RLQ pain, nausea, and fever", I would consider that history to be more likely to be true.
So in a narrow probabilistic sense, 1 is true, but when you consider that people sometimes make stuff up or don't have the full story, you might rationally judge a more detailed story to be more believable.