My IQ
blog.tanyakhovanova.com
blog.tanyakhovanova.com
Regarding Mensa itself, or any claims to intellectual superiority: one of the big lessons of my life has been that being intelligent has no intrinsic value. If you are unable to manifest your intelligence in a way that can substantially (disproportionate to your effort) improve your life or the lives of others, then your mind doesn't matter. It has no impact in the world, and I am not the first person to observe that having a powerful mind is at many times a burden.
You can see that other people may be more stupid than you, forwarding simpler arguments and relying on less rigorous thinking than yours, but if they don't want to be convinced of that, then you are powerless to make them see it. They go off blissfully, and you burn with anguish. Again, if you cannot manifest your intelligence in a way that forwards your agenda, who cares?
To phrase it in a more confrontational way: if nobody is forced to contend with your mind, nobody knows it exists. Or to summarize it in a quote:
"[Intelligence] is like being a lady: if you have to tell people you are, you aren't." -- Margaret Thatcher
This clicked for me during my final year of college, when a friend of mine was taking an LSAT preparation course. During a break, he was chatting with the instructor about realistic outcomes, and said earnestly that he hoped to get a score of 168 (which IIRC is a very high score.) The instructor, a man of maybe 30, who had been out of school for several years, scoffed and replied, "Good luck, I only got 165 and I'm a genius."
And what did the genius do when not teaching LSAT prep courses? Why, he worked at Target.
Edit: expanded a little on the first couple of paragraphs.
But working in the shipping department was only what he did in order to persue his passion, which was bird watching and photography. That is where he expressed his genius, although he couldn't make a living from it, it made him happy.
Reminds me of http://en.m.wikipedia.org/wiki/Wilson_Bentley who first photographed snowflakes. His genius never gained him fortune.
A driven person who's terrified of Maths and Sciences is unlikely to become a leading Physicist, but they might well power through enough basic sums to successfully establish businesses.
An intelligent person who's not driven might never achieve anything like fame, at least in their lifetime, but they still probably enjoy their free time and passions.
Working well with others takes many forms: being able to inspire and lead are obvious ones; but also the ability to take constructive criticism from others and learn from their perspectives -- which can be hard to do when you perceive the other person to be less intelligent (and perceiving others to be less intelligent is a problem by itself).
While some of them eventually shine really bright, most fail to achieve anything significant.
But, MENSA accepts 163 as a cutoff LSAT score for entry. So if MENSA == genius, then he is a genius.
Among people who take the LSAT though, he's only 92nd percentile. He doesn't have much business teaching the test. Anyone who's at an Ivy League law school has a better score than him.
I completely agree that intelligence, on it's own, is near useless. It needs to be combined with the other elements that make someone effective. THEN, and only then, do you have the ingredients for something powerful.
Our society lionizes intelligence to the point that many people with little but intelligence smugly rest on their laurels, and accomplish less than their potential due to defects in social skills, initiative, etc.
On an hourly basis, you can earn more teaching the LSAT than as an Ivy League law grad. Mind you, I don't work 80 hour weeks, so I don't make as much on an absolute basis, but the work is more interesting, and I've branched out into LSAT books and an online LSAT course.
But mainly, I wanted to point out that the instructor is unreasonably arrogant. By comparison with his reference group, he's far from exceptional.
By implication, the Mensa cut-off is also far from 'genius'. Top 2% is 1 in 50.
The Mensa cut-off is top 2% of the general public. 98th percentile on an IQ test.
For the LSAT, they accept 163 as meeting that cut-off. Relative to other LSAT takers however, a 163 is 92nd percentile. So the top 8% of LSAT takers are qualified to enter mensa, or roughly 1 in 12.
Kafka was a clerk at an insurance company. Einstein was an assistant patent examiner. Wittgenstein was a gardener at a monastery and then a primary school teacher.
Van Gogh only ever sold one painting during his life, and that was to his brother.
There are hordes of other people widely regarded as "geniuses", who lived and died in dire poverty. Many others were cheated out of the fame and fortune they deserved by others who got the credit. And many more are probably still unrecognized.
The cold hard fact of the matter is that the world generally does not reward intelligence, "genius", or even hard work. True, some have the temperament and luck to scale the greasy pole of "success", but many others don't -- or choose to focus their energies elsewhere.
I simply provided some counterexamples.
Furthermore, white collar or not, I doubt that many would contend that Kafka or Einstein were some sorts of models of "success" if judged by their jobs, and few would judge them "geniuses" solely by looking at what they did for a living.
The fact that Wittgenstein gave away his inheritance is completely irrelevant. We were talking about jobs as marks of genius, not about inherited wealth.
However, I will grant that being born in to a wealthy family, being given a first-rate education, and being surrounded by highly accomplished individuals (as Wittgenstein was) does usually give the beneficiary a tremendous head start over most people.
I would certainly have bet on someone born in to Wittgenstein's highly privileged position making something of himself than on someone born in to poverty.
That said, plenty of people born in to tremendous wealth squander it and their lives. A couple of interesting, relatively recent documentaries on the subject are "Born Rich"[1] and "The One Percent"[2].
No, I was judging him by his accomplishments -- a measure by which your counterexamples shine, and the LSAT instructor falls down. All of the people you mention were successes in some way outside of their job, and they met my other criteria: they disproportionately advanced the state of the world.
We were talking about jobs as marks of genius ...
You were talking about that, but nobody else was.
I still agree with the main point though - IQ doesn't mean anything except another avenue of high potential. Just like being extremely physically fit and a better athlete leaves you with high potential to succeed in something physical. A high IQ gives you the potential to succeed in something mental, but you still have to put in the work and do something with it.
What made you think your LSAT instructor's accomplishments were limited to his job titles?
What do you know about this man apart from those two facts?
Kafka and Van Gogh were completely unknown and unappreciated during their lifetimes. Kafka even burned 90% of his work before he died.
There are countless other examples of people who were only discovered and appreciated after they died -- often long after they died.
Conversely, many (if not most) of those who achieved great fame and fortune during their lifetimes are largely forgotten now, or considered of minor importance.
"You were talking about [jobs as marks of genius], but nobody else was."
On the contrary, that seems to be exactly what you were doing. Your lack of knowledge or appreciation of someone's accomplishments and apparent eagerness to judge this person by the job they held is what I was responding to.
Also, I never claimed that was your main argument. I just said that that's the part of your post I was replying to.
If you want to argue that a person's job is not indicative of ther level of achievement in life, then you absolutely do have to argue inductively. But I personally do not care about that argument. It is a stupid premise, and gnosis is picking an argument when one is not being forwarded. And he is doing it in a completely fallacious manner (overgeneralization, tu quoque), against what I clearly stated was an anecdote and life lesson.
Who does that? A person's own experiences, related as such, are anecdotes, not arguments.
If you are a genius, your time is not necessarily well spent on ANY job, whether it is considered menial or not by the intellectually unwashed masses, unless it is for exercise, relaxation, sustenance, or pleasure.
I refer you to the classic essay The Outsiders http://www.cpsimoes.net/artigos/outsiders.html
'The second kind of social adaptation may be called the marginal strategy. These individuals were typically born into a lower socio-economic class, without gifted parents, gifted siblings, or gifted friends. Often they did not go to college at all, but instead went right to work immediately after high school, or even before. And although they may superficially appear to have made a good adjustment to their work and friends, neither work nor friends can completely engage their attention. They hunger for more intellectual challenge and more real companionship than their social environment can supply. So they resort to leading a double life. They compartmentalize their life into a public sphere and a private sphere. In public they go through the motions of fulfilling their social roles, whatever they are, but in private they pursue goals of their own. They are often omnivorous readers, and sometimes unusually expert amateurs in specialized subjects. The double life strategy might even be called the genius ploy, as many geniuses in history have worked at menial tasks in order to free themselves for more important work. Socrates, you will remember was a stone mason, Spinoza was a lens grinder, and even Jesus was a carpenter. The exceptionally gifted adult who works as a parking lot attendant while creating new mathematics has adopted an honored way of life and deserves respect for his courage, not criticism for failing to live up to his abilities. Those conformists who adopt the committed strategy may be pillars of their community and make the world go around, but historically, those with truly original minds have more often adopted the double life tactic. They are ones among the gifted who are most likely to make the world go forward.'
[1] - https://en.wikipedia.org/wiki/The_Outsider_(Colin_Wilson)
I take your point, but Einstein and Wittgentsein are not good examples. You list jobs they held, but those jobs don't reasonably express the success either man had in his lifetime. Einstein was also a member of Princeton's Institute for Advanced Studies from 1933 to 1955 (when he died)[1]. Wittgenstein was also awarded a PhD at Cambridge - without having done coursework or exams - on the basis of his Tractatus Logico-Philosophicus and was an idolized professor for many years there[2].
Neither man was by any stretch of the imagination an unrecognized genius - they were both absolutely recognized as huge intellects with superior achievements in their respective fields when they were alive.
[1] http://www.ias.edu/people/einstein/
[2] http://en.wikipedia.org/wiki/Ludwig_Wittgenstein#1929.E2.80....
The fact that Einstein and Wittgenstein held more prestigious positions at other points in their lives would not have spared them from an attitude like the OP demonstrated at the time they held the more menial jobs, and judging only by those menial jobs.
Who knows what heights of glory this particular Target employee will climb to in the future? History is even more full of people who once held menial jobs but then went on to achieve vast fame and fortune (and perhaps even be judged a "genius" by many).
We also don't know what this particular Target employee did before he worked at target and taught LSAT courses. Perhaps, like Wittgenstein, he'd been a lecturer at Cambridge and was awarded some prestigious degree.
Ok, I know that's stretching credibility. He probably wasn't. But my point is that we don't know his past or his future. All we know are two job titles and his boast of being a genius. Should we judge him based on just those couple of facts? Or should we withhold judgment? My vote is for the latter.
Finally, I never claimed that Einstein or Wittgenstein weren't recognized during their lifetimes, just that Kafka and Van Gogh weren't, and that many others who are generally considered "geniuses" now weren't either.
I'm not trying to nitpick, just to say that I doubt I was the only person who took the post differently than it sounds now like you meant it.
The OP was talking about one point in the Target employee's life, the point at which he was a Target employee and an LSAT teacher, and he seemed to be judging solely based on job titles. That's a very myopic, and I would contend, unfair vantage point from which to judge a person's life, contribution to humanity, or achievements... as the examples I mentioned (and many others that I didn't mention specifically) demonstrate.
> in Einstein's and Wittgenstein's case, during significant portions of their lives, though Einstein was certainly vastly more successful than any of the others during his lifetime
No, see, you're doing it again. Wittgenstein was regarded from early in his life as a genius. He was considered a very eccentric genius (that's putting it mildly), but in his own lifetime he was successful and well regarded. If anything, he held lower status jobs because he chose to run away from the world of fame and regard that he already had.[1]
[1] http://en.wikipedia.org/wiki/Ludwig_Wittgenstein#Teaching_po...
In summary, the "menial" job in and of itself should not be seen as a failing for the genius.
Gandhi never won the Nobel prize for peace, Just imagine- Gandhi!!! Although there is hardly anyone in the past century who did explicitly more for peace than he did. He not just preached, but demonstrated an entire moment for independence of India and even succeeded all on non violence.
Not that he complained about it. But the world never recognized it at the time.
Quite possibly none. Intelligence doesn't equal success, ever.
We all have plenty of examples of people who, though very smart, fared poorly in the world and then later, in their own lives or afterwards, fared much better and even became famous---because those people became famous.
It has the salient quality of being almost completely unambiguous. Out of 1000+ of LSAT questions I saw when studying for the test, I found maybe half a dozen that didn't have one unambiguously correct answer and four unambiguously incorrect answers.
Of course, it assumes a level of familiarity with formal logic and English grammar/vocabulary.
I once found an LSAT question I considered ambiguous. I sent in a complaint to the LSAC. They replied with a three page letter that convinced me I was incorrect.
Grammar, not grammer.
It's a failing of our society that genuinely high IQ people are not out there working to improve the lot of mankind. This growing trend of derision for those who acknowledge their own raw intelligence is only making it worse.
Charisma? "Emotional Intelligence"? Mental flexibility? Gumption? Who knows.
Or maybe it's a failing of those high IQ people? The genius who proudly saves the codebase at Zynga every single day is completely useless compared to any single nurse working her shift right now. Even the doctor at the top of the hospital does not necessarily have a stellar IQ. What other way is there to put it? Should we feel bad for smart people because society tricks them into financial services rather than less-paid cancer research? (No one would look down on someone who does the latter.)
No, we should feel ashamed that we've created a society where the brightest are funneled into financial services. We should feel utterly devastated that we let genius wallow in menial jobs. It is a failing of the system at every level that these people fall through the cracks and end up doing nothing with their gifts.
Raw intelligence is a resource that society should work hard to 'exploit'. Sure doctors and nurses save a lot of lives and that doesn't necessarily require genius. But compared to, say, the guy who discovered the double-helix structure of DNA, their impact on humanity doesn't even come close.
Are we speaking of different groups of people? I agree that young geniuses should be treasured and given all the freedom they need to grow. But as adults, if they take on a job in HFT and stay there, it is not falling through the cracks anymore. They're irresponsible by choice, not because a faceless society forces them to (well, in most cases).
I agree about the double-helix discovery, but does anyone really need to do an IQ test to find out whether they'll be a scientist? I don't want to say "the proof is in the pudding", because there is lot of chance in science, but "the proof is in the job" comes close IMHO :)
Granted, IQ is correlated with intelligence. But there are lots of undeniably intelligent people who do not have remarkable IQs (eg Richard Feynman would not qualify for MENSA). Conversely your average MENSA chapter is full of people with undeniably high IQs and little sign of remarkable intelligence.
(Disclaimer. Based on my GRE scores, my IQ is about 1/10,000. Based on my experiences in the real world, I believe myself to be intelligent, but nowhere near what that hypothetical IQ score claims. I've always considered MENSA membership to be a filter for people whose only biggest accomplishment is a good score on an IQ test...)
But with that said, any definition that says that Feynman wasn't a genius is obviously wrong. In every environment that he was in, he was given the genius label because he obviously was one compared to (usually quite intelligent) peers. If a test says that he is not a genius, then the test isn't capturing what common usage says that genius means.
Conversely the flip side of the point stands as well. The fact that someone is labeled a genius by a test does NOT mean that they have whatever intellectual abilities would be required to better society. There is a correlation - they are more likely to have those abilities. But not a guarantee. And the fact that we see high IQ people failing to use their brains life is not always evidence that those people's gifts are being used. The true gifts may simply not be what was recorded on the test.
The conclusion that you seem to think impossible is that someone can clearly be a genius and yet that genius would not be reflected on IQ tests.
Let me offer you two alternate hypotheses.
The first fits the criticism in the above post. Many questions on IQ tests are ambiguous - there are multiple logically possible answers, and to find the "right" one you have to figure out what line of reasoning the test maker was most likely to have followed. This is a question about conventions, and Feynman clearly was extremely unconventional. He therefore is an example of someone who was literally too creative to do well on the test.
A second possibility is that Feynman's gifts were overwhelmingly mathematical, not verbal. His graduate entrance exams stand as evidence of this, as do comments from people who have examined his work. See http://www.psychologytoday.com/blog/finding-the-next-einstei... for an example. Since an IQ test has a large verbal component, on which Feynman would have only been somewhat above average, he did not score exceptionally well.
I like the first theory better, but there is significantly more evidence for the second. But either way there is no evidence from anyone who knew him, including Feynman, that the IQ score he got is not what he should have been expected to get based on what an IQ test measures.
>A second possibility is that Feynman's gifts were overwhelmingly mathematical, not verbal.
And of course this would be the "other factors at play". There are plenty of people who are savant-like in specific areas but aren't a "genius" in the generic sense. In a mathematical/scientific setting, Feynman would appear as a genius. In other settings, he would not. There's nothing surprising about any of this that would warrant a redefinition of genius.
- When a man widely recognized as one of the top geniuses of his generation is determined by a test to not be a genius, it is not Feynman's status as a genius that should be questioned.
- "Correlated with" is very different from "the same as." See the second half of http://bentilly.blogspot.com/2010/02/what-is-intelligence.ht... for some sample calculations on exactly how meaningful the correlations measured between an IQ test and other measures of intelligence likely are. (Short summary. If there is a 0.7 correlation between IQ and "true intelligence", then people with an outstanding IQ should be expected, on average, to merely be of somewhat above average intelligence. And vice versa.) Thus the measured correlations are actually evidence that IQ tests are at best moderately effective at identifying true genius.
- The body of data that we have on IQ and intelligence is COMPLETELY CONSISTENT with the prediction that we should expect a top mathematical genius to have only a somewhat above average IQ. Feynman's relatively modest IQ is therefore not a surprise. The fact that it is not a surprise is again evidence that IQ tests are a flawed method of identifying genius.
- We actually have no real evidence that IQ, g, or other related measures are measuring anything directly meaningful at all. The arguments about this are complex, please see http://masi.cscs.lsa.umich.edu/~crshalizi/weblog/523.html for an explanation. As a side note, to the extent that by "genius" we mean something other than the weighted average of random abilities actually measured by IQ tests, we should expect that IQ tests are not that good at identifying geniuses.
Yes, I know that I just gave you a lot of complex stuff to digest. But please give me the courtesy of assuming that I have at least a passing familiarity with the subject of IQ tests, have put some thought into it, and my opinions should not be immediately disregarded without investigation.
At the very least I hope I gave you enough to cause you to reconsider whether "top 2% in IQ" effectively captures what the word "genius" means in common usage.
He is certainly the type of fellow who would intentionally misreport his score for personal/philosophical/fun reasons, or not care enough to remember the number correctly decades later.
The 125 figure might or might not be exactly correct, but it is reasonable given other data we have about him. For instance, according to James Gleick's biography, Feynman was the only person to ever get a perfect score on the math and physics graduate entrance exams to Princeton. At the same time he scored below average on the verbal and history tests. (Note that the "average" at Princeton would be well above average in the general population.)
Feynman was an amazing genius. I have the deepest respect for him. That fact notwithstanding, his genius apparently did not evenly extend to all types of intellectual abilities. If so, it is unsurprising that he would merely score very highly on an IQ test.
Loving what we are has to be worth something, or why bother with progress?
We value lots of things in life that have no intrinsic value. Art, for example. There's nothing wrong with valuing intelligence too. It's what separates us from the apes after all.
That said, I don't disagree with your overall point. I've known many intelligent people woefully unprepared to deal with everyday life, and it always seemed sad to me. Two of them ended up as alcoholics, one with gambling debts. It doesn't matter how intelligent you are if it means you can't deal with life.
I'm sure this has already been explored somewhere, but I've always wondered what it says about us as humans that we consider our lives in isolation to be meaningless, but put a bunch of meaningless people together into a society, and impacting them becomes very meaningful. What's going on here, synergy?
Thinking about this the other day, I was noticing that many people of great intelligence act as though it is a free pass to not putting forward much effort. Similarly they may neglect to pay attention to others who they regard as less intelligent than them (often their teachers or professors).
IQ, "intellectual superiority" and improving-life-for-the-better or "happiness" are generally three separate things; or at least much more separate than we like to see them as clearly correlating if not causations.
And the people you meet at Mensa spread surprisingly many/all levels of society...
Mozart drank himself to death by age 30 and was hounded by creditors. Socrates, Nicola Tesla, Oscar Wilde and many other great minds died penniless.
It saddens me the number of intellectually bright people I work with who don't grok basics like sound engineering principles. Your code is off little value if it's unmaintainable, or impossible to prove it works, regardless of how clever the implementation may be.
> Cow, hen, pig, sheep.
> The standard answer is supposed to be hen, as it is the only bird. But that is not the only possible correct answer. For example, pig is the only one whose meat is not kosher. And, look, sheep has five letters while the rest have three.
These types of questions irritate the theoretician in me as well, but to play devil's advocate a bit, are they really illegitimate questions? I would guess that your ability to correctly answer an "ambiguous" question like that measures your ability to effectively communicate with other humans, who regularly speak in ways that require the listener to resolve ambiguities that are at LEAST as severe as the farm animal question. If you think hard enough about it, sure, you can come up with a justification for any of the four answers (trivially, a cow is a cow and the other three choices are non-cows), but I would say that if you truly can't come up with the answer to that question that they "want" (barring any possible cultural reasons for not being able to do so), your intuition for pattern recognition could probably use some work.
This is not to defend IQ tests in general; I'm only arguing that ambiguous questions may measure something meaningful about your ability to learn or process information.
So it measures how good you are at not thinking outside of the box? ;)
But I would say that "thinking inside of the box" is actually an incredibly important skill in practical situations. Probably 99+% of any functioning adult's thinking is "inside-the-box" (I realize that this is sort of a nebulous/meaningless statement, sorry). Things that we don't even think about, like looking both ways before crossing the street or watching our step as we get off the bus are all examples of inside-the-box thinking. But maybe a more interesting example of inside-the-box thinking (in the context of something you might want to actually test) is communicating effectively with other humans of a similar cultural background to your own.
So to get back to the portion of the article I quoted and maybe refine my point a bit, maybe it really isn't fair to say that any answer to questions like the farm animal one is more "natural" than another, but rather only seems more natural due to the biases of our own mental models that our evolution has optimized for survival rather than to achieve any measure of fundamental truth or purity. That's perfectly okay to me, as long as the test makes no claim to measure intelligence in any "fundamental" sense, but rather in a particular sense of intelligence that arises only as a human being living on earth. It's entirely a matter of opinion whether such a sense of intelligence is of any interest or relevance, though. Personally I think it is.
- that it's koscher? (very cultural) - the number of characters in its name? - whether it is born out of an egg or womb (e.g. mammal or not)?
If you seriously can't figure this out I have to doubt your intelligence - however I wouldn't say that about all test examples.
Second of all - just try some mind trick where you go outside yourself and try to observe something objectively, say as an 'Alien' who visits Earth for the first time. Which of the listed characteristics of a hen vs. a cow is important to you?
So it measures how good you are at blending in with other people, while being the entrance exam for a society of people claiming to be vastly different than other people. Hmm.
Today we have two daughters who are both completing science PhD's funded with fellowships. I paid for their undergraduate education at Tier 1 schools but they have done the rest on their own without loans. You can bet your life that they knew what an SAT test was when they took it. All their lives we've told them that they were definitely very smart but that it just doesn't matter, that IQ is irrelevant. The only thing that matters is what you do, not how smart you are.
(I remembered this story because Mr Treese didn't care much for the idea that a "number series" problem has a "best" solution.)
Mr. Treese fixed his eyes on him, gave him that crooked grin of his and said, "No that's incorrect. My guess would be much better than yours."
Years ago in our 20s my wife and I took an IQ test. I think she got in the 150s and I scored in the upper 140s.
But I felt I had done much better. When we looked at the scores, for several answers I had results that were clearly consistent with an internal model I could build of the problem, but it just wasn't the model the tester wanted.
So in a geometric progression, I noticed that each figure had an odd number of visible vortices. In a word analogy, I noticed that all of the words except one had an Old English root. And so on. Many times there were multiple patterns of resolution. How could one be better than another?
I could certainly understand her choices, I just thought they were no worse or better than mine. The fact that IQ tests are created by people with high IQs led me to believe that what is really being measured here is some sort of academic cultural cohesion; how much we begin to solve problems in similar ways?
Not necessarily a bad idea, but not what I think of when somebody says "intelligence test"
In any case, it is quite unlikely that the test you took could reliably distinguish between scores above 145.
To be frank, I have very little time for people who claim to have overthought a test. Typically, they've thought enough to complicate the simple answer, but haven't thought enough to realise why the simple answer is correct.
Short answer: you have to be familiar with the culture of testing to do well. Anecdote: I got 95+ on the GRE entering graduate school; I took advanced subject tests in subjects I had never studied. All it took was a knowledge of how the test worked.
However, as the school based subjects have broadened out from traditional liberal arts conceptualizations to include more serious discussion of alternative conceptualizations (CompSci being one example), the IQ test may be decreasing in efficacy of prediction even with regards to academic tests. I have no evidence for this, just an opinion. The issue you ran into seems to be a result of these differing conceptualization paradigms.
At the end of the day, it's an assessment of performance in a very limited subfield of life. I try not to take it too seriously.
For instance, IQ scores are bad at predicting wealth but are the best predictor of adult poverty we have.
And the US military uses them extensively. They generally don't allow anyone in with an IQ score below 90.
They were never meant to predict genius. The point is to organize people efficiently when you're slotting them for guard duty or heavy equipment operation or officer candidate school. Someone who points out BS about non-Kosher pigs and obscure integer sequences is being clever for the sake of it. IE, "Look how smart I am."
And, as you say, the military considers about 100 million Americans dangerously too stupid for the army. They didn't do this out of arrogance.
In most cases where these tests are used, it is far more mportant to separate the above median from below median, or +2 STDDEV from median, etc, and not to pick winners in the Krelboyne Super Bowl.
Even assuming there is a total ordering of intelligence, how would a single 300-question test rank 300 million people? Why fill half the test with questions that 99% of takers will get wrong, just to differentiate among the top 1% of scorers?
Or perhaps that being smart doesn't make you an enthusiastic state thug?
http://abcnews.go.com/US/court-oks-barring-high-iqs-cops/sto...
The second issue is that intelligence is multi-dimensional yet people try to measure it on a one dimensional scale. What's worse, the dimensions of intellect are somewhat elusive as well, so you can't just say something like IQ is the sum of values on each axis. Personally, I think that people that are high achievers in any field (business, arts, athletics, leadership, politics, science) are intelligent in some way. But when you look at all those people, there's no common denominator, no discernable pattern for intelligence.
Many iq tests actually measure multiple types of intelligences then combine them.
It doesn't change that there's a correlation, but anybody using divorce rates as an argument for the effectiveness of IQ tests would have to be quite subtle.
(Link: http://www.eui.eu/Personal/Dronkers/English/echtscheidingint...)
If the IQ metric is intended to correlate success on certain types of tests with success in other contexts, then it may be reliable in certain regards - e.g. people with higher IQs do tend to hive higher incomes, as 3pt14159 pointed out below - but the actual nature of that common causality is still something that may not itself be understood sufficiently to describe it in terms of dimensionality.
An analogy for this view of intelligence and knowledge is like a car's acceleration and speed. Some cars may be able to go very fast but they may take a while to accelerate.
As others have mentioned in this thread, what obviously matters more is how you apply intelligence, knowledge and understanding.
One of the most intelligent people I know IRL has trouble with the UK driving licence theory questions because she very rapidly provides 6 correct answers, and then has to try to detangle her correct answers from the choices.
I suspect that she'd do better if she tried when she was drunk (not the practical!!) because that would filter out the overthinking.
Of course, maybe this is all post rationalisation and some people just are lousy at some stuff.
I was able to ace the National Latin Exam, my drivers' tests, and my handgun license test all without studying. Just from picking what seemed the most rational/logical in the context of a test.
I'd rather be an over-thinker. I'm terrible at a lot of things that demand pure intellectual horsepower. I'm primarily useful for just producing code, not really at higher-order thinking.
This person I mentioned - when she doesn't understand something she'll ask a question. And then she'll ask another question. And then she'll ask another question. And she'll keep going until she gets it. And because she's asking questions, and really thinking about it, she gets a deep understanding.
Me? I ask a question. Someone gives me an answer. I think to myself "ok, so I kind of get it" and then I drop it. So I have a broad but very shallow knowledge.
I have learnt to ask more questions, and to ask better questions. (Also, importantly, to use available online stuff to try to answer the questions I ask before I find IRL people.)
In cold conditions, which of the following freeze before the others?
1. Bridges
2. Overpasses
3. Shaded Areas
4. All of the above
I wasn't entirely sure what a "shaded area" was, but I figured eventually that it must be some area under shade from the sun. Next, "all of the above" didn't make sense, because how can "all of the above" freeze before all of the above? I figured that water has a higher specific heat capacity than ground, and thus in cold conditions, the heat from the water would keep bridges warmer for slightly longer. I answered Overpasses.
Apparently the answer was "all of the above". The funny thing is, when I ask this question to people, I find a pretty consistent correlation between responses: for my non-university educated friends, they typically answer D. On the other hand, my university educated friends (whose majors vary significantly) tend to answer A, B, C, or "D doesn't make sense".
The problem as I see it is that the test-makers are trying to be overly clever and not ask the question they want the answer to. If I have to "extend" the language to say something it doesn't, the test maker has failed because the test question is the only source of truth.
This is the issue I see most often with these sorts of questions.
(And the "hidden question" is "did you know that shaded areas can freeze as quickly as a bridge or an overpass?" Most people who grow up around real seasons develop an intuitive understanding that thin, isolated objects cool off very quickly, but urban types are exposed to an artificial sense of ground temperature. The point is not so much to know D is correct before taking the test, but to realise that since both A and B are correct, then C must also be correct. It could have been better-phrased, yes, but a lot of questions on written driving tests are of that sort—they can teach as much as they test.)
If all roads can be precarious, the answer is all of the above, thus solving the dilema of the test taker understanding the need to be more perceptive in colder weather. We cannot control for all variables that would make a road slick or dangerous, and therefore cannot test them all individually, but must group them together into a situation (ex: road conditions changed). If we accept that driving in and of itself is an active activity, then we should not have to distinguish between specific changes in road conditions for any change in road conditions should be enough for the driver to reevaluate their driving.
This question on the test should then be changed completely to reflect the fact that a change in road conditions necessitates a reevaluation of driving style.
I would further contend that "hidden questions", such as the one described, test one's ability to take multiple choice tests. If I know A and B, I have no need for C and there is no need for me to even look at it. In fact, if there were 40 options, "All of the Above" still has to be the correct answer, even if I have no clue what the other 37 options are.
Teaching is not the intent of a test. Unless, of course, we are assuming that the test taker did not learn anything before taking the test; In which case the intent is not testing them at all but ensuring that they have some basic skills to get by.
However I agree with your intent, the creation of questions for a test is not a trivial pursuit.
While it's evident that D is nonsensical, it's just as obvious that it is indeed the intended answer.
I expect if you are smart enough to detect it is nonsensical, you are also smart enough to deduce what you are expected to answer. I fail to understand why people nitpick themselves to death by a million cuts.
Maybe I'm not smart enough!
Of course, you can bet that the writers of the test did not use anything related to algorithmic complexity when deciding the correct answers to these questions. We can't show the exact complexity of the different hypotheses, but we can sometimes compare hypotheses. Imagine programs that generate answers according to the different hypotheses. Between the program that answers according to taxonomy, and the program that answers according to word length, which one requires a look-up table of animals and their classes?
2. Another item to factor in, beyond lookup table size, is that questions are written in English, and the medium (spelled words) should not be snuck in as part of the content, except where sometimes it is done intentionally :) Also, in classification problems like these, one should also consider not just how efficiently a solution chooses an answer from the set, but also how cleanly it isolates the cluster of items in question from the unnamed items. That is, since [cow,hen,pig,sheep] are all animals, more so than a random word is, animality should be part of the rule used to choose among them.
3. As blauwbilgorgel notes, Google's very successful solution to model picking is to slurp of everything published publicly online (and with Books, also many things published offline) and sample over the combined output of humanity. This is still biased towards written text, published text, and loquaciousness, but it's pretty good.
That just strengthens my point. The word-length hypothesis doesn't require any information about English. If we change our assumptions to say that the program is being fed the raw visual stimuli (as a human is), then the word-length hypothesis gets even stronger, since it merely involves comparing the widths of the stimuli.
But most of the information about the medium can be ignored when comparing hypotheses because it is constant across them.
Allow yourself to entertain the thought that they did. Imagine programs in our brain. Things that are easy to decode/(de)compress and require little energy/instruction are of low algorithmic complexity to them.
Start writing down farm animals in your mind. How long before you get to "hen"? You may have written down "chicken", but that would be too easy for a Mensa question, so you'd have to substitute that one :P
Wrong or obscure "answers" to a test question take more energy to produce, they are more complex, more random, less orderly. For me, "Cow" is the shortest program that gets executed first when I think of farm animals. It was likely one of the first programs for the test writer too.
"cow" "pig" "sheep" 10.800.000 results
"cow" "pig" "hen" 5.040.000 results
"cow" "hen" "sheep" 4.810.000 results
"hen" "pig" "sheep" 4.480.000 results
Cow + Pig 0.4843358722880948
Hen + Sheep 0.5328799348807779
Multiple choice is particularly easy, if you are allowed to note the result counts on queries like "Answer A + Keywords from Question" and "Answer B + Keywords from Question"
Speaking of cultural differences mentioned in the post -- You could even search locally on these terms and notice a difference in the semantic distances. If cows were to go extinct, in a 100 years, the search index would probably reflect on them being the least fit candidate for a set of three farm animals.
Following a similar line of reasoning, we could say that the vision test your optometrist administers requires the cultural context, after all if you were raised in a culture that uses cyrillic then you might not recognize some characters, and you would fail the test even if your vision was 20/20! Therefore, all vision tests are flawed, and the very concept of vision acuity is meaningless.
So, I personally think IQ tests are worthless. I don't see any benefits that come from knowing what your IQ is. The only thing they give is arrogance, and as Albert Einstein (may have) said, "The only thing more Dangerous than Ignorance is Arrogance."
I thought "It's the only one that goes around on two legs." And then: "It's the only one that reproduces via eggs." Maybe not the right answer for the "right" reason, but works...
But then, as I have gotten older, my IQ is probably half what it was when I was tested as a teen (not a candidate for Mensa, sorry) Probably because I'm more certain now that I don't know all the answers. :)
1) Hen - only one with wings / only bird / only non mammal / only egg-reproducer / only creature using 2 legs
2) Sheep - only one with more than 3 letters in name / renewable wool provider (eggs and milk are renewing?) Only plural without an S (sheep vs cows, hens, pigs)
3) Pig - only animal without renewing product (no milk, no eggs, no wool); only male name (cow hen and sheep are female);
4) Cow - only sacred animal
Debatable, Pigs aren't kosher or halal, so you could claim they are sacred. I'm sure there are people who worship sheep or hens.
Surprised nobody mentioned that so far. That was my first go-to.
Now if you score above 115 and then you expect the whole world to treat you like something special you miss the whole freaking point. I’ve seen people who because they score something like 130 or 140 are intentionally acting like whims or try to say something very profound and they end up speaking BS. They are very hard to work with and constantly expect to be treated differently.
Bottom line if you happen to be supersmart try to do something useful and meaningful for the rest of the society, otherwise nobody gives a flying fuck how smart you think you are.
> But it bugs me that I might not have been
> creative enough to fail their test.
But she is certainly creative enough to fail, and way more intelligent (for pretty much any meaning of the term) than the test will have indicated.
I think it is mostly about bragging rights.
However the atmosphere at Mensa varies much from area to area (sometimes a bit pretentious, but in other areas very low key, pleasant and surprisingly open). And one should not take it too seriously.
It seems to me that a true "intelligence" test would also throw some generative problems at you: i.e., test you for as many correct answers to a pattern or sequence problem as you could come up with. I wouldn't advocate for generative problems to the absolute exclusion of deductive problems -- but the reverse is currently the case on most IQ tests, and it shouldn't be.
As for the broader point: there's been a scientific debate, for about as long as IQ tests have existed, over exactly what IQ tests are measuring, and exactly what IQ means. Most standard IQ tests are now regarded as measuring "g" ("general factor," sometimes called "general intelligence"). It may indeed be the most highly significant factor in intelligence, but even if it is (which, again, is debatable), it is just one factor of many. There are other factors, some of which are statistically significant. So simply isolating for the strongest factor is dumbing down the entire equation.
At any rate, "g" ends up being highly correlated with our concept of intelligence, but it is by no means a perfect or complete metric. And it does tend to favor a very specific type of intelligence, given that it is a narrow band on a broader spectrum. And vice versa: it excludes many whose minds we would probably consider genius caliber.
Creativity is seeing, connecting and coming up with things other people don't. The main prerequisite is usually grunt work. We only see the results, not the messy process behind James Dyson's vacuum cleaner with thousands of prototypes, or Bob Noyce's integrated circuit. What made him so creative was his knowledge and understanding of the world of semiconductors and the ability to question things.
I strongly believe that any inventor with a similar story would score high on an IQ tests and would not manifest his creativity by making up unlikely theories. Admittedly, its a sign of intelligence to understand the question and the context in which it is being asked. The world is full of opportunities to show ones creativity. Picking one of 4 answers in a test is not one of those.
I have a high but not super-high IQ, about 138. I went to a highly IQ-selective school, and as a result have met people who are smarter. Here is the interesting part: while they can do certain kinds of things better than me (puzzles, certain kinds of math, certain kinds of intricate code, analogies etc). Yet, I have gotten far more interesting things in life done than they have.
Even more interesting, I have discovered that I have thought harder and deeper about many problems than some of the exceptional IQ people I have met. As a result, I can simplify problems that they "only" (!) know how to solve. And this simplifying skill has proved to be extremely valuable in the real world. That skill doesn't seem to be directly implied by super-high IQ alone.
Not entirely true, but the few Mensa meetings I went to were all 'social' ones (met at a restaurant), and I had nothing demonstrable in common with the people there other than having the same biological brain ability. We probably did have some things in common, but no one went out of their way to welcome me, and I didn't bother to try to make myself welcome either, so I quit going.
What I think we may have had in common is some of the same social experiences of being outsiders looking in to 'normal' societal groups. Certainly everyone's experiences are different, but often highly intelligent people will be ostracized (through their own fault or the fault of others or both), and being able to relate based on those shared experiences can be useful in some capacity. The group I went to didn't seem to be in to that.
And you believe this because?
My evidence... I've not seen any evidence to the contrary. We do see evidence that physically messing with brains causing changes in brain function. Brain damage can cause impairment to memory - short term memory is one of the elements of IQ that gets measured in tests that I've taken.
If you have evidence to the contrary that what we measure as IQ is somehow not rooted in biology, please share it.
"everything is physical / material / biological."
I don't believe in ghosts/souls/the immaterial, but this is irrelevant to social conditioning having an objective effect on IQ tests. Yes, the brain that stores information is "physical", but the information is not stored in birth.
A few years ago I had 3 IQ tests administered to me over several weeks. There wasn't any major difference between any of the scoring/ranking, despite the fact that I was taking multiple. I never got much 'better'.
There is no shortage of people who will do poorly at IQ tests because of cognitive difficulties and (sure) lack of motivation to practice. Does loading on factors like these make IQ tests less predictive of academic or career outcomes? No, both of those things also require a combination of cognitive ability and motivation.
First tests I remember, I was 9. Then ACT at... 15? Maybe it was 16. Then more IQ tests (3 different ones, IIRC) about 4 years ago. All showed me to be in the same X percentile, with a very small degree of variance between tests and over 30+ years of taking them.
And to repeat, I never spent any time learning how to 'take' a test. It was actually disappointing to get the test results and compare them to ~30 years earlier, and not see any change. I thought I'd have been 'smarter' as an adult, but I do realize it's not really testing that so much as... raw capacity perhaps? That hasn't demonstrably changed, despite all the life that's happened to me. No major health issues tho; I can certainly imagine that had I had a brain injury, my IQ test scores would be lower.
The standard answer is supposed to be hen, as it is the only bird.
I believe the standard answer is a "hen" because it's the only one that is gender-specific. In fact, it is distinctly the odd-one out because it doesn't even refer to a specific genus (or even family) of animal.
I'm all for creativity (I have the same complaint often) but there needs to be some sort of standardization in communication and society. A genius who can't communicate any of his/her ideas might as well be no genius at all.
What about cow?
Cow / bull?
~~Sow / uh, pig ? (I don't know about pig)~~
Hen / cock?
(Pseudo strikethrough! See child comment from pbiggar.)
// answers correspond to growing up in ND
Not sure why I went in that direction, but I could imagine a lot of farmers and ranchers might think along similar lines.
As DanWaterworth noted, cow is gender specific and does not even refer to a specific genus either (see: Whales).
Silly city-dweller! Cows are females.
>>One test had many rows of small pictures, and I had to choose the odd one out in each row.
So the arguments about the number of letters, the order of the letter, etc wouldn't make sense here.
Cow can refer to female elephants as well
The simplest explanation is that "IQ score" is not fundamentally different than "SAT score" or "GMAT score" and is in a lot of ways worse because there are so many different interpretations and scoring methods for IQ. You can score 150 on one test but it might be scored completely differently than a test someone else takes. In other words, there is no agreed upon and copyrighted standard for IQ.
All of this type of testing really just sucks. All we really know is that IQ testing (or SATs, etc.) measures... something. And since people need to be quantified in some way, we use these scores for people-sorting. We TOTALLY overuse these scores without understanding them, and people feel great or feel horrible about their scores because of their usage (which is unavoidable) and for what it seems to say about them (which is very little).
It is hard to not feel good or bad about your score, even knowing all of this. We WANT to be rated, we want to score well and score better than others. That's human nature - just don't forget that your score will never impact what you can and can't achieve in life.
http://www.tanyakhovanova.com/
I remembered that I had seen her blog post "Should You Date a Mathematician?"
http://blog.tanyakhovanova.com/?p=319
posted to Hacker News (and other sites I read) before. I'll read more of her more purely mathematical blog posts over the next few days. I see one I can use right away in the local classes I teach to elementary-age learners.
On the substance of the post, I'm seeing several comments that equate "genius" to "person with a high IQ score." That was indeed the old-fashioned way that Lewis Terman (1877 to 1956) labeled a person with a high IQ score as he developed the Stanford-Binet IQ test. But as Terman gained more experience, especially with the subjects in his own longitudinal study of Americans identified in childhood by high IQ scores, he didn't equate high IQ to genius, and he became more aware of the shortcomings of IQ tests. Terman and his co-author Maude Merrill wrote in 1937,
"There are, however, certain characteristics of age scores with which the reader should be familiar. For one thing, it is necessary to bear in mind that the true mental age as we have used it refers to the mental age on a particular intelligence test. A subject's mental age in this sense may not coincide with the age score he would make in tests of musical ability, mechanical ability, social adjustment, etc. A subject has, strictly speaking, a number of mental ages; we are here concerned only with that which depends on the abilities tested by the new Stanford-Binet scales."
Terman, Lewis & Merrill, Maude (1937). Measuring Intelligence: A Guide to the Administration of the New Revised Stanford-Binet Tests of Intelligence. Boston: Houghton Mifflin. p. 25. That is why the later authors Kenneth Hopkins and Julian Stanley (founder of the Study of Exceptional Talent) suggested that is better to regard IQ tests as tests of "scholastic aptitude" rather than of intelligence. They wrote
"Most authorities feel that current intelligence tests are more aptly described as 'scholastic aptitude' tests because they are so highly related to academic performance, although current use suggests that the term intelligence test is going to be with us for some time. This reservation is based not on the opinion that intelligence tests do not reflect intelligence but on the belief that there are other kinds of intelligence that are not reflected in current tests; the term intelligence is too inclusive."
Hopkins, Kenneth D. & Stanley, Julian C. (1981). Educational and Psychological Measurement and Evaluation. Englewood Cliffs, NJ: Prentice Hall. p. 364.
So on the one hand there is the acknowledged issue among experts on IQ testing that IQ scores don't tell the whole story of a test subject's mental ability. A less well known issue is the degree to which error in estimation increases in IQ scores as IQ scores are found to be above the norming sample mean. Terman and Merrill wrote,
"The reader should not lose sight of the fact that a test with even a high reliability yields scores which have an appreciable probable error. The probable error in terms of mental age is of course larger with older than with young children because of the increasing spread of mental age as we go from younger to older groups. For this reason it has been customary to express the P.E. [probable error] of a Binet score in terms of I.Q., since the spread of Binet I.Q.'s is fairly constant from age to age. However, when our correlation arrays [between Form L and Form M] were plotted for separate age groups they were all discovered to be distinctly fan-shaped. Figure 3 is typical of the arrays at every age level.
"From Figure 3 [not shown here on HN, alas] it becomes clear that the probable error of an I.Q. score is not a constant amount, but a variable which increases as I.Q. increases. It has frequently been noted in the literature that gifted subjects show greater I.Q. fluctuation than do clinical cases with low I.Q.'s . . . . we now see that this trend is inherent in the I.Q. technique itself, and might have been predicted on logical grounds."
Terman, Lewis & Merrill, Maude (1937). Measuring Intelligence: A Guide to the Administration of the New Revised Stanford-Binet Tests of Intelligence. Boston: Houghton Mifflin. p. 44
Readers of this thread who would like to follow the current scientific literature on genius (as it is now defined by mainstream psychologists) may enjoy reading the works of Dean Keith Simonton,
http://www.amazon.com/Dean-Keith-Simonton/e/B001ITRL1I/
the world's leading researcher on genius and its development. Readers curious about what IQ tests miss may enjoy reading the book What Intelligence Tests Miss: The Psychology of Rational Thought
http://www.amazon.com/What-Intelligence-Tests-Miss-Psycholog...
by Keith R. Stanovich and some of Stanovich's other recent books.
Readers who would like to read a whole lot about current research on human intelligence and related issues can find a lot of curated reading suggestions at a Wikipedia user bibliography
http://en.wikipedia.org/wiki/User:WeijiBaikeBianji/Intellige...
occasionally used for the slow, pains-taking process of updating the many Wikipedia articles on related subjects (most of which are plagued by edit-warring and badly in need of more editing).
Hence the S, A, and T in "SAT".
It's not a particularly great measure of intelligence. The fact that you can often improve your IQ by 20 points (or more) just by practicing the types of questions that turn up should be evidence of this. Did you just become massively more intelligent relative to the population in the 2 days you spent practicing the questions?
After all, don't we usually think of "smart" people as the ones that can take complicated things and make them simple or explain them in simpler terms (think Feynman, Einstein...).
...now about "cow, hen, pig, sheep" ...this is obviously a biased example by choosing 3 words of 3 letters and one of 5 and can be replaced with something that does not overlap the number of letters pattern on top of the number of legs or animal group one...
the boy saw a mouse running in his pajamas
These two statements are not congruent unless the distribution of IQs in people applying to Mensa is equal to the ditribution of IQs in the general population. I doubt that to be the case.
But this does not mean that they reject 98% of the people that take the tests. People interested in taking that test must already be, I suppose, in the top 10% of the IQ distribution (125+?), or so they think. If this is true they they do reject 98% of the applicants but only 80%. Yet I suspect they reject even less.
Any source or third-party evaluation of their acceptance rate?
The trouble with such tests is that they like you to be intelligent, but not too intelligent. So what they measure is efficiency of conformance. Is that what we should be optimizing for? I don't think so.
It seems absurd that something so obviously unintelligent would be the key to measuring intelligence.
Edit: oh, I see I'm just repeating what the OP has already brilliantly said. What a great and devastating little article.
(Or some third thing?)
The answer was - 21... "You know, first we multiply by 2 and then we start to add 3" :) . Perfect "creative thinking" test.
The IQ tests, even if they are valid are still inherently broken, just forget about them and carry on.
To think.
Not bragging at all: I'm so creative and the IQ test is so culturally unfair that I just barely made it into freakin' Mensa.
But the animals question was really poorly designed
Witness my favorite renegade intellectual, Robert Pirsig, from his Wikipedia page:
'While doing laboratory work in biochemistry, Pirsig became greatly troubled by the existence of more than one workable hypothesis to explain a given phenomenon, and, indeed, that the number of hypotheses appeared unlimited. He could not find any way to reduce the number of hypotheses--he became perplexed by the role and source of hypothesis generation within scientific practice. This led him to an awareness of a (to him) previously unarticulated limitation of science, which was something of a revelation to him. The question distracted him to the extent that he lost interest in his studies and failed to maintain good grades; he was finally expelled from the university.'
https://en.wikipedia.org/wiki/Robert_Pirsig
I score pretty well on IQ tests, but only by suppressing a constant incipient rage at many of the questions, and a constant effort to ask myself "what would be the most boring and conformist interpretation of this?" That is usually the "right" answer.
Hence, you can sort theories, and then try to devise tests to falsify each one to differenciate between them.
By making an effort to choose the possible answer that I perceive as the most boring and conformist, rather than one I might find brilliant, funny, novel, and suggestive of further even more interesting ideas, I am pursuing a strategy of "favouring theories that match every other part of observable reality", as you put it.
"My first letter is in TOAST but not in ROAST".
It's quite detailed, but his first distinction is between 'static' and 'dynamic' good. A test-writer values static good more highly than dynamic, and while this is justifiable in the circumstances, in the big picture, dynamic good is the real deal.
http://coinread.com/lila_an_inquiry_into_morals.503624.html
The essence of Dynamic Quality is that you click on the link now :)
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
d9$lLsi
"Usually when you are asked to continue a pattern the assumption is that you are supposed to choose the simplest way. But sometimes it is difficult to decide what the testers think the simplest way is. Can you replace the question mark with a number in the following sequence: 31, ?, 31, 30, 31, 30, 31, … You might say that the answer is 30 as the numbers alternate; or, you might say that the answer is 28 as these are the days of the month."
This isn't to say that Occam's Razor isn't helpful -- it often is, if for no other reason than the fact that model/theory with fewer "moving parts" is simply easier to work with than another with more (and a similar level of explanatory power). Just that simplicity sometimes can't help or that it can be a value judgment.
Briefly and inexactly: A frequentist says that a probability is the answer to a question of the form "if we repeat this situation many times, what fraction of the time will this happen?"; a Bayesian says that many other things, such as (idealized) subjective degrees of belief, behave like probabilities -- i.e., they obey the same mathematical rules -- and that anything that obeys those rules deserves to be called a probability.
There is such a thing as Bayesian inference, but its opposite isn't "frequentist inference" but something like "classical hypothesis testing". If you take the frequentist view of probability then you will reject questions like "how likely is it that this treatment works?" because either it works or it doesn't -- there's nothing for a probability to be a long-run frequency of. But of course that really is the kind of thing you want to know, so you'll look for other similar questions that do make sense, such as "If the treatment doesn't work, how likely is it that we'd get results as impressive as these?". Asking that question leads to "Neyman-Pearson hypothesis testing": form a "null hypothesis" (the treatment doesn't work; the two groups of people are equally intelligent; the roulette table is not rigged by the casino; ...), do some measurements somehow, figure out how likely it is if the null hypothesis is right that you'd get results as unfavourable to the null hypothesis as you actually did, and if it's very unlikely then you say "aha, the null hypothesis can be rejected". Here, "very unlikely" might mean probability less than 5%, or less than 1%, or whatever.
If you've ever seen an academic paper in science or economics or whatever that says things like "eating more white bread is associated (p<0.01) with being a Mahayana Buddhist", that "p<0.01" thing is that same "probability of results as unfavourable to the null hypothesis"; in this (made-up, of course) case it would be something like "probability of the chi-squared statistic being as large as we found it, if bread consumption and Mahayana Buddhism were independent".
Bayesians, on the other hand, are perfectly happy talking more directly about the probability that Mahayana Buddhists eat more white bread. However, then another difficulty arises. Obviously the experimental results on their own don't tell you that probability. (Simpler example: you know that a coin was flipped five times and came up heads every time. How likely is it that it's a cheaty coin with two heads? You'll answer that quite differently if (a) you just pulled the coin out of your own pocket or (b) some dubious character approached you in a bar and invited you to bet on his coin-flipping.) The probability after your experimental results are in is determined by two things: those results, and what you believed beforehand.
On the other hand, here's a possible advantage of the Bayesian approach: Instead of just saying how confidently you reject (if you do) the null hypothesis, you can talk about the probabilities of various different extents to which it could be violated. (Imagine two medical treatments. One is more confidently known to have some effect than the other -- but the effect the second one might have is ten times bigger. You might prefer the second treatment even though the risk that it doesn't really work is bigger.)
So the Bayesian statistician might end up saying something like this: Here's a reasonable "prior distribution" -- i.e., a reasonable assignment of probabilities to the various possibilities we're interested in, before the experimental results. Now here's what our probabilities turn into if we start there and then take account of the experimental results.
Or they might just describe the impact of the experimental results, and leave it up to individual readers to combine that with their own prior probabilities.
Here's the recipe that's at the heart of Bayesian inference: Take the prior probability for each possibility. Multiply it by the probability of getting exactly the observed results, if that possibility is the case. The result is the final probability except that the resulting probabilities won't generally add up to 1, so you have to rescale them all by whatever factor it takes to make them add up to 1.
Bayesian inference gives you a subjective imprecise answer to a question you need to answer.
Classical testing gives you an objective precise answer to a different question from the one you need to answer.
As an example, most of http://www.evanmiller.org/how-not-to-run-an-ab-test.html will look dead wrong to any informed Bayesian.
Anyway, I’m with you on the business of IQ tests. The problem is that as answers become increasingly open-ended, they are both more representative of a person’s actual intelligence, and more difficult to score objectively.
I went to high school with a classmate who used to have the Pirsig family over for dinner fairly regularly. Robert Pirsig's father Maynard Pirsig was a law professor at the University of Minnesota, where the classmate's father (and, years later, I) studied for a law degree. I think, having read only Zen and the Art of Motorcycle Maintenance among Robert Pirsig's writings, and after limited personal acquaintance with Maynard Pirsig, that Robert Pirsig takes a lawyer's "that's debatable" attitude as he approaches scientific and philosophical questions, which bogs down his quest for certainty.
AFTER EDIT: I've seen an interesting pattern of drive-by voting on this comment, but I'd like to know more about what other people who are acquainted with the writings of Maynard Pirsig (father) and Robert Pirsig (son) think about the influence of the one writer on the other.
If anyone is interested, I just discovered William James Sidis's "The Animate And The Inanimate" here:
Anyone who's intelligent would know Kosher is a cultural test so is not valid.
If you don't know how to answer the question correctly it's a lack of intelligence, not you're smarter than the test. Knowledge of the state of mind of others is what is normally defines intelligence in animals. Knowing what answer they want is part of the test. Some people just don't get it...
This is the same as psych tests that ask if you want to commit suicide. If you don't know why this question is there then the test is working. You're not smarter than the test because you think it's a stupid question, you just don't understand the test.