Nassim Taleb on why IQ is baloney | Hacker News Reader

Nassim Taleb on why IQ is baloney

twitter.com

50 pointsl33tbro46 comments

Think it should be kept in mind that what Taleb is arguing here is the limited usefulness of IQ in predicting success above a certain IQ level, let’s say the mean. He 100% acknowledges its usefulness for predicting success of lower IQ persons.

He mainly just wants to point out that any other measure that was as bad at predicting higher levels of performance as IQ is would not be held in nearly as high regard. IQ therefore is in a vulnerable position as a hallowed metric, which he detects and proceeds to attack, hence the Twitter thread.

Think he could have done it with less bluster and fluster, but that’s all he’s saying at the end of the day. Not nearly as controversial as some are making it out to be

In my view the interesting part of IQ is not merely in its ability to predict human performance in areas where we expect intelligence to matter, but also in its underlying claim that there is even such a thing as a general intelligence (g), as opposed to bundles of domain-specific abilities.

Proponents of g don't claim that it's an alternative to domain-specific abilities. They claim that it exists in addition to domain-specific abilities, and is what explains the correlation between performance across domains. That is, people who do well on mathematical tests also tend do well on tests of verbal ability. The factor analytic model that yields g attempts to explain this by viewing mathematical ability as the combination of two factors: a domain-specific mathematical intelligence and a domain-agnostic general intelligence. A subject's performance on verbal tests would similarly be modeled as the product of their g and of a cognitive ability specific to verbal tasks. That a subject's performance on one type of test is partially predictive of their performance on the other type is then (according to this model) explained by the two cases sharing a single partial cause, g.

As another poster mentioned, don’t think that g is necessarily in opposition to bundles of domain-specific abilities.

I see g as an attempt at dimensionalality reduction - can the essence of what is admittedly a complex phenomenon be boiled down to a single metric which, while clearly not complete, is at least directionally correct / helpful.

g in some sense could simply be a low dimensional projection of these more complex bundles.

Just theorizing here, not an expert

It is exactly that: think of it as the first principal component, of some group of tests.

Obviously this always exists. The argument is really about how big the 2nd component is -- if it were comparable, then talking about the 1st alone would be very misleading. And for a typical bundle of school-like topics (like math/english/biology exams) it's quite a long way down: IIRC a factor of 3 or 4?

If the bundle of subjects is different, then the size and meaning of the components will vary. In the tests the army does to slot recruits into all sorts of roles, I think manual dexterity is one of the things they care about which is almost uncorrelated with g. Eyesight is I think one of the only things negatively correlated with g (and it's thought to be nurture: bookish kids spend too little time outdoors).

Was IQ ever meant to be a measure of financial success? Or rather, did anybody, at any time, ever think that the “smarter” people were the more financially successful? I know I never thought that or observed that anybody else appeared to think that. Being “smart” meant that you were (under the just-world hypothesis) guaranteed a certain level of comfort: you’d end up with an office job where you say all day in an air conditioned office rather than breaking your back under the hot sun, but the super-successful types were either (depending on your biases) the ruthlessly selfish or the risk-takers.

Have you ever read any of his books? Bluster and Fluster is his thing. I wanted to put down "The Anti-fragile" until I figured out that his persona is kind of like his advice on prediction. A kind of caustic sorting hat. Now it's sort of his brand and it works fantastically for clickbait.

...wasn't that known for years outside the tiny bubble of psychometricians? They like to go on about how it's the "best predictor" for whatever, forgetting that the absolute prediction itself is pretty lousy even when at its "best" and most of it comes from obvious tail-end cases (people with IQ below 80). There's also evidence from neuroscience [1] that's what's being measured is an higher order artifact from multiple other measurements. That's the thing: as soon as you go from the abstract, cognition-as-a-black-box mental model to an actual physical one with a model of actual brain regions, IQ falls short.

Which shouldn't be surprising given how futile it should sound to try and encompass the entirety of human abstract thinking (without even being able to give a satisfying, consensus making definition of intelligence) into a single number. Imagine if people came up with a Health Quotient built on very basic medical exams and went on about how that number is the best "predictor" of your lifespan... doesn't mean the number isn't damn useless.

[1] https://www.ncbi.nlm.nih.gov/m/pubmed/23259956/

> most of it comes from obvious tail-end cases (people with IQ below 80)

That is not a tail-end case, it is a huge number of people, about 28 million in the U.S. alone.

Wishful thinking is not a substitute for science, and IQ is (as far as we can tell) a good predictor of general cognitive ability, which is in turn a good predictor of a great number of things. If you think you have something better, we're all ears.

I'm not sure how what you're saying contradicts my point (apart from vague accusations of 'wishful thinking'). What Taleb is arguing, what I'm arguing, what the original designer of IQ was arguing and what seems to be common knowledge among everyone I know in neuroscience or medicine is that IQ is a very good predictor of life outcomes if you are mentally impaired to begin with. When you venture into the realm of >100 IQ, it loses a lot of its predictive power, which is why (where I live at least) it is mostly administered to people, mostly children, suspected of needing 'special requirements'. In that sense, it is a very good 'unintelligence quotient' but not much beyond that. It is disingenuous to argue for its general predictive power when what most people have in mind is definitely not disadvantaged children when arguing about it.

> When you venture into the realm of >100 IQ, it loses a lot of its predictive power, which is why (where I live at least) it is mostly administered to people, mostly children, suspected of needing 'special requirements'. In that sense, it is a very good 'unintelligence quotient' but not much beyond that.

According to whom exactly? There is a very large body of work predicting chess outcomes, productivity in maths, etc. in the upper range of the scale. Maybe it is not as proportionally meaningful (i.e. in most cases people are not running up against a limitation in general cognitive ability), but that doesn't mean it's meaningless, it means you're trying to predict the wrong thing with it.

I think a lot of this rhetoric stems from insecurity. I don't know my IQ, I don't personally care, but if somebody produces a study with me included then IQ (or similar measures) could yield good results.

It is not useful for all questions, nor is it useless for all questions; it is a simple measure of a specific stable characteristic of the mind, which is correlated to other characteristics which are harder to measure or predict any other way. IQ is not intended to judge your overall worth as a human being, it is just there to estimate your general cognitive ability. The fact that general cognitive ability is not sufficient on its own for you to succeed is immaterial to its utility as a measure.

You know, your arguments wouldn't sound so much in bad faith if you stopped implying IQ detractors were insecure or guilty of wishful thinking, as opposed to simply skeptical of the science behind it, especially in a field that's not exactly well known for the solidity, reproducibility and falsifiability of most of its findings.

All that said, what you said still doesn't contradict Taleb. Being able to (badly) predict a narrow range of outcomes like chess makes IQ a lousy indicator with respect to the ambition of psychometricians to elevate it as the best representative of general intelligence. Or to put it into other terms, if the best you can come up with is that, better drop the whole thing and move on to what actual hard sciences can show us (see the GGGP paper).

Disclaimer: Former Mensa member

From personal experience it appears that high IQ is a horrible measure of success. On the other hand when you combine high IQ with discipline you can come very far.

With all that being said I wish Nassim would provide some actual data instead of anecdotes.

The problem with discussing IQ is that people like to focus on the performance of very high IQ individuals where it's predictive power is weakest.

But it's very strong in other areas.

For instance say you have two groups of woodworkers, one group has an average IQ of 90 and another group has an average IQ of 100.

It's a smart bet that the 100 group has more (total) fingers.

"What can we do to help people with 90 IQs live dignified lives?" is a very important question that isn't discussed enough.

It's true that some tests give little information about the top end of the range -- if your goal is to see whether Joe is smart enough to be a GI or not, then you're wasting everyone's time if you try to separate the top 0.1% from the top 1%, so you don't.

But it's not true that measurable difference stop mattering. I'm lazy to find exact links but the big study to look up here is SMPY, https://my.vanderbilt.edu/smpy/ .

Part of the reason people are wrong about this is that, in many circumstances, they are used to observing a group which has already been selected on some finer measure. The sporting analogy is height in basketball, which is enormously beneficial... some large fraction of the world's 7-ft-plus population is in the NBA. But among NBA players, height does not predict performance very well... because to get hired, they were judged on a much finer metric of actually playing basketball. And the shorter players are selected from a vastly larger pool, and are extreme outliers in whatever other, non-height, talents are helpful.

> For instance say you have two groups of woodworkers, one group has an average IQ of 90 and another group has an average IQ of 100.

> It's a smart bet that the 100 group has more (total) fingers.

Not without knowing the size of each group.

More fingers per capita, sure.

Even if intelligence doesn't correlate with not chopping your fingers off with a chisel, it's well established that IQ is a bell curve with an average of approximately 100, so there'll be more IQ 100 people than IQ 90.

By definition it's a bell curve centered at 100, width 15.

But what was clearly meant was that IQ 90 woodworkers would be more accident-prone than IQ 100 ones (on the same tasks etc). And while I'm sure nobody has done the exact study of counting carpenters' fingers, it would be extremely surprising if this were not true. Armies, especially conscript armies, have mountains of excellent data on pretty similar things, and it's good data in that they assign people to tasks, and get detailed reports of every subsequent screw-up.

I have a strong hunch that for reasonably smart people, IQ tests are gameable the same way academic tests are. I mean, if you test someone "out of the blue" you get some measure on how much they know. If they prepare, the more they do that the higher the score.

First time I competed for admission at one of the best universities in my country, I failed. For the next year I kept preparing myself, 8 hours per day math and physics. Next year I got admitted first.

Am I smart or am I stupid? Depends how and what you measure.

I took an IQ test once, when I was 17. It was a sunny day in February, I was well rested, had nice breakfast and even managed to get a sneaky toke while my mother was in the shower. I prepared by singing to the radio in the car. It took a few hours and I got the results on another day, so it was a "real IQ test", though I'd have to look up the exact type. But it doesn't matter anyway. I scored 142 that day, and there is no way I'll ever take an intelligence test again. It made me kind of an asshole for a while, too, I found it hilarious to respond to stuff like "it's cold outside, don't you want to take a jacket" with "I have an IQ of 142, don't you think I'd know when I need to take a jacket?". But even then I knew I was feeling really good that day, and answered quickly, and what I didn't know I maybe sometimes guessed correctly. I didn't take it serious serious.

But it also wasn't just a joke to me, either. When I as a little kid, I was reading a comic with the neighbour's kid who was 2 years older, and I remember coming to the end of a page and waiting for him to turn to the next one any moment, and being surprised at how long it took for him to do that. I was excited for a second becuse I thought "I read faster than this guy and I only just learned it!", then I scolded myself and thought "no, he's probably just taking time to actually look at the images in detail". Knowing that there's at least some fuzzy objective-ish substantation for actually being above average in some areas offered some closure to that kind of stuff, and from then on out allowed me to not have to prove myself when someone who I felt wasn't that bright insulted my intelligence. But it's good I didn't take a test with such a result at age 12, that would have been a social disaster I'm sure.

Ultimately, I didn't make myself, nobody made themselves, so at best intelligence still only gift, and even more importantly a responsibility. Honesty and kindness are much more valuable in a person than any amount of intelligence, IMO. Intelligence often just allows us to deceive ourselves and others better, and I think many of the major problems in the world we ascribe to stupidity are in large part so big due to intelligent people either not pulling their weight, or actively exploiting and molding people more simple than them.

> Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius - and a lot of courage - to move in the opposite direction.

-- Albert Einstein, who had a very high IQ, so.

Screw intelligence, let's strive to be wise and kind. Intelligence is just a bonus, a nice to have, like being a bit more agile, or like a computer having more RAM, less latency etc. -- but it doesn't make a bad application something entirely different.

most sources suggest that Schumacher said this quote, not Einstein.

http://nanoscale.blogspot.com/2007/03/quote-verification.htm...

https://books.google.de/books?id=oA0IAQAAIAAJ&q="more+violen...

Thanks, and from that snippet I can tell he probably also said many other things that are true and still unheeded.

We will perish sooner than those things will stop being true.

Another comment mentioned something called 'progressive matrices', these tests are not based on any crystalized (studyable) knowledge.

You could practice solving the sort of visiospatial puzzles that they pose but then you'd ruin your results. People who use these tests are expected to have not had much/any experience with solving the types of puzzles it presents.

"You could practice solving the sort of visiospatial puzzles that they pose but then you'd ruin your results."

You are contradicting yourself. First you claim it is not based on studyable knowledge, then you claim studying for the tests ruins the results.

Some people solve similar puzzles during their life and get practice for the tests, others do not.

I've always thought the right analogy for intelligence is beauty. Could you come up with a test to measure beauty? Sure. Is beauty heritable? Sure. Is it a product of environment? Sure.

But the real deal with beauty is that you can look at any measurement of it and poke holes in it. There's never going to be some objective measurement of beauty. If you find someone saying person A. is 112 on my beauty scale and person B. is a 94 then the best response is a good laugh.

And this despite the fact that we would all agree that George Cloony is more beautiful that I am but quantifying that?

Beauty is in the eye of the beholder but some things are crudely objective.

Symmetry for one thing.

Visible signs of bad health are fairly universally negative.

A little stronger/fitter/taller than the norm (but not too much) as well.

Nice, bright, white, straight teeth.

And these things exist in tech as well, just not quite as much as in banking or consulting.

Man there are some nasty replies in this thread.

Not that I don’t agree with the high level premise, but I hate when people dismiss a hundred years of peer reviewed research generally without doing the peer reviewed legwork to back themselves up.

Examples of fields with hundreds of years of 'peer-reviewed research':

-Alchemy

-Phrenology

-Humor theory

-Spontaneous generation theory

-Astronomical geocentrism

-Ether theory

-Graphology

-Freudian/Lacanian psychoanalysis

-Astrology

all of them were once respected fields, complete with papers, journals, professorships, exams, etc.

By the way, speaking as someone in academia, you'd be surprised at how little 'peer-reviewed' means in terms of scientific validity. There's plenty of crap in Nature and Science and everyone knows about it.

I agree that "peer-reviewed" is a pretty low bar, although a better example would be more modern fields like those recently dubbed "Grievance Studies" -- formal peer review was not common even in hard sciences until maybe 50 years ago, quite a bit later than geocentrism.

But GP is correct that this is not junk science. The basic ideas of psychometrics were pretty clear 100 years ago. They've been widely attacked since then, both casual mud-slinging (Alchemy, Astrology...) and also people who drilled into the details and tried to prove things wrong. Arguably the former strategy has had more effect than the latter.

Next up: IQ on why Nassim Taleb is baloney.

>13- For a measure to be a measure it needs to be: + UNIQUE + MONOTONIC or, at least + TRANSITIVE

I'm not sure I buy this one. Intelligence is something that is inherently constructive in nature. Your knowledge and problem solving abilities build on themselves as you grow, so I don't think it makes any sense that IQ would stay the same throughout a lifetime.

Not to mention, I can think of many measures that don't fit these criteria. Height, weight, etc.

I didn't understand this either. What would it mean for a human measurement to be transitive or monotonic? Maybe there is a good definition for these but it's not obvious to me.

I think he's using a measure-theoretic meaning of a measure. Suppose I asked you what the difference in intelligence is between person A and person B. The problems with that would start pretty rapidly.

So by "transitive" he means "countably additive"? Monotonic and transitive aren't in the definition of measure and it's still not obvious what those definitions mean in terms of measuring people.

Here’s a little corollary too: assume the population is divided into two groups, one who is healthy, and the other which has suffered at least minor brain trauma or lead poisoning or something. If the average iq of the two groups is 100, then that means the healthy group has an iq above 100 (in my anecdotal experience about 115).

Human brain is a technological marvel. We know too little about it to measure its properties accurately. IQ tests do measure "something", but it does not give us a useful picture.

Analogy: We can use an electron as a wave or as a particle, but have very little understanding of what they truly are.

He’s right, I’m afraid. The kinds of things that iq tests measure are whether you have the patience and training and whether or not you care about the kinds of problems that show up on iq tests. I’ve worked with computer programmers, and forklift operators, and I have to tell you that the forklift operators are just as smart as the computer programmers. Taleb is also correct that there are a few people who have suffered some sort of brain injury for whom iq might be a useful test; but you can tell who those people are with a casual conversation. All the rest is just training and personality.

IQ + EQ = Constant

Well, as long as we have some way for me to make myself feel superior to others and deny them jobs or university entries - I think we’re good.

Lots of hating on IQ. I have nagging thoughts: if IQ doesn't mean anything, they why is it so very, very hard for a perso to score higher on an IQ test in subsequent tests? Its at least measuring 'test achievement'?

Is it true though? There's plenty of evidence that IQ test performance can be learned (i.e. you score higher) with prior training. That's why it's often insisted that tests be taken "out of the blue" without any preparation whatsoever.

Taleb doesn't mean IQ doesn't test something real. He says this "something real" is irrelevant.

I was in the US military, and they used something called an ASVAB test to check your suitability for various jobs. It's one of the few skills tests I've encountered that seemed both reasonably accurate and useful.

I'd much rather see the results of that for a potential job hire than I would want to see their SAT scores, IQ, etc.

The ASVAB effectively is an IQ test. There's more of a breakdown than other tests, but you can calculate IQ from it.

The SAT isn't considered a proper IQ test.

Surely I am dating myself, but pre 1994 SAT tests were said to be better.

Total SAT (math and verbal scores), divided by 10,were supposed to be close to the measured iq.

FWIW my SAT divided by 10 was within two points of my mother's IQ testing from decades before.

The breakdown is what was useful. Mechanical aptitude separate from other concepts, for example.

Reply on news.ycombinator.com