Comment by runarberg | Hacker News Reader

Comment by runarberg | Hacker News Reader

Parent Full thread

From OP:

> The mere fact that the results of these types of tests on large population groups tend to follow a normal distribution is alone good evidence that they're indicative, as in most things diverse large population groups tend to fall on a normal distribution.

The claim here is that OP is wrong. Firstly OP is wrong because normal distribution is not a sufficient indicator for some underlying phenomenon. Or as yxwvut puts it:

> [...] normal distribution implying some fundamental truth, the same can be said of an iterated guessing game (eg: take the sum of your correct guesses over 100 coin-flips - the population's scores on that game will be normally distributed).

And then secondly because intelligence doesn’t even follow a normal curve (only IQ does because it is transformed to).

What matter here is not the true distribution of intelligence. I’m not even sure there is one. What matters is that proponents of general intelligence use the normal distribution of IQ as a justification. By arguing that intelligence does not fall on normal distribution but rather on a skewed distribution, I’m simply pulling the rug away from this argument.

I admit this is not a strong argument against the existence of general intelligence, since if there exists such a thing, then there is nothing that says such a thing couldn’t exist on a skewed distribution (except psychometricians and proponents of IQ which only accept standard bell curve [I wonder why]). However in this subthread we’re only trying to correct OP’s claim that a normal distribution is indicative for general intelligence.

Thanks for explaining. It sounds like we agree that "OP is wrong because normal distribution is not a sufficient indicator for some underlying phenomenon". And that we're not sure that there's a "true distribution" of intelligence. But I don't know how you can then argue that "intelligence does not fall on normal distribution but rather on a skewed distribution". Again, how could we choose between calling skewed thing vs its log transform "intelligence"?

Ahh, I see the confusion now.

I actually don’t believe in general intelligence, and I don’t believe intelligence is this one thing that you can measure. So I see how it can be confusing when I go ahead and say: “Intelligence has a skewed distribution.”

But when I’m talking about “intelligence” in this context I’m using working within the assumptions of the psychometric. In other words, I forgo my priors and argue on the terms of my opponents. I.e. given that they are right about the existence of general intelligence and that IQ tests are a good and accurate way to measure intelligence, they are still not correct about it being normally distributed.

To be precise. When I use the term “intelligence” in this context I’m basically using the same operationally defined term as psychometricians. That is: Intelligence is whatever the IQ tests are measuring.

Now as for which distribution to pick for this thing that IQ tests are measuring. Why pick a skewed distribution instead of a normal one? Psychometricians pick the bell curve because they believe it gives this term some validity. I would pick a skewed distribution because that fits our intuition and fits the underlying data. I would call the former justification pseudo-science as you have no reason other then your prior assumptions to do that. Then why do the latter? I would say the only reason to do that is to prove the people doing the former wrong.

Thanks for elaborating. But I still don't understand in what sense a skewed distribution "fits the underlying data". I presume that for any given task, we would have to write down a model or likelihood that relates intelligence to the outcome of the task. No matter what transformation of intelligence we use when discussing population statistics, as long as it's monotonic then we can always undo that transformation when we write down a model.

Again, regarding earthquakes: Does a logarithmic scale "fit the underlying data better"? I don't think the question even makes sense. It's like asking which font fits the underlying data better. It's an arbitrary choice that doesn't need to affect our predictions or models at all. But you also said that "the only reason to do that is to prove the people doing the former wrong." so maybe we agree on this, too.

I think you might be misunderstanding psychometrics if your objection to IQ is the supposed implication that "intelligence is this one thing that you can measure". As far as I understand, psychometricians would agree that intelligence has many facets, and that most of them are hard to measure. I think the claim that they're making is simply that one can summarize estimates of these facets into a single scalar that will be correlated with all of them. Just like GDP correlates with many hard-to-measure and complicated facets of a country's wealth. Would you say the fact that GDP can't measure some important things about a country's wealth, or that it's a complicated thing, implies that there's no such thing as "general wealth"? The same analogy works for e.g. health, strength, or pretty much any property that we have a word for.

The way I understand psychometrics—in a simplified matter—is as a misunderstanding of this philosophical statement: “If it can’t be measured, it doesn’t exists”. I see it as a bastardization of logical positivism in many ways, including by totally misusing operational definition. Resulting in the believe that: “We measured it, it must exist!”.

Measuring wealth is an excellent example. GDP might correlated with other measures of wealth, but it hardly explains the whole story. It can be useful in models but you can’t summarize nation’s wealth by only looking at GDP. Similarly IQ can be a useful tool. IQ tests can be used to find people with mental disabilities so that appropriate accommodations can be provided (although most of the time, there are easier, cheaper and better tools to diagnose mental disabilities). However by looking at the IQ alone (especially around and above the mean) you really can’t say much about about the mental capabilities of a person, only correlations of varying usefulness..

There is an important distinction though. I would argue that GDP (arguably) proves it’s usefulness in different economic models. However the same cannot be extended to IQs. I don’t know of many psychological models which use IQ (or any notion of general intelligence for that matter) to explain behavior (other then a hacks trying to prove the superiority of the white race, but let’s ignore those, what they are doing, it is not science).

It sounds like you're just saying that IQ is overhyped. You agree that IQ can be a useful tool. And everyone agrees that having more data than just IQ lets you make better predictions.

How is IQ not "explaining behavior" a problem? We also wouldn't say that GDP "explains the economy", but that's not necessary for it to be useful. Both of these statistics are relatively strongly correlated with many outcomes we care about, compared to other things that are about as easy to measure. I think that's the main reason people use them, and it's a good one.

It sounds like you've seen people over-use aggregate statistics as a way to downplay nuance. That must be annoying. But that doesn't make their use philosophically or scientifically invalid.

> It sounds like you're just saying that IQ is overhyped

Yes. That pretty much summarizes it. If IQ doesn’t further our knowledge of human behavior nor the human mind it is at best a useless construct. We have other more targeted measures (such as SAT) which are easier and cheaper to administer, and are far more useful in both models and evaluations/diagnostics. GDP has the benefit here of being extremely easy to measure and is therefor useful in more general models. IQ doesn’t have this quality, so that is not why most people use it.

Now if that was the only problem with IQ I wouldn’t be so bothered by it. Scientist are allowed to use an inefficient means of measuring their constructs. However IQ advocates often have a vested interest in an IQ that is immutable and (to some degree) heritable. This is often assumed to be true and IQ tests are later fixed in order to make them more robust in score against subsequent trails. This—off course—is circular reasoning (or a self fulfilling prophecy) and is not good science. But it is still used to explain group differences. This is where using IQ becomes scientifically invalid.

Now without the claim of immutability and heritability this wouldn’t matter. We would see a group difference and then we could then go on and try to explain it (e.g. within the context of sociology). However with immutability and heritability, you don’t have to explain it further. The answer is in psychology and genetics. (This is by the way the basis for scientific racism mentioned before).

> If IQ doesn’t further our knowledge of human behavior nor the human mind it is at best a useless construct.

I mean, I think it does further our knowledge of these things, but even if it didn't, that wouldn't make it useless, because it still predicts things we care about pretty well.

> GDP has the benefit here of being extremely easy to measure... IQ doesn’t have this quality

Which measures of intelligence do you think are easier to measure than IQ? You mention the SAT positively below, which is very similar to an IQ test.

> IQ advocates often have a vested interest in an IQ that is immutable and (to some degree) heritable. This is often assumed to be true...

I mean, every researcher would say that it's only immutable and heritable to a degree, and there's tons of research quantifying the degree of mutability and heritability (e.g. twin studies, admixture studies).

> But it is still used to explain group differences. This is where using IQ becomes scientifically invalid.

Is there a specific claim you have in mind that you consider invalid? Can you give me an example of an explanation of group differences that you would consider scientifically valid?

> with immutability and heritability, you don’t have to explain it further. The answer is in psychology and genetics.

Are you saying these claims happen to be false, or that they're categorically impossible / bad explanations? Is there any aspect of human behavior that you think can be explained by psychology or genetics?

> Is there a specific claim you have in mind that you consider invalid? Can you give me an example of an explanation of group differences that you would consider scientifically valid?

Yes. The Bell Curve by Richard J. Herrnstein, Charles Murray (1994). I’m currently having a debate about it in another thread. It uses these assumptions about IQ and the demonstrates group difference with flawed studies, which it then goes on to justify conservative policies.

An example of valid—albeit trivial—group difference on the is height difference across genders. A more interesting group difference is how a mock jury is less likely to convict a minority group if a member of that minority is present in the jury. Then of course the field of medicine has many examples of interesting group difference based on genetics. However as a species we humans are remarkably homogeneous so if you find a group difference in behavior based on genetics it will most likely have an extremely small effect size, and as such be of limited interest. At best this is where IQ research falls in.

> There's tons of research quantifying the degree of mutability and heritability (e.g. twin studies, admixture studies).

These are still up for debate and not without bias. The original twin studies by C. Burt were likely falsified (https://en.wikipedia.org/wiki/Cyril_Burt#%22The_Burt_Affair%...) later efforts of reproducing the results are not without controversies either. And even if you look beyond these controversies there is an inherent bias as adopted twins are way more likely to be adopted within the same socio-economic group, adjusting for that is really tough. I’m not aware of any admixture study which explains a significant group difference in behavior, so I can’t comment on this. But I’ll leave this quote from Wikipedia instead (https://en.wikipedia.org/wiki/Race_and_intelligence#Racial_a...):

> Reviewing the evidence from admixture studies Hunt (2010) considers it to be inconclusive because of too many uncontrolled variables. Mackintosh (2011, p. 338) quotes a statement by Nisbett (2009) to the effect that admixture studies have not provided a shred of evidence in favor of a genetic basis for the IQ gap.

Reply on news.ycombinator.com