Hacker News Reader

Comment by iq_throw_123 | Hacker News Reader

Imagine that, 150 years ago or whenever, some clever soul had decided to make a written test to measure niceness, and called the test score Niceness Quotient. And the first version sucked, but some other folks iterated on it and over time the test was improved until it correlated pretty well to the sorts of things you would think that niceness would correlate to. 150 years of progress later, we'd have a whole field of Niceometry and researchers trying to isolate sub-areas like charity, friendliness, etc, and trying to suss out an underlying factor of general amiability, and the whole thing would be so well embedded in to the culture that almost no one remembers that "nice" is just a regular word with no objective or scientific definition, and that we measure it with a written test not because that's a good way to measure niceness but because we can't find a better way.

View on HN

This is a terrible analogy, IQ tests measure something real and objective called the g factor : https://en.wikipedia.org/wiki/G_factor_(psychometrics)

Also, intelligence tests are but a tiny part of psychology, I would hardly call it a "whole field".

> IQ tests measure something real and objective called the g factor

Sure, and NQ tests do too, because look how well graciousness correlates with cheerfulness! That can't be an accident, can it?

Less snarkily, a better analogy would be athletic ability. Suppose you take a bunch of people and measure how fast they can run, how well they can shoot free throws, and how far they can throw a football. Will the results be correlated? Of course, some people are more athletic than others. Does that mean there's a quantity called 'athleticism' that we can objectively measure with a number? No; and not because all people are equally athletic, but because you're trying to take a squishy subjective English language word and pretend it's a scalar value.

> I would hardly call it a "whole field".

The problem isn't the size of the field, it's that academics work within their field, they don't refute it. There's a very uncomfortable result about IQ tests that a generation of psychologists have tried to explain away, and I maintain that the reason they haven't succeeded is because they are institutionally incapable of saying, "Hey, maybe this is pseudoscience."

The g factor is predictive of performance in jobs and income though. Even in your own example, yes we could make a test that involves running and throwing, and yes it would be predictive of performance in various sports. One of the tests for aerobic capacity is called VO2max. It is a number with units mL/g/min. Like IQ, this is not the only factor but it does have predictive power.

How would that test do at predicting achievement in archery? How about curling? Are those not "real" sports, or is athleticism a squishy human concept rather than an objectively measurable value?

The amount of strength to draw an archery bow, and steady ones body, would be closely related to strength and VO2Max - with O2 being required for both muscle activity and holding ones breath. And curling is a highly phsyical sport, and for sweepers () its very similar to a full body sprint for each hurl.
There are plenty of metrics you can use to quantify someones atheletic capability, not least of which is your bodies ability to hold and transport oxygen.

() I don't know the technical term, but the point stands.

This study found a correlation between VO2max and archery performance of 0.68 https://aassjournal.com/browse.php?a_id=897&slc_lang=en&sid=...

As noted in the study, the largest influence they tested for archery performance was height. I'd bet a large sum of money that height is also a strong positive influence on many sports (eg basketball), similar to how G is an influence on many cognitive 'sports' like occupations. We can easily measure height and most would have no trouble believing that (largely genetic) factor greatly influences athletic performance. Why is IQ so different?

Wonder if it's likewise applicable to snooker.

Didn't take much googling:

https://aassjournal.com/browse.php?a_id=897&slc_lang=en&sid=...

I didn't see any specific research related to curling, but it is definitely a fairly high peak intensity activity that demands a high level of fitness:

https://curlnoca.ca/wp-content/uploads/2013/06/Curling-Heart...

Well i listed 2 activities, you could certainly add more to the test. Just as we add many questions to an IQ test.

Even if the test doesn't predict performance in every single sport, that doesn't mean it has zero predictive power in all sports.

---

Similarly, I'm not saying IQ is destiny or even the largest factor in any given endeavour, its not. But its equally wrong to say its not a factor at all. It is somewhere in the middle.

Like most metrics, it doesn't capture the entirety of the squishy concept as you say, but it does capture something about it.

I think we understand this in many other areas. Look at PE ratio or revenue growth for a stock, look at goals on target for a football striker, VO2max or running economy for a runner, mileage for a car. IQ is understandably more controversial but the concept is the same.

Is g factor more predictive of income than the financial class of the family you were born into?

Why not just associate intelligence with how wealthy your parents are and call it done?

> Is g factor more predictive of income than the financial class of the family you were born into?

No, but it predicts variation that is not explained by family wealth.

> Why not just associate intelligence with how wealthy your parents are and call it done?

Because they are separate factors with distinct contributions.

The result of a test can be predictive of job performance simply because the result of /any/ test would be predictive of it, because it shows you're willing and able to take and complete tests. This does not mean the score on a specific test is "real".

Minimal example: if you gave a test to a living person and a corpse, only one of them would finish it, and that one would also have a higher income.

Also note the "highest intelligence" jobs don't always have the highest income, like academia, but rather involve taking lots of tests as a prerequisite.

that something is predictive does not mean it measures something else. Nobody will argue with you that IQ tests are predictive of some things, but it still doesn't measure intelligence. It measures something, whatever that something is, that has some predictive power on some other things

Respectfully, you chose the wrong hill to die on. Psychology has many squishy parts, but psychometry is basically the closest thing it has to a hard science.

The other comment adresses your NQ argument, but here's something I don't understand: "There's a very uncomfortable result about IQ tests that a generation of psychologists have tried to explain away"

This reeks of anti-intellectualism by the way ("can't trust the experts!"), but I am curious to know what you're referring to. It can't be the validity or reliability of IQ tests, surely? Both have been very solidly established for a long time.

> This reeks of anti-intellectualism by the way ("can't trust the experts!"), but I am curious to know what you're referring to.

I'm referring to HBD. Respectfully, if you're not familiar with that then you have a lot of prior art to catch up on before you expound on how well-grounded psychometry is.

HBD is so fringe I had a hard time googling it. Just because a subset of a subfield is politically motivated and hateful does not invalidate the whole field. That intelligence testing can be misrepresented to defend racism is an argument against humans, not against the science.

Meanwhile, unbiased research has shown for a long time that there is no significant variations in IQ between races when considering socioeconomic factors. We've determined this with, guess what, psychometry, and it is the overwhelming consensus in psychology. If this was your main gripe with it, rest assured that the field you so underestimate is entirely dismissive of it.

It seems to me you're just looking for ways to dismiss decades of research based on misinformation perpetuated, overwhelmingly, by non-experts. Congratulations, you are in this group.

Thoughts on this comment about the supposed hidebound nature of the field of psychometrics?

https://news.ycombinator.com/item?id=29798887

The problem is there are no sources and many claims. I can barely tell what point, if any, they're trying to make.

They don't seem to know about the g factor, which I've linked to many times today. The existence of it means one thing: smart is smart. If you're above average at "mental labor", you're also above average at learning the tools you can leverage to make your labor better faster. This would explain why the calculator didn't shatter the economic predictiveness of the g factor, and neither did the computer, and neither will ChatGPT until it becomes a true AGI. Until mental labor stops being a thing, g factor will always be a strong (though not absolute) predictor of economic success, and so will IQ tests, since they measure it.

I don't see how the comment is related to the article however. The g factor going down in general wouldn't change this correlation; it's still better to be smarter no matter what the average is.

I believe based on that account's other responses in that thread they are highly dismissive of g ("Yes, psychometrics researchers believe in psychometrics.")

> The existence of it means one thing: smart is smart. If you're above average at "mental labor", you're also above average at learning the tools you can leverage to make your labor better faster.

But isn't this highly reductive? The very responses in this thread attempt to explain the improvement in spatial reasoning can be attributed to video games, at least three responses independently advance that hypothesis. If societal responses to the tests change over time, doesn't that mean the tests themselves are not objective across all time periods?

Funnily enough, that comment suggests that someone who grinded thousands of hours of Tetris would have higher spatial reasoning than a novice, and that's the precise category that the study in the OP reports has improved.

> Until mental labor stops being a thing

But the point is, mental labor changes over time. The measure of g measured by arithmetic tests might go down if society deemphasizes arithmetic because of technological changes. What exactly goes into g, and are those measures the only possible ways of quantifying intelligence?

> I can barely tell what point, if any, they're trying to make.

The comment is highly critical of modern psychometrics, because they argue that the tests in those field have not caught up with societal shifts that impact the results on those tests, using "working memory" as an example. Interestingly enough, the Wikipedia article on g you linked above mentions the following:

"One theory holds that g is identical or nearly identical to working memory capacity. Among other evidence for this view, some studies have found factors representing g and working memory to be perfectly correlated. However, in a meta-analysis the correlation was found to be considerably lower."

So sounds like experts disagree on the specifics.

Also, their thesis is fairly straightforward:

>> The more general critique is that "intelligence research" has ossified around Cargo Cult psychometrics. At one point, "orally administered arithmetic word problems" were a fantastic proxy for economically useful mental faculties. That is no longer true. As the set of useful capacities change, so too should psychometric evaluations. Both because treatment effects are going to make cross-generational comparisons worse than useless, and also because the thing being measured has become irrelevant.

> it's still better to be smarter no matter what the average is.

Certainly, but the devil's in the details.

https://news.ycombinator.com/item?id=35514949

> The g factor going down in general wouldn't change this correlation; it's still better to be smarter no matter what the average is.

Women are famously known to not be more attracted to smarter men (by which I mean, men who the women consider more intelligent, and who are emphasizing intelligence over other things.) They're probably right; if it was evolutionarily fit, we'd already have it. But we have enough trouble with childbirth as is, and populations that are considered smarter (by people who believe IQ tests are validly constructed and administered) also often have more genetic diseases.

People who actually understand statistics like Taleb don't believe that IQ is a valid measurement or that it makes you a better person, of course, because "single number go up means better" is not how real life works.

> Meanwhile, unbiased research has shown for a long time that there is no significant variations in IQ between races when considering socioeconomic factors.

I get that you're new to this topic, but you should at least understand which side you're on. "...there is no significant variations in IQ between races when considering socioeconomic factors" is indeed the mainstream IQ field's side, but the HBD position isn't the opposite of that, it's, "I totally agree, now explain to me again why a test of intrinsic aptitude would need to be adjusted for socioeconomic factors."

It's a mug's game, defending these tests. When IQ correlates to real-world outcomes like salary and academic achievement, they call it evidence that the test works. Then when you point out that salary and academic achievement level have group differences across race (and gender, and religion, and marital status, and height...) for reasons unrelated to intelligence, they say you need to control for those variables. But they don't actually do that! You will never hear an IQ researcher say that someone with a lower IQ is actually smarter than someone with a higher IQ when you consider the neighborhood they grew up in. They just wait long enough for you to forget the particulars, and go back to acting like IQ is an intrinsic genetic quality that's independently measurable. "Wechsler is much better than those dirty biased tests like the SAT" from one side of the mouth, and "We know it's accurate, look how well it correlates to SAT scores" from the other. A mug's game, I tell you.

And all because they can't just admit that g-factor is a statistical artifact and IQ just a test score. That was the point of the Niceness Quotient analogy: it would have the same problems, but the problems would be obvious without 100 years of thinking psychometry was scientific. If one race had higher average NQs than another, there wouldn't be a debate at all, the whole wiki page[0] would just be "Of course the results are weird, Niceness isn't measurable. We tried our best, and the results are kind of useful in some contexts, but it's not actually a trait, just a subjective concept."

0: https://en.wikipedia.org/wiki/History_of_the_race_and_intell...

Brilliant!

No, actually it is thoroughly debunked as a measure of anything useful (except scores below ~80) and its significance as a predictor is miniscule. I recommend you read this: https://medium.com/incerto/iq-is-largely-a-pseudoscientific-...

That article is largely a pseudoscientific swindle, and it has repeatedly been rebutted, please see https://archive.ph/PCvgk.

Overall I'd recommend not taking a single article and using it as something that can refute a whole science.

How about a whole book:

https://en.wikipedia.org/wiki/The_Mismeasure_of_Man

How about a video essay which summarizes other critiques:

https://www.youtube.com/watch?v=UBc7qBS1Ujo

What if you look at the numerous false predictions of IQ research:

https://en.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations

Or a peer reviewed summary which got published in an academic journal with over a 100 citations:

http://www.swisswuff.ch/files/richardson2002whatiqteststest.... (PDF)

Overall I’d recommend regularly reviewing the literature to assess where the scientific consensus is around a theory is. To date, the literature does not support any consensus around the theory of general intelligence nor the efficacy of IQ as a theory for intelligence.

> https://en.wikipedia.org/wiki/The_Mismeasure_of_Man

Doesn't this author have a terrible reputation? And he's known for standing by disproven theories? I'm surprised you didn't find a better example.

> How about a video essay which summarizes other critiques:

Sorry, I haven't watched these 2 hours and 40 minutes, can I read the summary somewhere? Hopefully it's better than Nassim's article.

> Or a peer reviewed summary which got published in an academic journal with over a 100 citations

What is a peer reviewed summary? Is it like a meta-analysis? But honest questions: how much weight do you think this has? Are the citations supporting it or criticising it?

> https://en.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations

Looks like an example of bad science that was caught and exposed? Why do you think this reflects badly on the rest of the research?

> To date, the literature does not support any consensus around the theory of general intelligence nor the efficacy of IQ as a theory for intelligence.

You are asking for consensus. Do you mean an ultra majority? There's no consensus even on whether climate change is real, only 97% of scientist think so. Could you give percentages of who supports what on the science of IQ.

Is there any source of scrutiny you’d accept as valid criticism?

Seriously though, the burden of proof shouldn’t be on the anti-IQ crowd. IQ is a new construct, that proponents have been trying to justify for over half a century. At some point, repeated failures to convince a broader field of its utility should be evidence enough that it might not be worth investigating. Yes, sometimes there are worthy theories which receive unfair treatment, like e.g. continental drift, however even Wegener’s theory took less than 50 years to be accepted, and I bet IQ research has received orders of magnitude more funding and attention than Wegener ever did.

To date, the only evidence for IQ and the g-factor comes from the tests them selves, the construct is not used in any models outside of the field of psychometrics (while psychometrics them selves is becoming more and more fringe as a science). It’s use has historically been focused around racist conjectures and eugenics.

Even though I shouldn’t I’m still gonna give one more shot at finding evidence of the increasing irrelevance of IQ in modern science. I searched for “IQ cognitive psychology” on google scholar. While I did find some attempts of researchers trying to merge psychometrics and Cognitive Psychology, however when Dennis et.al (2009) actually went ahead and looked how useful IQ was in a common neurodevelopmental model, they found it was simply in the way:

> Using IQ as a matching variable or covariate has produced overcorrected, anomalous, and counterintuitive findings about neurocognitive function.

https://www.cambridge.org/core/journals/journal-of-the-inter...

In my search results I also found another paper, which I couldn’t access, dating all the way to 1988, which seems to be making the same case as I that IQ is on its way to the history books: https://www.tandfonline.com/doi/abs/10.1080/0168863880840087...

The most important point of Taleb's is right near the top, in particular anything with graphs. This article pretty much just ignores that entire section (especially the quiz part is a very nice illustration for the statistically challenged). It doesn't even address the point that the outcome graph with the lower tail blocked is basically an entirely random distribution.

Then it happily continues to use the graphs and averaged results, that Taleb showed contain almost no signal, as if nothing happened.

This is not a serious rebuttal.

> Will the results be correlated? Of course, some people are more athletic than others. Does that mean there's a quantity called 'athleticism' that we can objectively measure with a number? No

Admittedly I haven't really studied statistics, but can't you do something like measure those abilities for a population, do a PCA, and define athleticism to be the first principal component?

And then you find out that the first component is height x age. :)

PCA/CCA almost never tells you what the components are. ICA is a bit better at that but still not great.

Thankfully I believe this is changing. Stephen Jay Gould is among the top cited authors within psychology despite not being a psychologist. The psychologists Gould criticized are slowly dying of (e.g. Arthur Jensen) or being fired in disgrace (e.g. Richard Lynn). A new generation of psychologist aren’t picking up their theories, and a new generation of policy makers are distancing them self from IQ science (e.g. the SAT was renamed for this purpose).

I think psychologists have spent enough ink on this non-sense and correctly moved on. It is now up to historians to talk about how damaging this theory truly was, and what the motivation was of policy makers that believed in this pseudo-science. Gould did a terrific job doing exactly that, and a more recent work of Angela Saini Superior: The Return of Race Science is another powerful account of this history, although she is not as optimistic about the state of affairs as I am.

IQ tests produce a number, because they're tests and all tests produce a number. This does not mean the number means anything or existed before the test did. (Especially now that we have AIs that can take tests, do they contain a "g factor"?)

A lot of scientific research is bad and you shouldn't trust it when it comes from a pre-credibility revolution field!

Then you would have a branch of computer science “Artificial Niceness”…

Reply on news.ycombinator.com