Thoughts on this comment about the supposed hidebound nature of the field of psychometrics?
They don't seem to know about the g factor, which I've linked to many times today. The existence of it means one thing: smart is smart. If you're above average at "mental labor", you're also above average at learning the tools you can leverage to make your labor better faster. This would explain why the calculator didn't shatter the economic predictiveness of the g factor, and neither did the computer, and neither will ChatGPT until it becomes a true AGI. Until mental labor stops being a thing, g factor will always be a strong (though not absolute) predictor of economic success, and so will IQ tests, since they measure it.
I don't see how the comment is related to the article however. The g factor going down in general wouldn't change this correlation; it's still better to be smarter no matter what the average is.
> The existence of it means one thing: smart is smart. If you're above average at "mental labor", you're also above average at learning the tools you can leverage to make your labor better faster.
But isn't this highly reductive? The very responses in this thread attempt to explain the improvement in spatial reasoning can be attributed to video games, at least three responses independently advance that hypothesis. If societal responses to the tests change over time, doesn't that mean the tests themselves are not objective across all time periods?
Funnily enough, that comment suggests that someone who grinded thousands of hours of Tetris would have higher spatial reasoning than a novice, and that's the precise category that the study in the OP reports has improved.
> Until mental labor stops being a thing
But the point is, mental labor changes over time. The measure of g measured by arithmetic tests might go down if society deemphasizes arithmetic because of technological changes. What exactly goes into g, and are those measures the only possible ways of quantifying intelligence?
> I can barely tell what point, if any, they're trying to make.
The comment is highly critical of modern psychometrics, because they argue that the tests in those field have not caught up with societal shifts that impact the results on those tests, using "working memory" as an example. Interestingly enough, the Wikipedia article on g you linked above mentions the following:
"One theory holds that g is identical or nearly identical to working memory capacity. Among other evidence for this view, some studies have found factors representing g and working memory to be perfectly correlated. However, in a meta-analysis the correlation was found to be considerably lower."
So sounds like experts disagree on the specifics.
Also, their thesis is fairly straightforward:
>> The more general critique is that "intelligence research" has ossified around Cargo Cult psychometrics. At one point, "orally administered arithmetic word problems" were a fantastic proxy for economically useful mental faculties. That is no longer true. As the set of useful capacities change, so too should psychometric evaluations. Both because treatment effects are going to make cross-generational comparisons worse than useless, and also because the thing being measured has become irrelevant.
> it's still better to be smarter no matter what the average is.
Certainly, but the devil's in the details.
Women are famously known to not be more attracted to smarter men (by which I mean, men who the women consider more intelligent, and who are emphasizing intelligence over other things.) They're probably right; if it was evolutionarily fit, we'd already have it. But we have enough trouble with childbirth as is, and populations that are considered smarter (by people who believe IQ tests are validly constructed and administered) also often have more genetic diseases.
People who actually understand statistics like Taleb don't believe that IQ is a valid measurement or that it makes you a better person, of course, because "single number go up means better" is not how real life works.