Hacker News Reader

Comment by fsckboy | Hacker News Reader

famous cognitive psychology experiments that do replicate: IQ tests

http://www.psychpage.com/learning/library/intell/mainstream....

in fact, the foundational statistical models considered the gold standard for statistics today were developed for this testing.

View on HN

> in fact, the foundational statistical models considered the gold standard for statistics today were developed for this testing.

The normal distribution predates the general factor model of IQ by hundreds of years.[0]

You can try other distributions yourself, it's going to be hard to find one that better fits the existing IQ data than the normal (bell curve) distribution.

[0] https://en.wikipedia.org/wiki/Normal_distribution#History

Darwin's cousin, Francis Galton, for whom the log-normal distribution is often called the Galton distribution, was among the first to investigate psychometrics.

not realizing he was hundreds of years late to the game, he still went ahead and coined the term "median"

more tidbits here https://en.wikipedia.org/wiki/Francis_Galton#Statistical_inn...

I took an IQ test as a high school student, and one of the subtests involved placing a stack of shuffled pictures in chronological order. I had one series in the incorrect order, because I had no understanding of the typical behavior of snowfall. The test proctor said almost everyone she tested mixed that one up, because it doesn't snow in the area where I live.

I have no doubt that IQ tests reproducibly measure the test takers ability to pass tests, as well as to perform in a society that the tests are based on.

I think it's disingenuous to attribute IQ to intelligence as a whole though, and it is better understood as an indicator of cultural intelligence.

I would expect that, for cultures who's members score below average on IQ tests from the US, an equivalent IQ test created within that culture would show average members of that culture scoring higher than average members of US culture.

Raven's Progressive Matrices is often administered. Is that test culturally biased? Does that test measure only ability to take that test and nothing else?

Yes, it’s almost certainly linked to quality of schooling and exposure to those types of problems, amongst other things, see the Flynn Effect.

https://en.wikipedia.org/wiki/Flynn_effect

Access to schooling etc can't be the whole story: "black students from prosperous families tend to score higher in IQ than blacks from poor families, but they score no higher, on average, than whites from poor families".

So IQ is malleable and SES-dependent and GxE interactions are real.

No. The first part of that quote is consistent with any hypothesis (G only, E only, G&E), i.e. cannot distinguish between them.

Puzzle tests have their own problems. They're only effective at measuring puzzles solving abilities when they are novel, so retaking the test would lead to higher scores, and practicing even more so. They also only measure puzzle solving abilities which are necessary in some but not all applied intelligence tasks.

I can't quite find the study but there was one mentioned to me about showing the Ravens progressive matrices test to hunter / gatherer tribes, and they did horribly. But those tribes do geometric pattern recognition on the daily basis during hunting, so the tester tried to modify the base shapes to mimic more "realistic" shapes for hunter gatherers (rather than unusual shapes such as perfect triangle, circles and rectangles, hard to find in nature) and the score normalized to median.

I was told this in context of "cultural psychology" how many tests or psychological observations and metrics poorly translate over culture. (especially when you try to pin it on some success metric)

A fun irony (every part of this scientific question is gnarly as fuck, which can make it interesting to follow) is that the more culturally biased an IQ test is, the more g-loaded it will turn out to be.

https://pubmed.ncbi.nlm.nih.gov/24104504/

I think humanity majorly underplays how much success is based on culture. I have a long-held theory that offices don't exist to accomplish work, but to establish social relationships, and that work itself is a secondary product of the office community.

My belief was reinforced when companies switched to remote work, and management at many companies complained that it was difficult to tell who was and wasn't working, when the managers didn't get to watch the workers. Abstracting the social relationship from the results of work will make it easier to judge the work itself, but more difficult to enforce the social relationship. When the abstraction occurred, those who were basing the status of their employees on the social relationship, and not the work output, were especially disadvantaged.

> I would expect that, for cultures who's members score below average on IQ tests from the US, an equivalent IQ test created within that culture would show average members of that culture scoring higher than average members of US culture.

A moment from the show "Good Times" in 1974. https://m.youtube.com/watch?v=DhbsDdMoHC0 at 1:25

Apparently it's referencing a real test, called the BITCH test: https://en.wikipedia.org/wiki/Black_Intelligence_Test_of_Cul...

Also, I forgot how annoying comic relief characters were in sitcoms. They are the opposite of relieving.

in my comment i gave a link to what a fairly large group of university professors, scientists who study, test, and measure intelligence, and what they say they've learned about intelligence. you think you know more, but you don't even investigate or reference what they say, you just think it's the way you think it should be based on ideas you have that you have not tested. not very convincing.

also, cultures don't have iq's, there is no known link to culture.

What exactly are they meant to replicate other than other IQ tests? They don't make a statement about anything that is falsifiable, other than that if you give somebody who scores high on a test carefully designed and tested to match the results of previously given IQ tests when given to the same people, they'll tend to match the results that those people will get on other tests that were calibrated in the exact same way.

If you're trying to say they replicate over the lifetime of the same person, I've had a 15 point swing between tests, out of the few I've taken. What did stay constant for me from age 10 to age 40 was my Myers-Briggs test (my dad was a metrics obsessive), and that's obvious horseshit. Consistency doesn't mean you're measuring what you claim to be measuring.

edit: if it matters, scores were between 137 and 152, so exactly an entire standard deviation. That's like the difference between sub-Saharan and European that racists are always crowing about, but in the same person. IQ doesn't even personally replicate for me.

You can prepare for IQ tests, just as you can for any other test, and you can get better at some of the problems in these tests the more you practice them, just as you get better at Sudoku puzzles the more you do them.

Related: that brain is plastic and can adapt to challenges in different ways. https://www.scientificamerican.com/article/london-taxi-memor...

>What exactly are they meant to replicate other than other IQ tests?

if a variety of different IQ tests sort the same people the same way, even though every question on the tests is different from the other tests, you have shown that the test is showing something about the subjects, not something about the tests. and that is replicable, and falsifiable.

if you follow the same people over time and provide them with new tests, and they continue to sort in the same relative fashion, you have increased confidence that you are measuring something relatively fixed, not variable. For statistical significance (look it up) you don't draw conclusions on the basis of one person (or one Dad) but on population samples tested under standard conditions.

this is like all study results published here, a thousand nerds who've never studied intelligence come up with a hundred objections to what was tested, assuming with arrogance that the people who specialized and did the work aren't considering what comes off the top of this nerd's head. Better qualified nerds did this work.

>Myers-Briggs...'s obvious horseshit

Myers Briggs is not complete horseshit, correlates closely to, but not as good a fit as, the generally accepted Big Five Factor system, the gold standard of personality tests: you should educate yourself a bit more. Myers-Briggs essentially tries to phrase everything in a postive way, where the Big Five separates and includes Neuroticism which is a more negative (for the person) trait. All these traits should be considered adaptive till proven otherwise, so resist the urge to judge.

The Big Five --- not all that great either!

https://www.stat.cmu.edu/~brian/Pmka-Attack-V71-N3/pmka-2006...

(1st section).

Survivorship bias. You can easily make someone's IQ test not replicate. (Hit them on the head really hard.)

Reply on news.ycombinator.com