Comment by forapurpose | Hacker News Reader

Comment by forapurpose | Hacker News Reader

On one hand is, "A researcher in human intelligence at Utah Valley University". On the other are, "the 29 best-selling introductory psychology textbooks in the US - some written by among the most eminent psychologists alive".

Why would I believe the former? I'm not saying that Utah Valley U. researchers shouldn't raise questions, but that the standard for what I'm going to take at face value is much higher than one study that contradicts everyone else. One study is just the start of the start of a discussion. I think I'll let the body of evidence accumulate and let experts weigh in.

One psychological concept I'm familiar with is confirmation bias. This article confirms beliefs held by some at HN about political bias, authority, and accuracy in science, (and unfortunately, an effort to rationalize and justify racism by finding some basis for it). That doesn't make this report any more likely to be accurate. Apparently, 29 leading textbooks contradict it; if not for the bias, why not take 29 publications at face value instead of 1?

EDIT: a bunch of edits; sorry for the mess.

EDIT: I'll add that this one study was published "in an open-access article in Archives of Scientific Psychology". It's not even peer-reviewed [EDIT2: It is, in fact, peer reviewed; sorry. I still expect the textbooks to be reviewed much more carefully than one paper.]

They make some pretty specific claims though, it's not like it's "one researcher" vs "29 textbooks". Textbooks can be written poorly too, you know. For example:

> In contrast, fewer than a quarter of the books covered the most strongly supported contemporary, hierarchical theories, such as Carroll’s three-statrum model and CHC theory each of which posits the influence of a general intelligence on other cognitive abilities.

> The most common inaccuracy (appearing in nearly half the books) was that intelligence tests are biased against particular groups or individuals. This contradicts the 1997 consensus statement which tackles this issue and concludes that “intelligence tests are not culturally biased”.

Note: it is peer-reviewed.

"Archives is staunchly peer reviewed, and the standards for publication in Archives are as high as those for any other APA journal. Therefore, only those manuscripts that are well-written and present research of the highest quality will be accepted for publication. Rejection rates are similar to those of other APA journals."

https://www.apa.org/pubs/journals/arc/index.aspx?tab=4

Well, there is nothing to believe or not believe in this study; it simply points out that most of the books disagree with what's widely accepted in the field[0]. Which do you believe, the widely accepted theory or 29 books?

[0]: https://en.wikipedia.org/wiki/Cattell%E2%80%93Horn%E2%80%93C...

> it simply points out that 29 books disagree with what's widely accepted in the field

The study claims that. The textbooks claim otherwise. Why do you believe the study's claims? (I'd believe Wikipedia less than either.)

You could also allow yourself to exist in a gray area where you find this interesting but do not believe you know the exact truth.

> I'd believe Wikipedia less than either.

That's where you make a mistake. Whatever the reason, the Wikipedia articles on the g factor, the intelligence quotient and Cattell–Horn–Carroll theory represent better what is widely accepted in the field than any other source. This somehow also turns out to be true for other academic fields.

Anyway, when source A says explicitly that source B is wrong but source B makes no such claim about source A, you should usually believe source A. But in this case you can know that the study is right by just learning for yourself what does happen to be widely accepted in the field and why it is widely accepted.

> represent better what is widely accepted in the field

How do you know this? Are you a practitioner in the field?

> when source A says explicitly that source B is wrong but source B makes no such claim about source A, you should usually believe source A

Yikes. That is scary: "You are wrong." You'd better respond or else that proves it.

"Newton is wrong." "Darwin is wrong about evolution." "Climate change scientists are all wrong." "Popper is wrong about postpositivism." "The legal system is wrong about the right to cross-examination."

Within psychology there's a subfield that studies intelligence. The introductory textbooks to the whole field of psychology should be giving a summary of the discoveries of each individual subfield. But when it comes to intelligence they are giving information that the intelligence specialists agree is incorrect or outdated.

It's not one expert against many, it's people outside a speciality refusing to accept the consensus from within that speciality.

> intelligence specialists agree

The author of one study says that; it's not a consensus of intelligence specialists. From what I know, textbooks are usually carefully reviewed by experts; in fact, they likely are far better reviewed than this study, which wasn't even published in a peer-reviewed journal [EDIT: As pointed out below, it is in fact peer reviewed; sorry, my mistake. I still expect textbooks to be more carefully reviewed than one random paper.]

From the article:

Reporting their findings in an open-access article in Archives of Scientific Psychology ...

Open access is something different from not being peer reviewed, it describes the amount of access granted to articles after publication, peer review is what happens before publication itself. It says it is a peer reviewed journal: https://www.apa.org/pubs/journals/arc/?tab=4

> Archives is staunchly peer reviewed, and the standards for publication in Archives are as high as those for any other APA journal. Therefore, only those manuscripts that are well-written and present research of the highest quality will be accepted for publication. Rejection rates are similar to those of other APA journals.

>which wasn't even published in a peer-reviewed journal

"Open access" is not the same as "not peer reviewed." There is no subscription required to get the articles in the journal, but it is peer-reviewed.

From the page for the Warne et al. paper from the OP link (http://psycnet.apa.org/record/2018-07714-001):

>Publication Type: Journal; Peer Reviewed Journal

From the publication announcing "Archives of Scientific Psychology" (http://psycnet.apa.org/fulltext/2013-03009-001.pdf):

>Archives of Scientific Psychology uses peer review to determine whether a submitted manuscript is suitable for publication. Authors are free to submit manuscripts that reveal their identity or that are prepared to keep reviewers unaware of who they are. We adopted this policy because we are unconvinced that evidence on peer review rules out the possibility that some reviewers’ evaluations of manuscripts are influenced by characteristics of authors (Shatz, 2004; Weller, 2001). Likewise, reviewers will be given the option of revealing themselves to the authors.

> From what I know, textbooks are usually carefully reviewed by experts; in fact, they likely are far better reviewed than this study

I've seen far too many minor-to-grave errors in almost every textbook I've read to believe that -- and I wasn't any kind of expert in most of the fields concerned.

By inspecting the supporting evidence for some of his claims. If the argument is supported by solid references then there is reason to take his analysis seriously.

Reply on news.ycombinator.com