Hacker News Reader

Comment by kgarten | Hacker News Reader

I'm wondering about the study ... I'm a complete novice so might get things wrong, yet the participant description has not much information (e.g. average, std of the age, gender etc.). N is also just given in the abstract.

Also the study does not mention how the experimental setup was: "Subjects were administered a comprehensive psychometric battery of fluid and crystallized intelligence tasks,"

There can be substantial differences, just on how the experimental setup was administered (e.g. potential ordering effects between the tests) and when the scans were done.

Does anybody know if that's a standard procedure to determine "general intelligence"? Sounds vague to me.

View on HN

Reading science papers (my field is biochem, YMMV for other disciplines) is a lot of work if you're not already expert in its very specific field and topic.

Paper authors try to be as succinct/brief as possible, and in my experience editors generally encourage this as well. There are incentives to do so.

In order to understand what that part of the paper means, you would need to actually read every reference, and maybe even their references, and maybe even go up pretty far up the reference tree. You might try and take a shortcut by reading recent or well-cited review papers on that specific subject. In order to take a shortcut from that, you might try and consult a handful of textbooks.

If you were an expert in the field, you probably wouldn't need to refer to the textbooks, and you might even know the rough ideas behind enough of the papers and might check only a few of the references to be sure.

Then beyond this, any variation from the afore-cited methods will usually be detailed within the paper, possibly in further sections or in figures or tables (and from what I can see, that seems to be generally the case in this paper).

Being succinct is also a neat excuse for leaving out details that are actually necessary for any scientist to replicate your paper. Oh and it allows certain interdisciplinary works to incorrectly abstract away technical details from the secondary domain.

You're not wrong, and in an ideal world it is a sensible way for experts to communicate amongst themselves. But at the same time this sort of deep reading is a fool's errand at least half the time IME, because the authors of the paper themselves did not carefully read every reference. And even when they did there are often subtle unpublished differences between a cited protocol and the protocol they actually used.

To be fair, it's possible this is a domain specific phenomenon, and I'm not especially familiar with human intelligence research. But it's a real problem in the areas of neurobiology and computational neuroscience I've studied.

I think it's in large part a natural consequence of trying to scale science in the wrong way, but there are also bad cultural factors and a few legitimately bad actors that push the issue even further.

Reply on news.ycombinator.com