Hacker News Reader

Comment by jrflo | Hacker News Reader

Always insane to me how people will just read the headline of an article and point out "flaws" based on what they assume the article says

View on HN

People jump the gun to show their perceived intellectual superiority. Typically same kind of people are last to admit of a mistake.

It's not necessarily about intellectualism. Science in modern times is plagued by publish or perish. Researchers just need to get things published to keep working, and negative results don't get published. So I think people often publish stuff without proper controls/missing confounding variables/inversion of causality/etc not out of ignorance, but because they know the conclusion is dubious - but if they don't publish, they're going to be out of a job.

I don't think many realize how huge the replication crisis is. You have 75% of studies in a leading social psychology journal failing to replicate. [1] Another replication effort on preclinical medical studies found only 11% were able to be replicated, a replication effort on cancer studies found the average effect size to be 85% lower than published, and much more. If you assume most science headlines, at least outside of the traditional hard sciences, are false - you're substantially more likely to be right than wrong.

[1] - https://en.wikipedia.org/wiki/Replication_crisis#In_psycholo...

> It's not necessarily about intellectualism. Science in modern times is plagued by publish or perish. Researchers just need to get things published to keep working, and negative results don't get published. So I think people often publish stuff without proper controls/missing confounding variables/inversion of causality/etc not out of ignorance, but because they know the conclusion is dubious - but if they don't publish, they're going to be out of a job.

This is very true. In my PhD research, I managed to perfectly predict the placebo effect using a random forest. Now, this was actually due to sampling variance, and the finding went away when I averaged over splits.

I often wonder if I'd still been in academia when this happened, if I would have been able to resist the siren call of a Nature/Science paper (and the consequent career benefits).

Oh wow, I did not know that. Thanks for the input.

I doubt OP's jaded response was alluding to your insight, but still thanks for that, it's genuinely good to know.

And then their comment also reliably becomes the top-voted comment in the post

They got trained to said assumptions by bad sensationalist 'science journalism'. Remember how power lines caused brain damage in children? Not through the magnetic fields or anything like that, but the lowered property values.

You actually read the link ?

Reply on news.ycombinator.com