Hacker News Reader

Comment by p_l | Hacker News Reader

The problem is that replication studies aren't done hard enough.

And by replication studies I mean "take this paper, try to replicate, write a paper on it".

Every singular paper can be wrong, that's why you're supposed to replicate them - that's at least what was taught in psychology, which many consider "social science of less rigour" - well, it was probably the course most hardcore about experimental design in my education and ability to replicate a study...

View on HN

>The problem is that replication studies aren't done hard enough.

>And by replication studies I mean "take this paper, try to replicate, write a paper on it".

You don't necessarily need to perform a straight-up "replication study" in order to do some useful sanity checking on published results. It can be done as part of another novel study that builds upon prior work. That's basically what happened with the fraudulent UCLA political persuasion study several years ago [1]. Some Berkeley researchers wanted to extend the ideas in that earlier published paper, reused similar methods under the assumption that the UCLA folks were correct, and couldn't get anything reasonable within the ballpark of what was published.

[1] https://fivethirtyeight.com/features/how-two-grad-students-u...

My experience of "representative" studies in psychology (university of Berne) was that they'd come up with something, then gather a group of "volunteer" psychology students - 80% women - who need to participate at x studies as requirement to pass, publish it as representative serious research, and call it a day.

Are they not done or are they done and nobody wants to publish them?

Nobody CAN publish them. An actual criterion for most top journals is literally "novelty". Reviewers will immediately levee "this has been done before" as a damning critique in peer review. There is no replication because you can't currently share replications, let alone sustain a career on them.

Reply on news.ycombinator.com