Hacker News Reader

Comment by 0thgen | Hacker News Reader

But to be fair, multiple studies have suggested (for benchmark psychology experiments) a replication rate of less that 50% (https://royalsocietypublishing.org/doi/10.1098/rsos.231240).

I think the quality of psychology research is actually a bimodal distribution. In one mode, you have solid, replicable, serious science. In the other mode (like the priming studies), 90% trash might actually be a generous estimate

View on HN

We can't experiment with humans, so social sciences will always have to deal with low quality, statistically noisy results.

I would modify this slightly. We do experiment with humans; we just cannot do _certain kinds_ of experiments with humans because of ethical considerations. We constantly do clinic trials, which are essentially experiments with a high degree of controls and randomization.

I think probably the major challenge for the social sciences is the nature of the phenomena under study. Psychologists, sociologists, and economists aren't studying physical quantities that can easily be measured. Instead, they typically study abstract and intangible variables. Measuring these accurately is very difficult.

Very well said. And this is partly why psychology researchers have to be so careful in (1) the claims they make, and (2) deciding what counts as a scientific contribution or career point.

In at least the original landmark replication study, the studies actually replicated very well, but (~60% of the time) the signal wasn't as strong in the replicated study as in the original reported results.

That tells us a lot both ways. Also, while they are valuable, the error could be in the replication study.

Reply on news.ycombinator.com