Hacker News Reader

Comment by mykowebhn | Hacker News Reader

Yes, I agree with the posts arguments.

One HUGE thing it's missing, though, is the deliberate hacking of results to reach statistical significance. I'm willing to bet that the results of a majority of psychology studies are not reproducible.

In another lifetime, I worked as a research assistant at a very large, well-funded, Ivy League psychology lab. Talk about p-hacking. Our PI would go so far as to deny potential candidates entry into our study, as well as the therapy, simply because the PI thought these candidates wouldn't help the therapy our PI developed look good in our study. Note, these candidates did meet all our OFFICIAL study criteria for entry into the study.

View on HN

"I'm willing to bet that the results of a majority of psychology studies are not reproducible"

Indeed

> Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%).

https://en.wikipedia.org/wiki/Replication_crisis

That is appalling, imho you might as well call an area of study, that has less than 50% reproducibility for studies published in “credible” journals, a pseudoscience.

I do not even think 50% is that good either and should be lumped in with pseudoscience as well.

The author talked about it in length in an other blog post. https://www.experimental-history.com/p/im-so-sorry-for-psych...

This is basically just scientific fraud, no?

It's also how most empirical science operates.

If someone collects data and the study outcome is not preregistered, you can assume p-hacking. It would be implausible not to. And in most fields, preregistration is not common. (And even if there's preregistration, regularly people just switch their outcomes, and nobody cares.)

And to play the devil's advocate: psychology is probably doing better these days than most other fields, because it's been the posterchild example of the replication crisis.

I personally lean to yes, but that's more about what people do with the results than the results themselves.

Here's an infamous example: https://en.wikipedia.org/wiki/Milgram_experiment#Validity

Honestly after reading that it seems impossible to really conclude anything...as it's just full of conflicting results...is that innately fraud? No but certainly careers/$ have been made from biased/agenda-driven interpretations which seems fraudulent.

There probably should be room in some of the social sciences for flexibility like this as long as it's called out right at the top as part of the experiment design so that the reader knows this is exploratory initial research being done for directional purposes - and that's it.

Unfortunately as the movement from History, Philosophy, and the other liberal arts disciplines became 'sciencified', the ability to deliberate on something rigorous but still with enough room to explore has been sacrificed in favor of trying to be more like the physical sciences.

Reply on news.ycombinator.com