> how reproducible (or not) many psychological studies are.
Combine small effect size with a complete lack of understanding of statistics and you can be fairly sure that most psychological studies' conclusions are borderline junk level.
Combine small effect size with a complete lack of understanding of statistics and you can be fairly sure that most psychological studies' conclusions are borderline junk level.
All psych students are required to study stats, and review the study design before starting.
It has everything to with a more systemic problem - publish or perish.
Students and professors alike are basically forced to run new studies. Conducting psych studies takes a lot of money and time because you're dealing with people. And that money and time isn't always available - so your standards drop a lot so you can just graduate or keep researching.
And so far my experience is that most (but definitely not all) psych student are usually choosing psych over STEM due to their repulsion of math, physics and 'anything number'.
They see stats as an 'unfortunate exam they need to get through to pass' so they aim for a barley passing grade in order to focus on other areas they are more interested in. When they are pressured into publishing something the stats part is once again seen as "the impossibly complex part they need help with to be able to publish their interesting findings".
They have guidelines such as "p must be <= 0.05" so they try to mess with the data until they reach that point. Usually, most of the paper (including the conclusion) is already written so stats are considered to be the last administrative bruden to get trough to be able to publish.
If that's true though, then the mathematical rigor that should be demanded from its students must be higher than in other fields in order to achieve the same level of reliability. I doubt that is the case today.
In most cases you have to find and convince a statistically representative sample to participate, and that’s so expensive and hard enough to often not be done completely, without having to consider that a statistically representative sample of individuals for many psychological phenomena is an impossible task.
So what is the point of publishing if you are fully aware you are pushing unreliable results out there? It won't do any good to your reputation as a future professional.
The problem is made worse by the fact that no journal wants to publish studies where it was shown that some factor probably _doesn't_ matter.
https://www.nature.com/news/over-half-of-psychology-studies-...