The problem with p-values | Hacker News Reader

The problem with p-values

aeon.co

30 pointsteslacar7 comments

The article fingers journals and grant makers, but it ignores the historical issues with collecting data in the social sciences: generating lots of data points is time-consuming and expensive.

In linguistics, getting a simple word count used to require hiring an extremely skilled worker to read a book and tally words. In psychology experiments, you have to recruit each participant and run them through an often elaborate deception.

However, technology (especially Amazon Turk) makes these excuses irrelevant. Even with in-person experiments, you can generally automate a large part of the administration and scoring of the task. There is no excuse for small sample sizes anymore.

Clickbait title; the whole thing only discusses psychology and no other field of study. Certainly not 'science' as a whole.

Perhaps, but that sort of generic objection never makes for an interesting discussion.

We'll change the title from the doc heading ("It’s time for science to abandon the term ‘statistically significant’") to the page heading in the hope of stimulating something more specific.

Edit: turned out to be a dupe of https://news.ycombinator.com/item?id=12685177.

No the title is accurate, the article is about use of statistics in scientific papers in general, with a focus on biomedicine - a huge field.

It talks about it, but it's based on a series of links to issues that were specific to psychology/psychiatry/brain-imaging. I'm not saying that the broader point isn't valid, but it's certainly not supported.

As I read it it refers to the biomedical field a lot...and the foundation of the article actually was built on Fisher's full explanation of statistical significance and backs it up with physicist Robert Matthew's insights...even so - you can't get away from the math - the math is sound.

> backs it up with physicist Robert Matthew's insights...even so - you can't get away from the math - the math is sound.

It's mind-blowing how bad the mathematical analysis is within the social sciences. As an undergraduate, the social science mathematics center chided me for using (standard) data-mining techniques as "fishing for results" despite having compensated for the number of parallel tests.

Later I learned that social science labs run pilot studies until they see a result and run a full study. Everyone is fishing for results, they are just claiming that each cast of their line is independent.

3-sigma should be the standard and no journal should accept a p-value greater than .01.

Reply on news.ycombinator.com