Hacker News Reader

The claim the significance has nothing to do with the magnitude of a difference is just wrong though. It clearly does, this information is just merged with other information about the variance and sample size to get the p-value, which is compared to a threshold to get significance.

View on HN

I think what the article meant to say was that, for the same number of samples, the p-value when you have delta=50 (the difference between groups), stdev=10 is the same as the p-value of delta=0.5, stdev=0.1.

Depending on the study, finding a delta whose p-value is significant does not necessarily mean that the size of the effect (i.e., delta) might be significant enough to be useful.

it's not "just wrong". On the contrary, it is asymptotically true. In a world of abundant data, that matters.

Please expand on this.

Reply on news.ycombinator.com