Hacker News Reader

Comment by acover | Hacker News Reader

The statement should be there is a 99.5% chance that we correctly rejected the null hypothesis.

I agree it isn't careful wording. Thanks.

I played a game of Monopoly where I started with a roll of 12. There was a 97% chance I correctly rejected the hypothesis that the dice were fair. Maybe this is true in some sense? But it's still somewhere between misleading and nonsensical to say.

I think a better analogy is that it's like having many pairs of dice, and rolling each pair in turn until you get a roll of 12. Then concluding "this particular pair of dice must be loaded".

Presumably, the researchers did not select rivers at random to study, they selected this river in particular because of the changes it is undergoing.

Yes, I agree that p-value tests have flaws. If you look at the data to determine your hypothesis it's easy to overfit.

Bayes factor appear to solve this issue. I disagree that this is a basic education issue. It is a lack of agreement among scientists as to what statistical analysis is appropriate.

"So it’s 99.5% that it occurred due to warming over the industrial era"

There are at least a couple of statistical fallacies in this conclusion. And there isn't a lack of agreement about that.

One problem with p-value tests is precisely that people misunderstand what the p-value means, which is where basic education comes in. It could save people from believing a lot of things they shouldn't. (Like many health and fitness crazes over the last generation, for instance) Or at the very least, we could train science journalists.

Do you agree to the statement "so it's 99.5% that we correctly reject the hypothesis it occurred without warming over the industrial era"?

I think you're mainly correct. The main issue is that there's an implied "given that the model from http://www.nature.com/ngeo/journal/v10/n2/full/ngeo2863.html is 'perfect' (at least for this case)". Since it seems unlikely that the model is perfect, the numbers they give are almost certainly inflated.

Nope. I'm totally wrong.

Your second sentence should be in quotation marks to clarify that your comment is debunking it.

...as best I can tell, you have still swapped them? It sounds like you're still talking about P(null hypothesis|evidence), whereas p-values are about P(evidence|null hypothesis). (Well, not quite the latter but something like it.)

Yes. Ugh. You are right.

I treat P(evidence | null hypothesis) equivalent to P(null hypothesis | data). When it should be

P(evidence | null hypothesis) = P(null hypothesis | evidence) * P (evidence) / P (null hypothesis)

As we don't know either of those extra terms we can't determine P(null hypothesis | evidence).

Thanks.

Interestingly wikipedia notes that P(Reject H | H) = pre-defined threshold for rejecting H [1]

[1] - https://en.wikipedia.org/wiki/P-value

Reply on news.ycombinator.com