Hacker News Reader

Comment by mikeash | Hacker News Reader

Confusing "the odds of this occurring with a fair coin" with "the odds that the coin is fair" is by far the most frequent statistical error I see, and by far the least often corrected. It's mildly terrifying.

View on HN

Is there a formula you can use to convert between the two?

Bayes theorem. The problem is that it requires a prior, which is usually unknowable.

Yes, but you need to estimate a probability distribution across various types of coin biases first.

http://en.wikipedia.org/wiki/Bayes%27_theorem

Reading some other comments in this thread, I feel that I really ought to have included some more stuff here. There isn't actually such a thing as "the odds that the coin is fair." Either it's fair or it's not. What we can talk about is what probability we should ascribe to it being fair given what we know. Even a single coin flip will have a single result, uniquely determined by the way in which it is launched into the air and caught. Probabilities only exist in the presence of our ignorance of the actual facts, and some people consider probabilities to be themselves a measure of our ignorance of the world.

Oh I get it

P(H0|observation) = P(observation|H0) * P(H0) / P(observation)

P(observation) = P(observation|H0) * P(H0) + P(observation|!H0) * P(!H0)

A normal significance test tells you P(observation|H0). [Though I'm not sure about P(observation|!H0)]. To apply Bayes' Rule you need P(H0), where P(H0)=???

Reply on news.ycombinator.com