Hacker News Reader

Comment by mdpopescu | Hacker News Reader

A second problem...

"As a simple, but numerically stronger example illustrating this, if we toss a coin 1000 times, then no matter what the result is, the speciﬁc observed sequence of heads and tails has a probability of only 2^−1000"

No... if you toss a coin a thousand times, the probability of observing the exact sequence you just observed is 1. It will ALWAYS happen (that's what probability 1 means). Yes, if you had an independent specification for the sequence (like writing it down before tossing the coin, or converting the binary representation to ASCII and discovering it spells "Kilroy was here") then the probability would indeed be 2^-1000; but that would be a different case.

View on HN

It's actually not a problem. You can come up with any number of hypotheses about coins. Some of them take the form, "This coin will produce <some specific output> in the next 1000 typical flips". That hypothesis and others with similar, more complex form, like the pair of hypotheses that predict flip 1001 after the same first 1000, GAIN CREDENCE when you perform 1000 flips that conform to them. Others of the similar form lose it. Other hypotheses of wildly different construction, like, that a coin is more or less fair, lose and gain credence according to whether or not they predict the observed result.

The fact that you didn't write a hypothesis down before you did the test has very little to do with whether or not the data supports the hypothesis. Hindsight bias matters, but only as far as it corrupts your experience. The machine with the infinite library of coin-flip-hypotheses updates just fine.

On a side note, coins are not fair, in general, and Jaynes actually goes into some detail about the process of cheating at coin flips.

Obviously, Jaynes was talking about a reasonable prior probability you had before you flipped the coin. And he was talking about flipping the coin in a way that the head/tail outcome is completely unpredictable for you (50/50) probability, minus epsilon for weird stuff like landing on the edge. He later addresses the coin flip problem more rigorously.

Besides, once you have observed the outcome, it's probability is not 1. You could have misremembered the sequence, or you could have missed a flip, or otherwise done your observation wrong. The probability of making at least one such error with 1000 coin flips in an informal setting is actually quite high.

Reply on news.ycombinator.com