Also the context of that 1 in 12 in the quote is important. They’re saying that was the chance on the first flight in retrospect. The shuttle and it’s supporting facilities and processes didn’t stay static, they improved over time.
Erm, over a large sample size, yes this is exactly how things work. Sure 15/5 may be too small, but 150/50 isn't. That's well after the point where its reasonable to believe that your coin is rigged.
As the other user mentioned, if you don't have the prior that the coin is fair, a 15/5 outcome may (and does) indeed imply that heads are more likely than tails. In fact, 15/5 would imply a less than 1% chance of a fair coin, given uniform priors. (Beta[15, 5] -> 1st percentile is 50.175)
Now, once you've answered that question, you can integrate to get a probability that the next coin is heads, but with a single flip you can't even answer the first question. You need to have gotten heads and tails each at least once.
When a or b is very large and the other is zero, you can treat it as 1 and get a decent approximation, but it'll be a an estimate that biases towards the center.
If your rocket has a 50% chance of being explosion proof and a 50% chance of blowing up half the time (you could imagine this as any number of situations, like two otherwise identical rockets which you use), any given launch has a 25% chance of rapid unplanned disassembly.
You can do the same thing over a continuous random variable.
You built the rocket, launched it once, and it did not blow up. What should be integrated to get the probability of rapid unplanned disassembly?
The beta distribution[1] is a cool statistical distribution defined by BetaDist{a,b} (or alpha, beta, but that's too much work), where a is the number of successes and b is the number of failures you've sampled.
It has a number of cool properties, chief among them that given X = BetaDist{a,b}, then cdf(X, x) = the probability that the mean of the distribution you are approximating is less than x. It has a bunch of other nice properties too (like E[X] = a / (a + b), which should be obvious), but those aren't as relevant here.
So let's say that you assume a uniform prior. This is defined as BetaDist{1,1} [2]. this is probably the wrong prior. So you might have a better idea. If, for example, you believe there is a 10% chance of your rocket exploding based on some calculations you've done, you might use a differently tuned beta distribution, like BetaDist{9,1} or BetaDist{4.5,.5} if you were feeling uncertain (but in general it would likely be better to use {8,2} in that situation iirc). But let's assume {1,1} for now.
So you launch your rocket. Everything goes great. You update your distribution. Its a success. So you get BetaDist{2,1} [3]. So what is the chance your rocket explodes? Well the cdf of your beta distribution is the probability that the mean is less than x. So the pdf of the beta distribution is the probability that the mean is exactly x. So then
The integral from 0 -> 1 of `(1 - x) * pdf(X, x) dx` is the estimated probability that your rocket explodes on its next launch, since that's "for every value x, the likelyhood of the distribution being that one multiplied by the chance your rocket explodes given that distribution". For the one rocket case, this happens to be equal to E[X] = a / (a + b), so it's 1/3.
For the two rocket case, you apply reinforcement learning/k-armed bandit strategies like UBC1[4] or Thompson Sampling[5]. These are algorithms that will result in you picking the best rocket with as few unnecessary explosions as possible, provably.
You can see some related discussion I've had on HN about these algorithms [6].
[1]: https://en.wikipedia.org/wiki/Beta_distribution
[2]: http://www.wolframalpha.com/input/?i=beta+distribution+(1,1)
[3]: http://www.wolframalpha.com/input/?i=beta+distribution+(2,+1...
[4]: http://banditalgs.com/2016/09/18/the-upper-confidence-bound-...
[1] https://en.wikipedia.org/wiki/Rule_of_succession
[2] http://users.stat.ufl.edu/~aa/articles/agresti_coull_1998.pd...
But my point was you cannot look at a historical event and say "Well this happened, so that was the chance." That's the basis for the silly internet joke "The chance is 50/50, either it happens or it doesn't."
Which is what the people I replied to were doing.
If the event happens repeatedly, you absolutely can! If someone attempts something 20 times, and it works 10 of them, you can conclude that there's an approximately 50% chance of success. You have a sample size! The exact same thing is true for a statement like "the chance of an astronaut dying is approximately 1 in 25". We have the sample size to show that.
>That's the basis for the silly internet joke "The chance is 50/50, either it happens or it doesn't."
No, that's totally different. That's a misunderstanding of priors. What you're doing is more like forgetting that the law of large numbers is a thing.
It only means that’s your chance if you have the technology from quantum leap (the tv show) and you randomly land in the body of one of the participants.
as I said already the shuttle and everything around it evolved. The crews changed. Trying to say “the actual chance of death was this based on how many people died.” is silly.
No, then the event already happened, so you know the outcome with certainty.
> The actual shuttle missions were not a statistical experiment or a consistent action where you can say “well this is what happened so that was the chance of death on an individual shuttle flight.”
Indeed there are confounding factors that make the error nontrivial. This doesn't invalidate the entire experiment.
>Trying to say “the actual chance of death was this based on how many people died.” is silly.
Its exactly as silly as saying "the actual chance of getting heads is based on how many heads you get". The problem with your argument is that that isn't silly. To calculate how likely getting heads is, you flip the coin a bunch of times, see how many heads you get, and then you have your answer (and a confidence level). Its the opposite of silly.
In other words, there's no big difference between "We'll flip a bunch of coins to see how likely we are to pull heads" and "we'll launch a bunch of people into space to see how likely they are to end up dead". The second isn't as rigorously controlled as the first, but that's fine as long as you account for it.
A rocket or space shuttle launch is like a thousand coin flips, where any one result, sequence of results or other combination of events results in death. As an added bonus, any number of unknown external events, from Ambient temperature, to bird strike, to sabotage can render the model useless and kill you in some unforeseen way.
Say you launch your rocket 50 times, out of which 3 times it explodes - first time because of a bird, second time because of the legendary ULA Sniper, third time because of internal problems. That 3/50 is still closer to the truth than just assuming "I really don't know" (1/2) or refusing to answer the question - it has huge error bars, but implicitly captures some of the phenomena that make launches go wrong.
My perspective was probably a little impacted from spending the first truly beautiful day of the summer dealing with a failed "high availability" system. :)
Isn't it the best you can say, though? 20 is a small number but when it's all you have, there's nothing you can do about it. Of course we know the "real" probability is a 1/2 because we know how coin tosses work, but if we didn't know it and coin tosses were black boxes, we'd have to go with this 3:1 chance.