Han Solo and Bayesian Priors

You also need to consider the survivorship bias of a story from a long time ago in a galaxy far, far away.

All the other potential stories where the protagonist predictably wins the Darwin Award would be too uninteresting to reach us over such an immense time and distance.

C3PO's estimation would have been a lot more useful if he'd calculated their odds.

Also his calculation suggests there's a strong open data regime in the empire, which is a nice thought.

I like the concept, it's a cool way of introducing Bayesian probabilities!

The conclusion strikes me as overly confident, though. It implies that if we had 100 Han Solo's, we are very confident that about 3/4 of them are going to make it through. This comes about because it uses this:

We're going to say that C3PO has records of two people surviving and 7,440 people ending their trip through the asteroid field in a glorious explosion.

as data for Han's odds of making it. But 100 out-of-shape people dying on the ascent of Mount Everest does not tell us anything about the odds for someone who is very fit.

I would rather model that there is no one true probability of making it through - it's going to be dependent on the pilot. Mediocre pilots might have odds in the neighborhoud of 1/3720, but there is presumably a lot of variance depending on skill, and my prior belief is that Han would wind up in the upper end of the distribution.

The 75% result (chance to survive) seems about right, but the confidence in this result seems completely unnatural. The sharp peak in the posterior plot suggests that I have nailed down the exact probability to +/- 2%. No way.

Thought experiment: imagine there is a second robot, C4P1, on one of the TIE fighters. He independently comes up with an estimate of 10,000:1 for Han's death. Are we now going to slide our estimate all the way down? Or, what if C3P0's line was revised to say 1,000,000:1? Would the author now say, 'whelp, I was pretty confident at 20,000:1, but now that C3PO's number is so much bigger than mine, I guess Han's gonna die'?

I think this is the kind of analysis you need instead:

http://tvtropes.org/pmwiki/pmwiki.php/Main/PlotArmor

Original source: http://www.countbayesie.com/blog/2015/2/18/hans-solo-and-bay...

Does the author have something backwards or do I?

P(RateOfSuccess|Successes) = Beta(α,β)

"In Bayesian terms, C3PO's estimate of the true rate of success given observed data is referred to as the likelihood."

BUT we know likelihood(rateofsuccess|data) = probability(data|rate of success).

I am confused.

One factor seems missing: the deviation of Han's skills from that of the average pilot. Does C3PO take it into account in his likelihood? Do we take it into account in our probability of success? Although I do get the point, it seems there's often something of that sort, a simplification, in that kind of bayesian reasoning.

Someone needs to do this with Game of Thrones characters. Everything I thought about important characters' survival chances turned upside down as I watched it.