Selection bias and bombers (2008) | Hacker News Reader

Selection bias and bombers (2008)

johndcook.com

103 pointsawjr18 comments

Interesting - I had always thought of this as an urban legend to demonstrate survivor / selection bias. I always love to discover some source material - http://cna.org/sites/default/files/research/0204320000.pdf

I liked this version:

http://youarenotsosmart.com/2013/05/23/survivorship-bias/

Thanks for posting that link. It is a much better telling of the story, and a much better explanation of survivorship bias.

I much prefer the johndcook article. No banging on about "geniuses", just a to-the-point explanation

Same with criminals.

Criminals are the ones that failed.

Successful criminals are unknown as we never get to know about them as they are never captured.

(Insert edge cases about gangsters writing their memoirs, petty crime etc etc )

Or that successful criminals are like unicorns ;-).

Nah. I'm not aware of the statistics for unsolved violent crime (it's certainly not zero), but the ongoing drug trade is a pretty good example itself.

Not just anyone can manufacture MDMA or LSD. There's the occasional bust, but clearly smart people are getting away with it.

MDMA is a lot easier than LSD, and dumb people get away with just selling Meth (even easier) as MDMA. But ok, maybe some people get away with making synthetic chemicals for some period of time. I don't know that it's necessarily the same people, as opposed to an unending supply of people willing to join the illegal trade. shrug

I expect long-term expected returns of the drug trade are negative, but people aren't good at evaluating risk.

That assumes that success as a criminal entails keeping your activity a secret. I would argue that true success as a criminal is when you're able to keep operating even when people know what's going on. For examples, consider certain drug cartel bosses, or previous US Vice Presidents.

> Criminals are the ones that failed.

In that case it's even quite a bit worse than that. At least with bombers you can reasonably assume the Nazis will be doing everything they can to keep you from dropping bombs on them and the bomber pilots will be doing everything they can to not be shot down over Nazi Germany. That may not be such a safe assumption for criminals and law enforcement.

Plenty of "criminals" don't even know what they're doing is a crime. If the most common ways of getting caught are easy to avoid (e.g. don't confess to the police or keep records of the crime) then the people you're catching are not just bad criminals, they're totally oblivious criminals. Which completely invalidates the theory that ratcheting up penalties will reduce crime -- no penalty can deter people oblivious to the possibility of them being punished. And publicizing the penalties may result in an increase in actual crime by encouraging previously oblivious criminals to take measures that prevent them from being caught. The result can be an apparent reduction in crime (i.e. a reduction in criminals getting caught) even though the actual result is that there are more criminals out on the street.

And then on the other side, the police and prosecutors have incentives to score political points to win promotions or launch a bid for political office. So whether you get prosecuted may have more to do with whether prosecuting someone like you is perceived to have political advantages (e.g. Lori Drew) than whether you've done anything that scores of people who haven't been prosecuted have also done. Or as in the case of Wall St, whether prosecuting you is perceived to have political disadvantages (i.e. bankers withdrawing financial support), preventing prosecution of those who have demonstrably caused great harm for reasons having nothing to do with knowing who they are or being able to prove it.

Reminds me of setting goals for my team around "things that are not to happen." How well have we done if none of those things did indeed happen? How close were we to disaster and just didn't know it?

With, obviously, very different stakes.

Longer writeup with actual sources here: http://youarenotsosmart.com/2013/05/23/survivorship-bias/

Reminds me of this (seen in /usr/bin/fortune):

Certain old men prefer to rise at dawn, taking a cold bath and a long walk with an empty stomach and otherwise mortifying the flesh. They then point with pride to these practices as the cause of their sturdy health and ripe years; the truth being that they are hearty and old, not because of their habits, but in spite of them. The reason we find only robust persons doing this thing is that it has killed all the others who have tried it. -- Ambrose Bierce, "The Devil's Dictionary"

There is more to this story. If you have an hour, I highly recommend watching The Secret History of Silicon Valley. See https://www.youtube.com/watch?v=ZTC_RxWN_xo for a link.

The other side of this story came up there. The real solution to the bomber problem turned out to not be armor, but to start dumping strips of aluminum foil out of the bombers to confuse enemy radar.

  > See ... for a link

Is the link you provide a link to the The Secret History of Silicon Valley? Or should we listen to the presentation to find the link to The Secret History?

The presentation is the secret history.

For many hours of further education, see http://steveblank.com/secret-history/ which links to the same place, then has a blog series about how that one came to be, and the continued history.

This work by Wald is mentioned in this genetics lecture: http://youtu.be/BJm5jHhJNBI

I recommend it if you're interested.

He's updated his website. Looks much better. Though I did like the ship.

Reply on news.ycombinator.com