Hacker News Reader

Comment by utborin | Hacker News Reader

99% of folks here don’t deal with life and death matters. When you do, you get a lot of practice at canceling, which helps. There’s also the fact that you can say, “if you do this, people will die,” which is a pretty bright line when you’ve got something to back it up. I actually found it more difficult to take a stand in less consequential fields. But it’s still very, very hard to look someone more powerful in the eye and tell them no. I’ve had to do it and it’s not fun. (Nothing on the scale of a Shuttle launch. I once had to effectively revoke a judge’s flying privileges, for instance.)

McDonald is on a whole other level though. That’s not just your employer, that’s the entire space program. I hope I never have to find out whether I’m made of the same stuff he was.

View on HN

>if you do this, people will die

In my experience, this is not how it works. Very rarely can you say definitively that “people will die”. Reliability is shrouded in uncertainty and the best you can say is, “people may die, eventually ”. If you make too absolute of a claim and it didn’t occur you will gradually erode credibility. Look at Columbia... they knew the foam shedding was out of spec but had so many instances of probability being on their side, if you claimed people would die act each of those launches soon you just become noise. Over a long enough timeline you may be right, but phrasing in that way is unlikely to help in low probability events.

This man understands. It’s the .001% chance that kills people

There was an anecdote from one of Czech researches in Papua-New Guinea who said he was surprised that local hunters absolutely refused to sleep under suspicious, not entirely healthy trees, and were very careful about examining the spot before they chose it for rest. He considered the chance of the trees falling down very low, perhaps less than one in thousand.

He changed his attitude when the elders explained to him that while the chance was very low, the hunters took a lot of naps in the forest throughout their lives and over decades, the low chance translated to almost-certainty.

I think I read this in a column of Jared Diamond's.

No, it's the 10% that kills people. They knew the O-rings were failing because they had analyzed previously discarded rockets. So the engineers said, "We got lucky this time; eventually this is going to kill people." But after launching 4-5 times and having O-ring failures with no problems, management (and probably everyone) started thinking it wasn't really as dangerous as it was.

The percentage loses meaning without an expected number of opportunities to calculate the expected value. 10% risk x 1 flight might be deemed acceptable. 10% risk x 100 flights means you should plan on losing 10 shuttles. Neither statement about risk probability is wrong (0.001% vs 10%), but both are incomplete. The actual shuttle risk was calculated between 1/78 to 1/254, if I remember correctly

aka Normalization of deviance https://en.wikipedia.org/wiki/Normalization_of_deviance

And a short video in context of challenger accident : https://www.youtube.com/watch?v=tC0qkVf5SmE

Mullane was on STS-27 -- that was a disaster on the level of Challenger or Columbia but by pure luck they avoided a deadly ending. The failures were predictable. And this was the second flight after Challenger's last.

I think the underlying point is that there are cognitive biases that prevent decision makers from effectively understanding that risk. It’s very much like financial crashes; after the fact it becomes obvious the risk was there but our biases prevent most people from seeing/understanding it effectively in the moment

It's enough that people may die (or even get injured) as a consequence of your work. That's the line you shouldn't cross and it actually helps to have such a clear and unambiguous definition of that line.

I understand that stance, but when humans are the decision makers and have all kinds of other pressures (cost, schedule, etc.) it’s often not enough that people “may” die. Especially if we’re talking about risky endeavors like human-rated space travel, “may” is only one level across the gradient, and that level could be considered an acceptable risk by some. You will never drive risk down to zero, so “people may die” will almost always be true. If that’s your threshold, you will never launch.

The real difficulty is accurately gauging risk, especially when there isn’t substantial data.

Sts-27

https://youtu.be/BswkvaAaqSM

1988. Damage to orbiter Atlantis.

Not the same foam that shedded (pieces from SRB rather than external tank). But still demonstrated failure modes.

The failure mode was known (hence why it was considered out-of-spec) but it didn’t translate to an accurate risk assessment. It was deemed “in family” meaning it was known to be outside of spec but not a risk. Somehow, over many flights, something that was originally characterized as a safety risk lost its credibility as a risk. It’s not that it was unknown, but the understanding of the risk was changed. I wonder if STS-27 led to a false sense of confidence

I'm curious to hear what you think about the Therac-25 situation because the line between a code change and fatalities isn't as clear cut as with Challenger, but definitely still there.

Reply on news.ycombinator.com