Thanks for posting this. It will help me professionally to formulate my opinions on such situations more clearly and will serve as a reference point for future discussion.
Thanks for posting this. It will help me professionally to formulate my opinions on such situations more clearly and will serve as a reference point for future discussion.
Like many books in sociology it comes at the problem from multiple perspectives (in different chapters) such that you can't really hold the author to any one clear conclusion.
Big picture, the space shuttle was "unsafe at any speed" from the very beginning. It was estimated that it had a 3% or so chance of blowing up on any flight which turned out to be just about right in hindsight.
One point of the book was that the Space Shuttle had many things wrong with it: they had a committee who's job it was to review all the safety risks and decide which ones they could accept. "Normalization of deviance" was not a bottom-up matter of bad practices, but a formal part of the organization.
If things had gone a little different with the O-Rings then perhaps the "Challenger Disaster" wouldn't have happened, but then it would have been something else. I remember in the early days the thermal tiles were falling off, but after they had done a few flights they convinced themselves it was OK. It was the Columbia disaster that proved that they weren't OK and couldn't ever be OK.
Having a committee dedicated to assessing risks seems like a good idea. Having a group consciously deciding between risks you can live with, risks you have to live with, risks you can mitigate, and risks that stop everything is a fundamentally good idea.
The problem is taking risks without thinking about them, or taking risks without knowing exactly why they haven't killed you before. And even more unfortunately, this is all on an experimental project That Cannot Ever Be Allowed To Fail, Ever™.
So often - risky decisions are made - many pass without incidence and yet, problems happen. Often such risks are reviewed in hindsight and the weight of the realisation of risk that pass into actuality. Though when they don't fail, nobody passes a second thought and over time, such risks get normalised.
The issue is that all mechanisms to do so are susceptible to some variant of the problem you highlighted. As so highlighted in the OP's submission, individuals can make the exact same mistakes.
I feel that a lot of things comes from the fact that we are constantly encouraged to over-quantify risk. That is we take things where REALLY we don't have a good grasp of the real numbers, and then make something up. If we're good, they'll even be in the right ballpark. If we happen to choose to follow the course of action, and everything is working "just fine", we go back to our mental risk model, which we always knew was kinda made up, and then say "well, I guess I guessed wrong", and assign a lower risk.
In a comittee maybe you can justify the human deaths as nessesary risk, after all people die in traffic all the time. But that ignores the political dimension. The Challenger Disaster was a giant blow to the entire shuttle program: The space shuttle was only funded because of support by the military. With the entire fleet grounded for 2 years and much needed performance improvements canceled that support vanished (see for example: the last DoD shuttle mission was in '92).
But "normalization of deviance" is also partially how the science / the engineering works. The science and engineering is pushing the boundary and turning what used to be unknown into known and tested through experiment / experience. Learning from experience is fundamentally how the science and the engineering work. So it's really difficult to tell when it's a normalization of deviance and when it's a normal scientific/engineering process.
Note that the crew of the Challenger wasn't killed by the "explosion", they were killed when the vehicle compartment they were in hit the water.
If the shuttle had an escape mechanism the way every other manned space vehicle had, they might have survived. Had they built in an escape mechanism, there wouldn't have been any room for cargo. From a safety perspective launching cargo + passengers on the same rocket was a mistake.
Also the shuttle was designed to do a military mission that it never actually did, which would be launching, orbiting the Earth once, and then landing at an airstrip close to the launch point. That requires a large "cross-range" (ability to turn when reentering the atmosphere) which in turn requires larger wings which are more vulnerable to something like what happened to Colombia.
In hindsight, we can see mistakes in terms of specific design and technical decisions. But I don't think those are the biggest lessons from the disaster. We will continue making such mistakes, since technical misjudgement itself will always happen. I think Challenger Launch Decision's main point, that the organizational structure and culture have large impact on individual decision making is the lasting lesson. The book's closing sentence is excellent:
"The lingering uncertainty in the social control of risky technology is how to control the institutional forces that generate competition and scarcity and the powerful leaders who, in response, establish goals and allocate resources, using and abusing high-risk technical systems".
Seems to me that the relationship between Boeing and its various 737 MAX customers could be seen in a very similar light. Abusing high-risk technical systems as a tool to extract resources to cover for artificial scarcity in budgets or market expectations.
normalization of deviance is when a process isn’t followed. when the reality of the process is different than the understood, risk-managed, process.
http://www.consultingnewsline.com/Info/Vie%20du%20Conseil/Le...
Just consider something like a software launch. No matter how much effort you expend trying to make things go perfectly, or how many months of beta testing you engage in - something's going to go wrong. And that's in a nearly completely deterministic environment. Space travel involves countless external nondeterministic factors. Another big difference is that in the software world we get instantaneous scale - up to millions of instantaneous independent trials. Spaceflight advances one costly trial at a time. In the entire history of rocketry, we haven't even begun to approach scale and so all the pretext of safety we could ever elevate is just that - a pretext.
It's for this reason that being open about risk and danger is important. It's also why I think aiming for complete safety can be paradoxically harmful. It creates scenarios where open discussion of danger and risk becomes taboo since it can result in negative consequences. The most critical factor in all of this should be informed consent. Making risk taboo goes directly against that.
Normalizing deviance is more akin to deciding that things that were obviously out of the norm are now accepted as a new normal without the gradual degradation, at least in my way of thinking.
I also think it’s not a very accurate analogy.
If not identical, it at least seems closely adjacent.
I'm curious to hear your perspective; what's wrong with the analogy?
Broken window theory is real: imagine a random person in spotlessly clean city versus a city where there is trash everywhere. More people would litter in the latter city, I don’t think that is too controversial. Thus a run-down neighborhood with lots of broken windows will normalize behaviors that would be stigmatized elsewhere.
The unfortunate part was when NY adopted “broken windows policing”, which is to go into those neighborhoods and make lots of arrests for petty crimes, as if replacing all the broken windows might somehow reduce crime. Of course this is pretty dumb, since entrenched normalization of deviance (decades of bad/discriminatory policy) can’t simply be fixed by zealous application of punitive measures.
Of course, but...
>... as if replacing all the broken windows might somehow reduce crime. Of course this is pretty dumb
Calling it dumb seems to contradict the first quote above... If broken windows normalize bad behaviors, less broken windows would seem to prevent normalizing those behaviors.
>entrenched normalization of deviance (decades of bad/discriminatory policy) can’t simply be fixed by zealous application of punitive measures.
Why not? Punishing deviance prevents the normalization of deviance.
Curious, what would you think of broken windows policing as applied to high-crime white neighborhoods?
Citation needed. There are plenty of counterintuitive things in sociology; the broken window theory sounds plausible but that in no way implies that it's true.
But surely that's because littering causes litter?
Which is exactly what the author did with the landing gear of his aircraft, no?