Personal Experience: During and after the economic collapse of 2007-2009 I wondered who was at fault; who to blame. I kept waiting for a clear answer to "THE root cause". Since then I've read things like Dekker's work, and come to realize that blame is not a productive way of thinking in complex systems.
A quick example: in many car accidents, you can easily point to the person who caused the accident; for example the person who runs a red light, texts and drives, or drives drunk is easily found at fault. But what about a case where someone 3 or 4 car lengths ahead makes a quick lane change and an accident occurs behind them?
Catastrophe requires multiple failures – single point failures are not enough.
For example, let’s say you have a service that uses another service that raised its cost from free to $100/hour and you call it 1000 times per hour.
Even though you may not have a fallback, and your service may fail, you need to be able to disable it. In this case, an admin is unavailable and the only recourse would be to lower the capacity to 0, since you have that control.
That doesn’t negate the benefit of validation, but don’t be too heavy-handed with validation, just as a reaction to failure without fully thinking it through.
https://en.m.wikipedia.org/wiki/Greenspan_put
Now, how exactly to solve the problem is a complex question, so I suppose in that respect it's hard to think productively about it.
https://www.history.com/news/beanie-babies-value-criminal-ac...
With a bubble in Beanie Babies there was accompanying fraud and other criminal activity, with most consumers not wittingly participating, but they were affected by it.
So with the asset bubble of the 2000's, this kind of behavior affected everyone, as everyone needs a house.
In another thread I have said I don't blame the masses who may have made bad decisions in hindsight, as they did not have access to all the relevant information. So I do blame the fraudsters, but in the end I really only blame the Fed for keeping interest rates too low, and of course that can circle back around to society as well if you like.
https://www.ft.com/content/bbe858d9-8678-4d1a-84c1-a62ac426a...
You'd have to be going extremely slowly for 3 or 4 car lengths to be a safe following distance. On a typical 60-70mph freeway you should have a gap of at least 15-20 car lengths, and then accidents like that will happen only when other factors are at play (and if those factors are predictable like water on the road then your distance/speed should be adjusted accordingly).
While I think that example was bad, your point about there existing accidents without single points of blame is still valid.
This is a bad example: traffic is a complex sistem with a century of ruleset evolution specifically intended to isolate personal responsibility and provide a simple interface for the users, that, when correctly used, guarantees a collision free ride for all participants.
The systemic failures of trafic are more related to the fallible nature of its actors. The safety guarantees work only when humans demonstrate almost super-human regard to the safety of others, are never inattentive, tired, in a hurry or influenced by substances or medical conditions etc.
We try to align personal incentives to systemic goals with hefty punishments, but there is a diminishing return on that, at some point you have to consider humans unreliable and design your system to be fault-tolerant. Indeed, most modern trafic systems are doing this today with things like impact absorbing railings, speed bumps, wide shoulders and curves etc.
The only way to win is not to play.
I was stationary in a traffic jam when I was rear-ended by the car in back of me. Fortunately, I was not hurt at all.
How could I have avoided this? (see above)?
The systemic failure here is expecting people not to phase out and pay less attention to the road when driving for hours at high speed on monotonous highways.
I believe yholio's statement about "when correctly used" was meant to be "when correctly used by all participants"; i.e., no single participant can guarantee their own, or anyone else's, safety, no matter how careful they are.
On the other hand, the guarantee is, as yholio notes, of an almost entirely theoretical nature even given universal cooperation:
> The safety guarantees work only when humans demonstrate almost super-human regard to the safety of others, are never inattentive, tired, in a hurry or influenced by substances or medical conditions etc.
Well, there can be a difference between "who is at fault" and "THE root cause". Quite a large difference, potentially.
In the case of '07-'09, maybe nobody was at fault (seems plausible) but there was a very neat root cause. The government handed out a lot of money to people who shouldn't have gotten it, the people responsible were largely protected from bankruptcy and the system forced to reform largely as it was. The people who took excessive risk earned an excessive reward - they should have all gone bankrupt. The financial system should actually have changed, and people who made productive investments and didn't take risk should have become ascendant. Instead we have the same old crowd playing the same old game.
Disabling the major feedback mechanism of capitalism is about as root-causal as can be gotten. Nobody in particular chose to disable it though, it was a consensus decision among the powerful.
That is such a over simplification and ignores a lot, most, of the issues in the GFC.
* Regulatory capture
* Illegal behaviour by trusted actors (chiefly banks)
* Corrupt judges
* Carelessness by trusted agencies e.g., credit rating agencies
Long long list of the collection of failures in the international financial system in general and American civil society in particular.
"Oh but it's complicated" is a completely standard line of misdirection that comes up very regularly when people are making repeated bad decisions. Even small children sometimes try it. It is very, very rarely true and particularly in totally synthetic systems like the monetary one. There is always a point of greatest leverage that could be changed and it makes sense to call that the root cause and try changing it.
Now I'm totally open to the idea that I don't know what the point of greatest leverage is in the GFC. I haven't read the regulations and I wasn't in the room when the money was being handed out. But there were billions to trillions of dollars in fake wealth that turned out never to have existed. The fact that the banking industry skated through with the same people largely in charge suggests strongly that no serious attempt was made to figure out who exactly was screwing up.
"no serious attempt was made to figure out who exactly was screwing up" Exactly. Some very powerful people got very rich and it was in now bodies incentives to find anybody accountable.
These are interesting and may help you understand: https://en.wikipedia.org/wiki/Minsky_moment
Complex systems are also simple systems when viewed as a black box from the out side. A lion always eats a gazelle given a chance and a bank always explodes if not regulated to within an inch of its life.
That people like to pretend internal complexity matches external complexity is a very odd mental quirk. It is false in both directions of implication. Conway's game of life is as simple a game as you can get yet it has the most complex behavior possible.
Sometimes it is possible to at least narrow things down to an underlying inherent instability. In the case of your example, a huge underlying cause is an economic system based on debt (backed by interest and usurious transactions). It's for a reason that usury/interest is banned in Islam, Christianity, and Judaism for example. It's a parasitic practice that makes the economy fundamentally unstable. This includes dangerous practices such as selling debt for debt (again part of the same crisis), and things like stock shorting (which, interestingly enough was also banned during the crisis, at least for some critical company stocks).
That being said, if someone is negligent and consistently does negligent things they should probably be put into a position where their negligence won’t cause catastrophic system failures or loss of life. Sometimes that does mean firing someone.
To give you a more concrete example, it moves the analysis away from "Bob deleted the production database" into a more productive space of "we really shouldn't have a process that relies on any human logging into the production database and running SQL queries by hand, that's prone to human mistake".
One very common mode is to take a complex causal web, trace until you find a person of low status, and then yell at and/or punish said scapegoat. That desire to blame is a very human approach, but it a) isn't very effective in preventing the next problem, and b) prevents real systemic understanding by providing a false feeling of resolution.
So if we really want to figure out why the failure occurred and reduce the odds of it happening again, we need to give up blame and look at how systems create incentives and behaviors in the people caught up in them. Only if everybody feels a sense of personal safety do we have much chance of getting at what happened and discussing it calmly enough that we can come to real understandings and real solutions.
The phrasing used on the web site is "Post-accident attribution to a ‘root cause’ is fundamentally wrong." At first glance, it sounds like the author means there is no cause that can be found so you shouldn't try to determine the cause. First they clarify by saying there are many causes not just one. However, this phrasing made me scratch my head:
> The evaluations based on such reasoning as ‘root cause’ do not reflect a technical understanding of the nature of failure but rather the social, cultural need to blame specific, localized forces or events for outcomes.
I don't know what other organizations are like, but where I work, when we do a "root cause analysis," we aren't literally looking for a single cause, despite the name. The "root cause" is almost always that pieces a, b, and c came together in an unexpected way. I can definitely think of places where I worked where they were mostly out to place blame, though, and I guess that's what they were trying to caution against.
Organizations that are people dependent rather than process dependent tend to have higher risks of failures.