How Complex Systems Fail (1998) [pdf] | Hacker News Reader

How Complex Systems Fail (1998) [pdf]

web.mit.edu

131 pointsswolpers33 comments

Threads from 2017: https://news.ycombinator.com/item?id=15002683

https://news.ycombinator.com/item?id=14127543

2014: https://news.ycombinator.com/item?id=8282923

I guess this is one of the main keys:

A corollary to the preceding point is that complex systems run as broken systems. The system continues to function because it contains so many redundancies and because people can make it function, despite the presence of many flaws.

"Slowly at first, then all at once."

The question we should be asking is whether these complex systems are actually required.

In absolute terms, our ancestors lived much much simpler lives and somehow managed to survive. So, it's not that we're required to have such complex systems, it's that we've decided that the reward is higher than the cost (which is fine). But, I wonder, how accurately have the costs and the rewards been evaluated? I suspect that we don't always take into account the full cost and risks of complexity. If we did, I suspect we'd build fewer highly complex systems, and the systems we did build would be better.

If you think of the entire IT experience as a system, somehow all the various social, economic, and technical forces have conspired to produce massive complexity which shows in constant bugs, failures, broken websites, and security breaches. I wish we could just step back, take a deep breath, and find ways to simplify and clean some of this up.

One of the beauties of complex systems is that these individual failures do not produce system-level failure unless you are extremely unlucky and they all fail simultaneously.

If you are simply counting the number of failures, complex systems do look a lot less appealing than simple ones. But if you count only catastrophic failures, complex systems begin to look more worth the energy tax spent maintaining them.

This is the guiding design principal behind most biological, social, economic, and (now) internet-related systems. I don't even think it is intentional. It is simply the best solution so it was discovered by natures and is constantly rediscovered by engineers.

Technically we could simply stop all of this IT stuff, re-allocate the world so everyone gets an acre and dissolve the government. No one would work more than a few hours per day (tending the garden, etc..) The problem is trust. We can't trust someone else because there will be stealing. Those thieves will meet others like them that steal, rape, pillage. Cycle of mis-trust starts all over again and societies form to do something about it. Then things go from there. Now that I think about what I write, we have Facebook because of thievery (just a joke). I do get your point of not starting over, but just listing the case why we get complexity. Some of it is likely not necessary but it's usually because of something.

Technically we could not do this. This is wildly inaccurate. The massive redistribution of population alone would be totally infeasible.

And if everyone is going to make due with what they can grow in their acre, how is famine going to be dealt with? Is everyone expected to become vegetarian? What about diseases? What about transportation?

Who cares about transportation, lack of water would kill the majority very quickly.

this would result in the life expectancy hitting ~25-30, just like the good old days.

If you lived past childhood it is highly likely you would live to your 50s or 60s. Surviving your first few years was the hard part.

It would be hard again, under your arrangement.

> Technically we could simply stop all of this IT stuff, re-allocate the world so everyone gets an acre and dissolve the government. No one would work more than a few hours per day

Well, apart from all the people who would be dead; the diabetics, many of the elderly, and those with various other chronic conditions. The gradual reintroduction of fatal measles would kill from the other end of life. Women would probably be extremely upset about the end of access to contraception, especially as the maternal mortality rate crept back above 1/100.

> In absolute terms, our ancestors lived much much simpler lives and somehow managed to survive.

The quality of life was radically different, and they didn't live as long.

The complexity we see today exists for a reason, or really, a multitude of reasons. Would it be nice if things weren't as complex? Definitely, but do we have any reason to believe that you can have a world-wide distribution network of goods without this complexity? I would be very interested to hear any argument that this is possible, but I don't think it exists.

It is certainly possible to simplify some things in isolation, but the complexity we see is the product of billions of humans interacting with different governing and power structures, cultures, monetary systems, value systems, and more. So any simplification would demand a simplification of those systems. How realistic is that? How desirable is that?

Of course I wasn't suggesting that we should all give up and live in caves or something. It's not a binary choice. I refuse to believe that the way our world, society, and lives currently work is the best it could possibly be, because it seriously sucks. There's got to be a way to improve and simplify the system incrementally.

Also, I dispute the idea that simpler societies have lower absolute quality of life in every way. Having lived part of my life with an isolated tribal society, there are definitely pros and cons. (A close knit community is one huge pro that I really really miss.) Instead of a blind "our complex systems are better just because they are" I think we should step back and take actual stock of the costs and rewards and really ask ourselves, for each individual thing, whether it's worth it.

>Also, I dispute the idea that simpler societies have lower absolute quality of life in every way.

I didn't argue this. I only argued that our ancestors didn't live as long on average as we did.

Nor did I argue that the way things currently are is the best it could possibly be.

I think there is a way for us to improve incrementally, and in some cases that means simplification. But if that simplification does not respect the root causes for the initial complexity, the resulting system will be just as flawed as the current system. That is my point. Complexity is a symptom of the underlying system, and to try to address complexity on it's own without the underlying causes for that complexity will just lead to more pain in the future.

That's the whole point of the document - that we should simplify systems whenever possible, to avoid the dysfunctions listed.

In this context, it's interesting (to me) how different "how" is from "why," in terms of the answers you might come up with... especially (but not the only) when "complex systems" involve people.

You may enjoy this book: https://www.amazon.com/Logic-Failure-Recognizing-Avoiding-Si...

I second Dorner’s book. Very readable and equally insightful.

Thanks for pointing at it!

The author seems to be using a rather narrow definition of complex system without actually ever specifying it. Complex systems (https://en.m.wikipedia.org/wiki/Complex_system) are strange beasts that occur in all aspect of science and technology and fail in spectacularly different ways.

Interestingly, i promise you they do not use a so narrow definition. Dr Richard Cook is one of the leading researcher in this particular space. His definition of "complex system" is used here indeed in that wikipedia definition you provide and the way they fails may seem spectacularly different, but the research in them have shown that they are not so different and all stems from the same "laws of nature".

I would be happy to provide links to the research if you are interested :)

Not OP but I would be interested in other reading you would recommend on this topic.

Linked to a good starting point here. https://news.ycombinator.com/item?id=20382694

Thanks!

Links to original sources are always helpful.

linked to a good starting point here https://news.ycombinator.com/item?id=20382694

Yes I would be keen on the links to the research. Think in an age of big data and AI i think this is relevant.

Linked to a good starting point at https://news.ycombinator.com/item?id=20382694

A good starting point is https://github.com/lorin/resilience-engineering/blob/master/...

My email is in my profile if you want to go deeper :)

This is precisely my point. You're focusing on specifically engineered systems. Complex Systems occur naturally in nature (there's even a whole branch of statistical physics that studies them). So yes, the author might be well know in the specific reliability engineering sub-field, but it's still using a rather narrow definition of what a complex system is and how it fails.

To clarify, there is nothing wrong with using a narrow definition or in discussing a specific aspect of a grander topic. I just wish the definition was made clearer.

I think my point is more that i am not focusing on these at all :)

But the interest in these is more visible because we can modify them and build experiment in them.

A very different sort of beast... https://iopscience.iop.org/article/10.1088/1361-6404/aa5a87

Reply on news.ycombinator.com