Because large systems are complex and have very complex and hard to detect failure modes.

What is more impressive is how rare these events are, given the complexity of the underlying systems. Redundancy is not without its own problems (source of truth, for instance).

A lot of times large scale outages like this are because of the redundancy. The whole system is interlinked with automatic failover. then it hits a corner case that was not engineered into the fail model and you get cascading system failure where each node starts bringing down other nodes automatically. basically the lesson is: In highly interlinked systems you get highly interlinked failures.

And then after a lot of angry words and finger pointing this new failure gets added to the failure model.

My personal takeaway after chasing the long tail of automatic failover on a few projects, is quite often it is better to drop a few 9's from your service goal, decouple some of the systems, and accept that while some parts of the system may go down, it should not bring everything down with it.

The US air traffic control crash at the start of the year turned out to be a decades old system which had not been upgraded, I think the UK one is a little bit more recent, but like alot of software, people dont want to pay for the real costs of software developments.

Complexity tends to hide problems. An over-engineered system is going to be less stable than a simple one where you know how things can break and how to bring them online pretty quickly.

They agiled software to the point where it’s written by product managers and wanna be engineers that cargo cult “good practice” without understanding what it’s meant to do. This types of issues are common in british made software.

System redundancy rarely covers software faults.