Software bug made Bombardier planes turn the wrong way

It wasn't clear to me what the exact problem was from reading the article, just that it occurred under a very specific and uncommon set of circumstances.

But if the missed approach or departure procedure is wrong the crew has a high probability of not noticing it (this all happens in a very high workload situation). If you don't notice you can't fix it. What makes it worse is that this bug happens in situations whether the procedure requires a turn "the long way around", they wouldn't design the procedure that way unless it's really necessary. So there is a big chance there is terrain or an obstacle on the other side.

Source: I'm a commercial pilot

This hiring problem is compounded by the oversight problem. The program managers are similarly inexperienced. Or they came from strictly a testing side with no concept of what software development itself entails (I've seen this a lot). So they aren't bad at managing requirements, they may actually be really good at it, but they absolutely fail to understand that software is a hard problem (especially when dozens of subcontractor are involved) that extends beyond just the technical problem, and to the communication and coordination problem. That's assuming they're experienced, USAF program managers for software (IME) are straight out of college history majors. DoD programs are scary.

[0] Most avionics systems, in my experience, boil down to rather straightforward state machines. Understood this way they become much simpler to write and test. The hard part is hitting your timing constraints, but that's easier to achieve with correct-but-slow-and-maintainable code than with incorrect-but-fast-and-unmaintainable code. Inexperienced developers won't see this possibility, either by failing to spend time studying the requirements or failing to understand how to implement state machines at all.

There are two reasons why you can expect software to be more and more the cause of airliner safety issues:

- software is eating the world

- software is getting more complicated

The first is a long-term trend now. Look under the hood of any automobile from before the 80s: no computer to be found. Look under the hood of any automobile from the past 30 years: computers abound. The reason for this is that many problems are easier to address in software than in hardware. Of course, you go from N hardware problems to some possibly smaller set of possibly simpler hardware problems at the cost of gaining a set of software problems -- but this trade-off usually pays off. In some cases this trade-off enables functionality that would be infeasible to create otherwise.

The second problem is also a long-term trend: CPUs, systems, operating systems, and applications have all tended to get more complex. In embedded systems the trend has been less strongly towards ever-increasing complexity, but even in embedded systems things have gotten more complex.

Whether the problem is less competence among today's programmers is hard to establish here. First, we need much more software, which means we need many more programmers, which means the quality of programmers you get probably does decrease, though then again, we do have more programmers overall as more people (competent and otherwise) are attracted to the industry. But more importantly, the increase in complexity of today's systems could very well be enough to make yesteryear's competent programmers incompetent today -- you can't really compare software development 40 years ago to software development today.

(I object to this idea that lower-wage programmers necessarily can't be competent, though that isn't quite what you wrote. It's true that a lax process for outsourcing can mean you get less competent programmers, and it's probably true that higher GDP/capita correlates with availability of competent programmers. But it doesn't follow that there are no competent lower-wage programmers in India, say.)

Many things that used to be done by analog computers or manually done by the pilot are now done in software.

In addition, the Boeing MAX was a largely system design issue; the software was operating as-designed, and had it been implemented in hardware, it would have likely failed in the same manner.

Theres no surprise that's where most of the failures occur.

This is pretty good intuition but often a systemic change is not economically feasible. For avionics software at least, a rewrite of the software would likely have to be recertified from scratch before it would be allowed to fly.

We do, however, have several different quality assurance programs in Aerospace that are supposed to address this sort of thing.

Once you identify the root cause, the process found to be deficient is supposed to have a Process Owner who is required to create a preventive and corrective action plan to prevent a recurrence, with more severe problems requiring more robust action plans. Done right, the process owner is supposed to be empowered to make the changes that need to be made.

These systems tend to be evolutions of ISO 9000 as pioneered by Toyota (IIRC). They are highly bureaucratic and soul-sucking, but they are also the least-shitty solution that's been tried.

You should also keep in mind that real systems have fault modes aside from software bugs and hardware glitches, such as unanticipated edge cases and user error, which may dominate your actual failure statistics.

The difference in reliability between normal software and airplane software is so vast that "best practices" from normal software can not be applied to airplane software since that would be gross criminal negligence. To explain, in the 10 years prior to the 737-MAX problems there were 50,000,000 flights and software was not implicated in a single passenger air fatality. The average flight is ~5,000 KM which is ~4-5 hours. So, in ~250,000,000 flight-hours, there were two crashes due to software. A plane takes ~3 minutes to fall from cruising altitude, so we can model this as a downtime of 6 minutes per 250,000,000 hours which gives us an downtime of 1 in 2,500,000,000 or a 99.99999996% uptime (yes, that is 9 9s). In contrast, I think most software people would agree that AWS is high quality. The AWS SLA specifies a 99.99% uptime (1 in 10,000 downtime). So, by this metric, airplane software is 250,000x more reliable than normal high quality software.

The point of this is that the standard for airplanes is almost inconceivably high compared to normal software. To think that they are incompetent or suggest that all they need to do is adopt X or Y common-sense/best-practice is a gross misunderstanding of what is being done and what needs to be done to improve. It would be like someone trying to tell a civil engineer making a 50-story skyscraper that they really need to adopt high quality wood construction techniques from makers of doghouses. To actually improve it, you need to consider practices 250,000x better than "best practices" and go from there.

To put it another way, the solutions are actually really really good, unfortunately the problems are really really really really hard.

The Spectre/Meltdown issues are deep and architectural, not simple to fix. It's not just a batch of CPUs that's the problem, but all of them.

Besides, if a CPU ships with a bug that can be fixed via a microcode patch, then it would be a tremendous economic waste for all humanity to throw those CPUs out.

Even when new CPUs come out that can be shown not to have Spectre/Meltdown issues, it will take a long time to replace the installed base of those that do because it's not a matter of a little bit of money, but a matter of a great deal of money and opportunity costs.

So microcode patches and software mitigations is all there is. Absolutist attitudes don't help.

This sounds pretty good in theory but in practice you will just trade the current set of issues against new issues.

In reality, Systems and their interactions are so complex that there is no amount of software design that can avoid bugs and fixing them. We sure can improve but it would be naive to think you can design 100% reliability into something like an airplane.

In a way every real software improvement (not fancy language flavor 'x' of the year but entirely new ways of developing software) have always been with the main goals of writing software with fewer bugs faster.

That's the whole reason we have abstractions, compilers, syntax checkers, statical analyzers and so on. In spite of all those, software still has bugs and budgets are still not sufficient to write bug free software.

On another note: this problem is getting worse over time. As tools improved codebases got larger and the number of users multiplied at an astounding rate resulting in many more live instances of bugs popping up. After all, software that contains bugs but that is never run is harmless, only when you run buggy software many times does the price of those bugs really add up.

Somewhere we took a wrong turn and we decided that more of the same is a better way to compete than to have one of each that is perfected and honed until the bugs have been (mostly...) ironed out.

If you’re trying to keep planes from crashing at all costs, sure. If you’re trying to reduce deaths from travel, that’s a terrible plan. Every family that you price out of commercial air travel and convert over to private auto travel instead has been placed at significantly higher risk as a result of the excessive pursuit of safety.

It’s the reason the FAA allows lap infants under 2 years old. Not because that’s “safe” in absolute terms, but because it’s safer than the likely alternative.

[0] https://www.flyradius.com/bombardier-q400/fuel-burn-consumpt...

[1] https://www.engineeringtoolbox.com/co2-emission-fuels-d_1085...