I'm with you that a couple layers of the "safety in depth" worked impressively well here. But a bug of this types points to egregiously bad process and standards with the software vendor. And the airlines or FAA should have audits to catch that well upstream of this incident.

The SpaceX rockets launch routinely with automation at takeoff and landing. There is no need for human heroic drama in the loop when the software is AAA grade.

I see this as a near-miss. At my work, there'd be a post-mortem written explaining what went well and what went poorly. Just because not every line of defense was breached doesn't mean everything's peachy.

The first failure was the bug itself - without knowing more about the details it's hard to say more. But one should ask what kind of practices exist at DynamicSource - e.g., compile time thread safety checks, code review practices, testing requirements, rollout procedures - that would prevent bugs like this from getting out into production.

The second failure was that the software failed to validate its own output and fail noisily or generate a warning of some kind. It's safety-critical software - bugs may be excusable, but not giving at least a warning for an unusual output is poor engineering.

The third failure was that the flight staff failed to notice the error until tail strikes happened. Human errors do occur, but we need better systems to assist human operators. E.g., perhaps the flight software itself could act as another line of defense to prompt the pilot when values appear out of normal range.

Yes, in this case the planes weren't anywhere close to crashing. But if the assumption is that bugs are unavoidable, then there needs to be better systems in place to catch those bugs before they cause a major accident. Because the next time we might not get so lucky with detecting the issue and having someone with integrity calling the shots.

Also, no need to start off your comment with an ad hominem - a simple "software errors are unavoidable" would've said just as much.

> How does a "SaaS Founder, Senior DevOps Engineer" get the idea that software errors are entirely avoidable?

It's literally a safety critical system, not move-fast-and-break-things front end web dev code.

Remember the extraordinarily in depth review of their software while their entire fleet was grounded, where each and every individual bug found was reported in mainstream media?

Where they weren't allowed back in the sky until we were all assured everything was fixed?

The bug they're reporting here sounds like they didn't do reasonable testing. And that's the kind of thing that shouldn't be happening with Boeing 737* aircraft especially, after their recent problems. :/