In 2009 an Airbus A330-200 crashed into the Atlantic ocean killing more than 200 people because a confused pilot kept pulling the plane up, despite loud stall warnings from automatic safety systems [1]. In the discussions of that event, one commenter asked, "Why did [the plane] not override the pilots' inputs and force a pitch-down?" [2]
Designing safety critical systems is difficult. If the automated system fails, the pilot needs to be able to take over manual control. But at the same time, if the pilot does something stupid, the automatic system should prevent that.
I'm not sure how to square that.
[1]: https://www.telegraph.co.uk/technology/9231855/Air-France-Fl...
Since you already have allegedly redundant pilots and alarm systems and hopefully sensors, the remaining part is figuring out which failure detector has ultimately failed to work and why. Also if redundant pilots are failing, it's a systematic failure in training, fatal to airlines and whoever designs training simulators.
The most potentially fatal to Boeing scenario is a systematic failure or negligence in terms of redundancy. Hampering pilot training is bad enough. Another potential failure point is a software issue which means negligence in safety critical software engineering.
The lack of stall warning meant two different things: "not stalling", and "stalling so extremely it can't be sure, so don't report it"; as you say. I wondered if there should be a different sounding alarm for "can't determine". Or, the pitch of stall alarm could rise, such that pilots could tell "which side of the stall" the plane is on, so that entering stall from one side would side start with a very high pitch or very low pitch. (I'd guess there's a better way..)
So, in this case, it's more subtlety more tragic than you describe, because the pilot did correct the AoA, and nose down, but when he did that, the stall warning resumed (albeit from the extreme other "side" of the interval), confusing him: no alarm, alarm resumes, so he thought he "re-stalled" it and he pulled up again. This kept him mentally "stuck" in the extreme stall. They needed to "pass right through" the stall warning window, and come out the other side. They had sufficient altitude for this to succeed.
Even the most heavily analyzed aircraft can produce surprises once they actually take to the air. Look up "Super Hornet uncommanded wing drop". Or look at dynamic scaling tests (https://www.nasa.gov/pdf/483000main_ModelingFlight.pdf) which uncovered two spin modes in the F-15 that were previously entirely uncharacterized. (Not sure if the linked doc covers that instance specifically -- I learned this from speaking with the study PI. And no, most aircraft programs don't get to do dynamic scaling tests.)
All this is to say that a variety of potential non-linear behavior is lurking inside any given airframe, and all the wind tunnels and CFD in the world will merely allow you to model most of the useful operating conditions, but certainly not all of them.
Simulators, just like autopilots, are useful for finite input ranges. If you leave those input ranges, your simulator will precisely and repeatably model a fantasy, and your autopilot will disconnect.
Source: Short career as an aerospace engineer.
The human, by and large, figures it out. We generally accept that risk. But how will the autopilot react to the unexpected event for which you haven't trained it? Who knows?
1. Why did they not inform operators about changed system behavior?
2. Why wasn't the system capable of detecting sensor failure and/or act accordingly? In the automation industry one goes to great length to be able to tell good sensors apart from bad ones. Why was that not the case here? If it is not possible to tell bad sensors from good ones, why wasn't there (sufficient) redundancy (=multiple sensors) which at least deactivate the system on conflicting input?
There are two AoA sensors. However, good question - why aren't there 3 (so you can have majority quorum in case of one failure), and why wasn't automation disabled once a discrepancy between the 2 existing ones was detected?
Did it disable the ABS? No. Instead, it continued to rely on the faulty sensor information, activating ABS under moderate deceleration on dry pavement, greatly reducing the available braking force and creating a severe danger.
I think there may be a bias among people designing safety equipment to always try to provide the intended safety benefit, even when part of the system isn't working. The problem is it's easy to lose sight of how the safety feature fits into the overall picture. A pilot can almost always safely fly a plane that doesn't use automation to recover from a stall, but not necessarily one that uses the trim to point the nose at the ground in spite of control inputs to the contrary.
This 'roads are so dangerous' is part of the self driving crowd scaremongering program. That 'humans can't be trusted' but their tech can obviously.
But when looking at the global picture and the sheer number of cars in all sorts of traffic conditions on roads everyday without incident this narrative falls apart.
We need far more facts, data and critically 'contextualization' than sweeping disingenuous statements by parts of the tech community motivated by self interest and greed.