So, let the two software systems fight with each other during emergencies and hope it somehow results in something positive?
That sounds like a terrible idea.
That sounds like a terrible idea.
Independently-working override-capable systems are the base of engineering redundancy safety. See airborne collision avoidance systems (ACAS), which will automatically and forcefully steer an aircraft to avoid a collision if necessary: https://en.wikipedia.org/wiki/Airborne_collision_avoidance_s...
Redundant safety systems are a great idea, evidently Uber needed more of them, and integrating with the Volvo system might have been a reasonable option. It's silly to suggest that the integration would necessarily have been trivial, though. That's what I'm objecting to.
(Were any Professional Engineers even involved in this? Or was it just a bunch of "software engineers"?)
There is no "guarantee" that uber "engineers"/engineers put more thought into this than "their system is inconvienent, tear it out" or "we don't need their system because ours will be better, tear it out." Nobody can guarantee Uber engineers were not stupendously negligent until the investigation is complete. Anybody who thinks they can guarantee that has an irrational basis for thinking they can provide such a guarantee.
The above traits aren’t exclusive to Uber’s “engineers”, but are lethal when applied to the engineering of life safety systems.
A lot more than one, that's for sure.
See how that holds up in civil court in front of a jury, or any legislative body Uber might need to convince to allow their operation in a jurisdiction in the future.
“Just think of how many more people we would’ve killed if we didn’t care at all!”
I had thought an audience of developers would "get it," since we deal with fallout from ill-conceived integrations every day, although admittedly in a far less spectacular form than control system engineers.
Unfortunately, the uber hate train has already left the station and there's no slowing it down until the investigation finds (or doesn't) actual evidence of negligence rather than clickbait guesswork by armchair engineers. Too bad.
You do a disservice to the audience by assuming it would be understanding of grossly negligent behavior.
Failures happen, that is to be expected. If you're building self-driving vehicles, you're supposed to be engineering for those failures. Disabling two life safety systems (Volvo's AEB and Uber's own AEB) and relying on a single inattentive human driver? I don't understand how that's understandable or justifiable in any scenario besides a carefully controlled test track.
If any of the systems are vulnerable to any such false positives, and equipped to enable emergency braking to avoid them, even on the highway, it's not hard to imagine why they might be disabled, especially during testing phases.
I think it's fair to say that at this early stage in product development, there's probably no 'always right' answer for how to handle a given obstacle without considering locality, driving speed, road conditions, likelihood of false positives and negatives, etc.
Hey there's an idea for Uber, maybe instead of disabling the forward collision system entirely they could just decrease the sensitivity to lessen false positives (like in our Jeeps)?
Instructions from TCAS are near absolute in their priority. If ATC says to do something different, you ignore ATC and do what TCAS says. If the pilot in command says to do something different, you ignore the pilot in command and do what TCAS says. If somehow God Himself is on your plane and tells you to do something different, you ignore Him and do what TCAS says. Compliance with TCAS is non-negotiable, and the Überlingen disaster[1] is the bloody example of why it's that way.
Self-driving/autonomous-car systems need to have a similar absolute authority built in. If Uber disabled theirs because of false positives, it's a sign Uber shouldn't be running those cars on public roads.
[1] https://en.wikipedia.org/wiki/2002_%C3%9Cberlingen_mid-air_c...
Somehow, it works out.
The point is that so far most (all?) of these computer systems don't have a failure mode where it kills pedestrians, because their scope is very limited.
Same with power steering: it won't steer into a different directorion. You don't have to "fight" with it.