It was well known all along that humans are poorly equipped to remain actively engaged in situations that require split-second reactions, while exerting zero effort or control for hours on end.
Air traffic is as safe as it is because the accident analysis always pushes past "pilot error" as the root cause. If one pilot can make the error, why won't another? Did the system set them up to fail? Even if it didn't, could the system be enhanced such that the failure (or class of failures) could be prevented?
In the case of the human in Uber's self-driving car, they were set up for failure. And in such a well known way that makes any claim Uber was not negligent very hard to defend.
> According to Uber, the developmental self-driving system relies on an attentive operator to intervene if the system fails to perform appropriately during testing. In addition, the operator is responsible for monitoring diagnostic messages that appear on an interface in the center stack of the vehicle dash and tagging events of interest for subsequent review.
I think if you are going to pay someone near minimum wage and let them hold onto their phone while in the car, you know they are likely to use it. Even if the request is reasonable, you know minimum wage workers are going to do that sort of thing if not monitored via an in-car camera directed at the driver's seat.
The technologies are not ready. They are being tested. Aircraft get to such a safe state like today after so many iterations and improvement, both in technology and process.
A terrible tragedy happened in this case. Hopefully it will help save a lot of lives in future.
https://www.consumerreports.org/car-safety/where-automakers-...
Absolutely. We know from centuries of evidence that humans are profoundly bad at monitoring 'mostly autonomous' systems. Waymo/Uber know this as well, they just employ the driver as a convenient way to offload legal liability while they test unproven and unsafe technology on the public without consent.
You can pass all the prerequisites, but none of that guarantees that it doesn't fail at the real world testing phase.
All that said, in this specific case, Uber absolutely hadn't passed all the prerequisites and shouldn't have been given any permission to be on real streets.
Responsible engineers deal with this in a number of ways, such as the test images TSA screeners see throughout the day, Waymo’s reported careful monitoring of the drivers, or redesigning the process so the human is continuously engaged. Uber engineers apparently say YOLO! and disable the car manufacturer’s own safety systems.
This is honestly what bothers me most. Firstly, I expect self driving to have massively more information at hand than humans do. I'm talking long range, 360deg sight. I expect multiple sensors to be recording at all time, analyzing. And finally, I expect a "black box" to be able to visualize all of this for cases like this.
Visualizing is something I think needs to be black-box level standardized. It's what we need to hold them accountable, I think. We're dealing with proprietary implementations - so humans should be able to review black box "footage"[1] and decide if the scenario was handled better or worse than a human.
There's going to be times where this system fails, and I'm not expecting perfection. Yet, humans in non-private positions need to be able to regulate how Uber and friends are behaving on the road.
A short sighted grainy video that looks like it was from 1990 is not cutting it. Fully unacceptable in my view.
[1]: I say footage loosely, because I imagine a lot of the scanned data is just that, data. So it needs to be rendered in a human friendly, visual manner, fully understandable by lawmakers.
Just because one party is neglicent does not mean other parties are cleared of their neglicence.
Waymo specifically monitors the driver plus checks their attention span before hiring them. There's various things which went wrong for this to happen. It's not just one cause.