In my eyes self-driving should still be called driving-assistance.
So far there isn't any evidence for this assumption.
When a self-driving car is involved in an accident, it's in the news all over the world.
Human drivers kill or get killed every day, in every country.
Unless you're talking about getting rid of the steering wheel and deploying the current system as Level 5. In that case, yes, interventions should count against it.
Tesla was criticized quite a bit at one point for comparing deaths per Autopilot mile to deaths per all motor vehicle miles. This was a bad comparison because motor vehicles included motorcycles, as well as older, poorly-maintained cars, etc.
Then Tesla released a comparison between Autopilot miles in Teslas and human-driven miles in Teslas where Autopilot was eligible to be engaged. This felt like a much more fair comparison, but Teslas are lenient about where Autopilot can be engaged - just because the car will allow it doesn't mean many people would choose to do so in that location, so there might be some bias towards "easier" locations where Autopilot is actually engaged. There's also the potential issue of Autopilot disengaging, and then being in an accident shortly afterwards.
This is morbid, but I also wonder about the number of suicides by car that are included in the overall auto fatality statistics. If someone has decided to end their life, a car might be the most convenient way (and it might appear accidental after the fact). That would drive up the deaths-per-mile stat for human drivers, but makes it tougher for me to decide which is safer - Autopilot driving or me driving?
So, the hard failure for humans is pretty bad, too, just different. I suspect there’s little overlap on a Venn diagram of the hard failure modes for AI and humans.
Citation needed, as far as I know this is not true at all.
"In the 4th quarter, we registered one accident for every 3.07 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.10 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.64 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 479,000 miles."
Yes, people disengage auto-pilot (or it does so itself), but it is at least plausible to say that in this like-for-like comparison of drivers and vehicles self-driving is at least comparable in safety.
I don't know what disengagement rate is for Tesla, but we do know that early 2019 waymo it was like roughly every 11k miles driven[0]. Given that waymo uses lidar/generally considered to be closer to actual autonomy than Tesla this speaks poorly of actual autopilot safety.
[0] - https://9to5google.com/2019/02/13/waymo-self-driving-disenga...
I think every human would slow down if they see things that they cannot explain. An AI will not.
It's basically the same problem as when an image recognition AI is 99% sure that the photo of your cat shows guacamole.
Current AIs do not have a concept of absolute confidence, they only produce an estimate relative to the other possibilities that were available during training. That's why fully novel situations produce completely random results.
Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too.
https://www.youtube.com/watch?v=ufNNuafuU7M
This was not a hard fail for the AI.
The vehicle was equipped with both radar and lidar. The victim was detected as an unknown object six seconds prior to impact, and at 1.3 seconds prior to impact when the victim entered the road the system determined that emergency braking by the driver was required, however the (distracted) driver was not alerted of this.
Why would the system notify the driver that emergency braking was required instead of simply braking?
With the sensor array available to it the car should have done better, no question.
But to make the claim "fails harder" I would be looking for a clear cut situation where a human would almost definitely have outperformed the AI.
Human eyes do have miraculous dynamic range so we would likely see more. Can we say with 90% certainty that a human would have saved the situation?
Moreover, try this dashcam video: https://youtu.be/typj1asf1EM It's 10 seconds long, and makes the pedestrian look almost invisible except for the soles.
However, when I took that video, both the crossing pedestrians were clearly visible, not vague shapes that you only notice when you're told they exist. So much for video feed fidelity.
Yes, given the misleading nature of the dashcam video I think we can. This was not a pitch dark road lit only by headlights where an obstacle "appeared out of no-where". This was a well-lit main street, with good visibility in all directions. An ordinary human driver would have had no problem identifying Elaine as a hazard and taking the appropriate avoiding action, which was simply to slow down sufficiently to allow her to cross the road.
The backup driver in the car was apparently looking at their phone or some other device and not watching the road at the time.
"According to Uber, the developmental self-driving system relies on an attentive operator to intervene if the system fails to perform appropriately during testing. In addition, the operator is responsible for monitoring diagnostic messages that appear on an interface in the center stack of the vehicle dash and tagging events of interest for subsequent review"
She was looking at a device, yes, but not her phone.
Uber put one person on a two person job, with predictable results.
https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg#Distr...
The rest of my point still stands though.
Just adding this if you think we should somehow give Uber the benefit of the doubt here. They released footage from a pinhole dashcam sensor that is not used by the system, knowing fully well it would be pitch black and send the ignorant masses into a "she came out of nowhere!" chant.
Just like LIDAR would have picked up https://www.extremetech.com/extreme/297901-ntsb-autopilot-de...
And just like LIDAR would have picked up https://youtu.be/-2ml6sjk_8c?t=17
And just like LIDAR would have picked up https://youtu.be/fKyUqZDYwrU
And just like LIDAR would have picked up https://www.bbc.co.uk/news/technology-50713716
These accidents are 100% due to the decision to use a janky vision system to avoid spending $2000 on lidar; and that janky vision system failing.
https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg
The car had LIDAR.
"the car detected the pedestrian as early as 6 seconds before the crash" [...] "Uber told the NTSB that “emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior,” in other words, to ensure a smooth ride. “The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.”" [1]
'Oh well it was very dark' is not a factor in the crash that killed Herzberg
[1] https://techcrunch.com/2018/05/24/uber-in-fatal-crash-detect...
There are many other cheaper LIDARs, even in the Velodyne lineup, but they are less capable.
(Or, more charitably, "oops, somebody forgot that object persistence is a thing" does not excuse the result)
It’s clear that a) the road is well lit and b) visibility is far, far better than the Uber video would suggest.
An ordinary human driver would have seen Elaine & taken evasive action. This death is absolutely Uber’s responsibility.
Looks like this was a hard fail for the AI then. We can say with better than 90% certainty that a human would have saved the situation, probably would have stopped or avoided easily. My mistake.
We can reasonably assume that pja is aware of the existence of abysmal drivers and fatal crashes that should not have happened. I doubt their intent was for "would" to be interpreted as "100%".
Those failures include driving straight into barriers.
Other systems like OpenPilot show the same.
When it fails you better take control of the wheel or you will crash hard.
Still seems like people treat auto-pilot like auto-drive and die as a result.
Thats not a tech fail imho
These are high risk areas, if autopilot is "failing hard" with a regularity equal to or higher than normal than this would be good to demonstrate with stats. Guessing Tesla doesn't really release that info?
https://www.extremetech.com/extreme/297901-ntsb-autopilot-de...