What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost.
I really think folks don’t realize how critical all of our human understanding is to driving.
A vehicle coming from a side road has a lot of movement within your field of vision. When driving on a highway we're well used to the fact and expectation that stuff isn't stopped. When the vehicle ahead emergency brakes or stops you have the growth of size of the target to gauge by. That's not much movement, until the last moment when it's far too late to evade. Some stationary debris is much easier - it's not meant to be there at all - you transition to evasion right from the get go.
For a human in a car, the dangerous highway case is the stationary vehicle in the fast lane, maybe one or two vehicles ahead. When flying it's the other aircraft that's on a collision course. Neither will move within your field of vision. Neither is easy to detect as problem, until it's very late.
FWIW 4 year old humans often run into things. :)
That humans get distracted and often make mistakes is no surprise, it should also be obvious that the amount of risky behavior is correlated to the amount of _perceived_ risk in any given situation, which is probably why so many are "a simple crash."
It is, however, somewhat scary that a machine that has nothing else to do other than to drive the car as safely as possible still makes these "dumb" mistakes.
Cruise control makes dumb mistakes.
I know people who think automatic transmissions makes people worse drivers.
Well, it's certainly a bit trickier to to respond to texts or browse instagram when you need two hands to operate the vehicle.
Well, thats half of the problematic drivers, and i suspect texters as well. /s
Humans get inattentive and distracted, and humans follow too close for current conditions and speed. There's no excuse for either of those factors to apply to an autopilot system. If they do, it's badly designed.
> Except humans crash into stationary vehicles in lane all the damn time.
And Tesla's argument is that Autopilot is far safer than humans.
In the 2nd quarter, we registered one accident for every 3.27 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.19 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.41 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 498,000 miles.*
*Note: Since we released our last quarterly safety report, NHTSA has released new data, which we’ve referenced in this quarter’s report.
Whatever is the reason (may it be better road conditions) when you use Autopilot you are less likely to crash.
To wit, if Autopilot is unavailable or disengaged because of poor weather/road conditions, a human still has to drive those miles. Autopilot gets the _best_ conditions to show its abilities. Humans get all conditions to show their abilities.
The question is "how did human drivers do driving the same miles, at the same times, as Autopilot", which is impossible to calculate.
The question is: Are you willing to accept only 33% less likely hood of an accident with autopilot engaged as compared to when you drive on your own in not Autopilot safe conditions?
Do you use Uber knowing that their average driver will be ... well ... average, or just somewhat better than average? Knowing that the average driver baseline is 500 000 miles per accident?
https://www.technologyreview.com/f/612346/uber-and-lyft-are-...
None of which changes the difficulty in detecting a stationary vehicle, that's in your path, as quickly as you would like.
You see the brake lights up, and as long as you're alert and keep some distance you're going to be alright. If you're not alert and/or driving too close you hit it.
Computers look at the world in a different way. So they are going to have accidents that seem very obvious to humans making people go "look how bad driverless is because a human would never...". Many people will take that as a reason to avoid driverless tech.
Yet driverless will (or should) avoid many more accidents humans would have made if they were in control.
Because of this we need to look at overall statistics. If we only look at specific driverless accidents from a human lens it's going to be easy to non-lateral thinkers to get caught up in criticising but it doesn't actually understand the benifits or not.
The failure modes of these systems need to be much more gentle: when I'm confronted with an uncertainty while driving, I slow down and pay more attention. I don't speed up until I'm confident that a higher speed is reasonable. This is far far more sophisticated than simply "can I see something in front of me" -- I'm evaluating the probability that I need to stop based on the events I can see happening. If I see a car in front of me slow down and swerve out of the way of something, that means something much more than "obstacle detected" -> "obstacle removed".
Why not? Statistically, you're safer with an autonomous system. Unless you're sure you're a much better driver than average, your position is irrational.
I simply don’t believe the “safer than humans” narrative is being honest with the number of caveats.
Until self driving car executives are willing to send their children into the street in a real life game of frogger with their cars being driven by a bunch of drunken frat boys in a blizzard, they shouldn’t expect the general public to blindly agree to be their guinea pigs either.
This is not the whole equation. The question is also would you be able to react to all accidents that Autopilot is avoiding? Are you 100% diligent ALL the time? Can you react in 200 ms ALL the time to someone else stupidity? Last-minute breaking for the exit, veering into your lane, speeding trough crossroad on red etc.?
https://www.youtube.com/watch?v=P68XGJticuo
https://www.youtube.com/watch?v=FrJ2uPRRtz0
https://www.youtube.com/watch?v=QVdTAwU07Jc
https://www.youtube.com/watch?v=9cFJI6Qf9GA
> I expect a sober human driver to have a much lower accident rate than Tesla’s autopilot when driving the same highway miles.
This is based on what?
What about Tesla miles without active safety features vs. Tesla miles with active safety features?
Based on Tesla safety report just having active safety features active without an autopilot is decreasing the risk of collision by 36%. As far as I know, both numbers include miles out of highways.
"For those driving without Autopilot but with our active safety features, we registered one accident for every 2.19 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.41 million miles driven."
https://qz.com/1414132/teslas-first-accident-report-claims-i...
Tesla’s autopilot miles are primarily driven on highway miles (in low speed, bumper to bumper Bay Area/LA traffic), in new cars (with well engineered safety cages and multiple airbag systems) which are well maintained (Tesla owners are unlikely the be driving on bald tires and experience blowouts), unlikely to rollover (one of the most fatal accident types) due to battery weight distribution. The car, without autopilot, is much less likely to experience fatal accidents. Couple that with the type of autopilot miles driven, and I expect sober human drivers in the 30-69 age bracket would have been less likely to experience accidents in those same miles.
I expect Tesla is intentionally trying to cloud the statistics to build the public narrative that self driving cars are safer. I believe that Tesla truly expects them to be safer eventually, but they’re smart enough to have the numbers on equivalent human driver crash rates... I’m sure that data was compiled by a very energetic Tesla engineer, put into an power point deck and emailed around. Perfect for a lawyer to subpoena.... and prove that Tesla knew their product wasn’t.....
And Tesla isn’t the only self driving aspirational car company pushing the “self driving cars are safer narrative”. Even Waymo (or maybe the media) claims their vehicles are involved in fewer accidents per mile... but they’re not self driven cars. They’re self driving cars with a human overriding the car’s decision every ~10k miles. Human drivers have an accident approximately every 500k miles (eliminating the under 25 and over 70 age bracket, and non-sober drivers - I expect that number to be closer to an accident per 500k-1M miles). So, a self driving car, without a human, would have been in some type of bad situation 50 times in those 500k miles a human would have driven with just one accident. My suspicion is that in a high fraction of those 50 disengagements, there would have been an accident had a human not intervened. My real suspicion is that self driving cars without human supervision are 50-100x worse than humans.
Lawyers are the only ones who are paid to figure out if my statistical assumptions are correct and to bring them to light.
And yet, I drive with Tesla Autopilot every day. Why? Because I believe that on a portion of my commute, it’s absolutely a better driver than I am. That portion of my commute is straight high speed traffic, periodically interrupted by people slamming on the brakes. It’s boring and mentally draining at the same time. I wish everyone on that portion of the freeway was driving with Tesla autopilot, as accidents are the major cause of my commute delays. Those 20 miles of new, well maintained, straight, properly marked freeway miles, driven during perfect weather, not into the sun, are ideal for Tesla autopilot. Nobody should take those ideal miles and extrapolate them to the rest of my much more dangerous commute.
I think we generally agree on our understanding.
See my other answer:
As I said, whatever the reason, there are fewer accidents with Autopilot, than without. Autopilot is a convenience feature.
The question is: Are you willing to accept only 33% less likely hood of an accident with autopilot engaged as compared to when you drive on your own in not Autopilot safe conditions?
------
"My real suspicion is that self driving cars without human supervision are 50-100x worse than humans."
Yes, Waymo in their fully mapped safe heaven may be around that.
Teslas in cities are probably more like 1000-100000x worse than humans, I don't think they would be able to drive even 100km in random city traffic, like a taxi, without an accident. They would probably be super eager to get rid of the steering wheel if they would think that Teslas are better unequivocally, all the time, no matter the conditions. Maybe the next iteration will improve it 10 times, but they are a long way from self-driving taxis.
Precisely. Autopilot has the luxury (need) for disengagement in suboptimal conditions. Human drivers don't have that. So Autopilot stats self-select for ideal miles. Of course, we don't have as concrete metrics for "how humans drive in clear dry well marked highways" to the exclusion of crappy conditions.
Tesla's "autopilot" collisions are all very similar. They're all "hit big stationary obstacle that wasn't the rear end of a car". So far, a fire truck, a street sweeper, a stalled car at a freeway median, a crossing semitrailer, and two freeway barriers. This is a well-defined defect.
As far as I'm concerned, there is only one criterion that matters when evaluating self driving cars: is it better than human drivers?
The fact that we are still discussing this _one_ accident that happened _a year ago_ indicates to me that the answer is a resounding yes.
We don't really know how well the Tesla automation does vs human drivers, because the statistics are complex, and Tesla only gives us deceptive summaries.
The capabilities that Tesla cars have aren't that much different than other luxury cars, but the other manufacturers don't have NTSB investigations, because their marketing isn't convincing people the car can drive itself.
It's actually somewhat impressive, if frustrating, how well Tesla's marketing works.
In a previous article here a couple of weeks ago, a Tesla owner was touting "how much better" Tesla's blind spot monitoring was than "other manufacturers", because it took speed differentials into account, while my 2015 Audi A4 did exactly the same thing. If you were faster than someone in your blind spot, it wouldn't activate. If speeds were similar, it'd activate at close range. If they were vastly outpacing you (I'd see it when adjacent to a HOV lane), it'd activate well within a safety distance to factor in speed differential.
While there are "dumb" Blind Spot sensors, multiple Tesla owners were of the fervent belief that this was an example of how unique Tesla's technology is.
I agree. But do they when alert and focused and when they clearly see the object? I highly doubt it.
This Tesla was all of those things, and still crashed.
Stop acting like it's all or nothing. The technology behind autopilot is being touted as driverless/self-driving, but it can do passive accident avoidance. That should be the benchmark to compare too.
In this particular case, humans get distracted and get in an accident. Autopilot is dumb as shit and gets in an accident. But a human with an emergency braking system is smart and doesn't get in an accident on the account of driver distraction.
That's your Level 1 example. Yet Level 1 systems are not banned.
By that measure, no tech company has contributed more to self-driving than Facebook Inc, masters of distraction. Relative safety isn't a good metric.
You can make moralistic arguments about why you don't like this or that, but at the end of the day you are causing unnecessary deaths when you choose to suppress a system which is better than human drivers.
The bar is much higher than that. People accept people killed by other humans suddenly becoming ill or briefly not paying attention. We accept human error. We do NOT accept machine error.
So the criterion is that autopilots should avoid a lot of accidents that humans risk getting into (e.g. lack of attention) but it must not be involved in ANY accidents that a human paying attention would get into.
In the end, no one really cares about the overrall safety or the statistics of it all, when it comes to acceptance of new systems. What matters for public perception (so in the end legislation) will be an almost impossible bar for autonomous vehicles, because they don't have the benefit of acceptable human error that we give flesh drivers.
I do disagree about the mechanism. I don't find that humans are very generous to one another - public opinion very often assumes the worst and takes the least generous interpretation. People are more than happy to vilify a reckless driver who caused a death.
I think the mechanism here is media coverage. We are still seeing articles pushed out by media corporations about this one accident from a year ago. Meanwhile, human drivers kill thousands of people a day and you don't hear a peep about it. So naturally public opinion becomes scared of self driving cars and ignorant of the real issue.
All of that is really tangential to my point, though. My point is that "feelings over facts" is grossly immoral when ignoring facts causes tens of thousands of preventable deaths each year.
It takes a long time to do an investigation. This is one of a number of incidents. You should try actually reading facts instead of Elon's Twitter.
Hell, I get confused sometimes in driving SF - flakey drivers, aggressively solipsistic homeless folks crossing (or just standing around in) the street randomly, bicycles, shopping carts, dogs, kids...
I want to see the robot death boxes competently handle less fraught environments before allowing them around humans.
And industry suggestions that the humans not in robot death boxes cede the streets to those who are? Ah, no. Cars own practically everywhere else - you can go play with your toy on the freeway or drive the damn car yourself around other humans.
False positives are dangerous. Trying to detect ‘Anything’ may result in more deaths than hitting the tiny number of exceptions.
Alternatively, the system would be overly flakey and hard break all the time.
Self driving cars need to have a threshold before hard breaking. Hard breaking is risky, it can cause serious accidents. Reducing that threshold thus increases some risks even if it lowers others.
So, now we need to balance a risk of say ~1/10,000,000 per car per year vs what? I don’t know what changing those thresholds would result in, but I do know it’s not free and there would be false positives.
As software engineers, we can understand the hidden costs in complexity and bugs when implementing features that don't need to be implemented. I could imagine that false positives for bolder avoidance causing more harm than false negatives.
Obviously it should be smart enough to detect any and every obstruction, but I'd rather the 99% use case get ironed out first.
as someone who has driven all over the west , all the time.
You have New Mexico and Arizona that believe that steep incline cliff faces on both sides of the highways are a fashion statement, states like California, Washington, and Oregon that like to build new properties on rapid-eroding mountainsides, and states like Nevada that simply don't patrol sparsely populated highways -- and are simply ignorant to the boulder some kids rolled into the street some weeks back.
All anecdotal, but i'm a fairly heavy driver and I see them commonly.
I agree that false positives are to be avoided -- but that doesn't really lessen the value in boulder-avoidance ; it just brings up yet another point to worry about with the system as a whole.
And worse (heavy cynism ahead): they would make the car look stupid even when not causing an accident, whereas boldly plowing into whatever might or might not be an obstacle makes the car look strong and confident, right until it looks all bent and crumbled. Car sales are incredibly marketing driven and a self-driving implementation that routinely chickens out of ambiguities that look easily solvable to humans would be a worse marketing problem than twice the number of accident investigations Tesla is facing.
Similar to how cruise control (Level 1) doesn't know how to drive a car. Yet people are willing to put up with the concept that it causes some accidents.