Tesla's 'Autopilot' Will Make Mistakes. Humans Will Overreact
bloomberg.com
bloomberg.com
Tesla has two basic problems with their "autopilot". One is hype, and one is bad technical design.
Tesla's "autopilot" is just automatic lane keeping and automatic cruise control. Mercedes and BMW have been shipping that for years. The other car companies have hardware to check if the driver's hands are on the wheel. Tesla not only didn't do that, they hyped those basic functions as being an "autopilot", as if they had something comparable to what Google has. Tesla has significantly overhyped their technology, to a dangerous level.
Here's another Tesla crash, from yesterday.[1] This is a Tesla rear-ending the corner of a stalled van on a highway. This may be an "automation in wrong mode" error, and complicated excuses from Tesla will probably be forthcoming. That's not good enough. Airline pilots get into "wrong mode" errors, and they get extensive training, including simulator time. Drivers don't get that. This driver presumably thought the automation would do something sensible in that situation. It didn't.
The other big Tesla problem is a pure technical design error - the bumper-mounted radar is blind at windshield height. This is the immediate cause of the fatal accident involving under-running the semitrailer. It's also the cause of the parking accident with the truck with overhanging load sticking out the back.[2] Tesla's implementation has deadly blind spots. They need a radar at windshield-top height. It's quite likely, now that the NHTSA is looking into this crash, that Tesla will have to do a recall and retrofit one. (Along with hands-on-wheel sensors.)
Tesla was relying on stereo cameras too much. Depth from stereo is always iffy; there needs to be some detail on the target to get range. You can't range a uniform surface with stereo vision. (Human flyers have the same problem flying over ice.) A big white truck under some lighting conditions has that property. Cameras are useful, but backed only by today's algorithms, not enough. Teslas come with a sensor suite inadequate for the job.
From a software perspective, Tesla's software seems to be overoptimistic that there's no problem ahead. 3D vision systems know when they don't have range info for a large area. Tesla's software was willing to drive into that. It didn't note the absence of range info for the road surface; it may not even be profiling the road ahead. (We all had to do that in the DARPA Grand Challenge because we were operating off-road. That's a solved problem. Google does it with their roof-mounted LIDAR.) Tesla owners have asked for "pothole detection" as a feature. Lack of this may eventually take someone off a cliff.
The press is gradually picking up on this. Until recently, Tesla's autopilot was treated as a magic black box in the press. Now some automotive writers are starting to ask the right questions.
This is not a setback for automatic driving. This is just a Tesla problem.
[1] https://www.youtube.com/watch?v=qQkx-4pFjus [2] http://www.roadandtrack.com/new-cars/car-technology/news/a29...
> [1] https://www.youtube.com/watch?v=qQkx-4pFjus
minor nitpick but this video is from May, this didn't happen yesterday, did you mean to link another video?
And this is the same problem mentioned elsewhere in this thread. The adaptive cruise being 99.99% effective at stopping trains the driver that it will always work. Especially because Tesla doesn't force the driver to touch the steering wheel, they seem especially susceptible to this issue - and drivers do things like this: https://www.youtube.com/watch?v=5fDbfy0409U&t=0s
This is not an issue until it is.
One way to do that might be that if the car successfully avoids an obstacle on its own without the driver reacting at all, after the obstacle is passed the car pulls over and stops.
> This is not a setback for automatic driving. This is just a Tesla problem.
If the general public doesn't get the distinction it can become an automatic driving problem.
You want to make a bet on that?
This is just completely false. We've been putting them at bumper height for 10 years because that's where the system ends up working best with the greatest number of targets to see. NHTSA has been regulating these sensors for years and designing the tests for them; they are not new to this. Tesla is not going to have to do a recall on these, and spare me the idea that they are going to "retrofit a sensor at windshield height." This is just mindless speculation you made up.
Who's "we"?
Do you have a source for this? It was my understanding Google went with their roof-mounted approach because of the blind-spot issue. (They also went with Lidar over cameras because of the white-truck-in-bright-daylight problem.)
[1] http://www.pcworld.com/article/2025386/volvo-v40-watches-the... [2] http://www.fujitsu-ten.com/business/technicaljournal/pdf/38-...
Let's all hope that's how this is remembered 10 years from now (or whenever) when automatic driving is actually ready.
It's not a given that people will remember this as a Tesla-only problem. Instead it might be remembered as that time when autopilot got people killed.
And when it's actually ready... what Tesla is doing today, might delay or slow the adoption of real autopilot. And that would be a real setback for the real thing.
But that's an unfair comparison.. because humans drive in all conditions, while autopilot is more likely to be used on highways (and in good weather). Even human drivers are 3x safer on highways. So until Tesla is willing to compare like with like, we do not know how it compares.
It's also too early to waive off the damage that Tesla may cause. It's been 6 months since they rolled this thing out.
Over the next few years, they're going to give 300,000 cars to consumers with this enabled.
The few incidents that have been reported may just be the tip of the iceberg. We'll see how bad it gets when you have a few hundred thousand people driving around with what they think is autonomous driving.
This is a beta product that explicitly says not to use it without paying attention. there are alarms that are supposed to go off and the vehicle is supposed to slow down to a halt if you do not have your hands on the steering wheel.
I see it like the beginning of the airline industry. At first they weren't as comfortable and safe as they are now. any time they have an airline disaster they do a thorough investigation and if it is warranted they will issue new mandates that airlines need to follow. over time, every accident makes air travel safer from those mistakes/edge cases/etc.
Will they? Considering that those are driver-assistance systems it's always the human in charge, no?
The driver would also have died if he had just enabled "dumb" cruise control and noticed the white trailer.
Of course more advanced automation systems should do better and tesla should improve, but from a technical perspective it's still just an assistance system.
Hopefully in the future Tesla will add some additional checks to make sure the driver is actually using it as an assistance system instead of an auto-pilot. And when their auto-pilot actually becomes an auto-pilot they can remove those checks.
And then they can call it autopilot again...
I wouldn't be surprised if Tesla get slapped for marketing it as an autonomous driving system. Musk has publicly said their autopilot is twice as safe as a human driver. That is misleading at best considering it should never be used without a human driver.
Rear ending a semi is one of the most fatal types of car crashes. People should definitely not trust automatic cruise control when driving behind a trailer. And regular drivers should give themselves some extra space when driving behind a semi.
It's rumored that the next version will have three cameras looking forward, so camera can be used for distance calculations.
Of course they do, which is why it's such brilliant and irresponsible marketing.
- tesla does have hands on wheel sensors
- tesla does not use stereo cameras (autopilot uses a single forward facing camera combined with radar and sonar)
- the second crash 'from yesterday' you linked is from May
Tesla doesn't take the absence of hands too seriously. "I'm driving down the highway at 55, and I'm not looking, and I have no hands on the wheel".[3] Hands off driving in New York City traffic.[4] "I managed to go 50 miles without a single nag"[5]
Apparently Tesla does have only one camera on currently shipping products. They must be trying to do stereo from motion. There are rumors of a revision to the Model S with multiple cameras.[5]
Yes, the van crash was in May, not yesterday.
[1] https://forums.teslamotors.com/forum/forums/how-get-rid-hold... [2] http://mbworld.org/forums/new-s-class-w222/626411-steering-w... [2] https://www.youtube.com/watch?v=LS7pAC0OmLI [3] https://www.youtube.com/watch?v=M4BGlQoASyo [4] https://forums.teslamotors.com/forum/forums/nag-hands-wheel-... [5] http://cleantechnica.com/2016/05/04/new-tesla-model-s-2nd-tr...
This is not correct.
It is actually camera-only per comment of Mobileye's CEO.
And lane changing. But yes, they never claimed the current system could do any more than that.
> Mercedes and BMW have been shipping that for years.
Yet, their systems on their 2017 models still isn't as good as Tesla's current system. [1]
> The other car companies have hardware to check if the driver's hands are on the wheel. Tesla not only didn't do that…
Tesla's system will indeed periodically prompt you to hold the steering wheel, and will disengage if the driver does not react to several warnings.
> … they hyped those basic functions as being an "autopilot", as if they had something comparable to what Google has
No. What Google has is called autonomous driving. You've completely misunderstood the meaning of the word "autopilot". From Wikipedia:
> An autopilot is a system used to control the trajectory of a vehicle without constant 'hands-on' control by a human operator being required. Autopilots do not replace a human operator, but assist them in controlling the vehicle, allowing them to focus on broader aspects of operation, such as monitoring the trajectory, weather and systems.
And that's exactly what Tesla's Autopilot does: assist the driver in controlling the vehicle, but does not replace a human operator.
[1] http://mashable.com/2016/06/16/2017-mercedes-benz-e-class-fi...
http://www.askthepilot.com/questionanswers/automation-myths/
Just because people mistakenly believe something doesn't make it right. The general public once thought the Earth was flat, and how did that turn out?
Tesla promotes it as autopilot. Words matter.
If Tesla itself selectively denies that the common understanding is the meaning they intend, then they appear to be trying to have it both ways: cashing in on the PR and cachet that the common meaning confers, while disclaiming the responsibility that goes along with owners actually believing it.
The unfortunate truth is, if I gave my mom a car with the autopilot feature, she'd end up eventually trusting it and not care about the implementation details that make Tesla's system not an automated driving feature. That's scary and dangerous to market to consumers.
Those constrains could be constant like a certain heading or a direction (not the same thing by the way), a series of vectors or a single vector (like a flight-path or an approach to a runway) or constraints that depend on dynamic variables like controlling the altitude and speed according to the weight of the aircraft or automatically avoiding traffic.
Of course the pilot must be standing by because a) autopilots do not keep track of everything (and aviation control interfaces are quite complex) and b) sensors can and do malfunction.
Yes, Tesla should not call it "autopilot," but if you want to really prevent people from using the system in a way in which it's not intended, you need to enforce it in hardware.
I suppose I don't really know, but that was Elon's answer to "why didn't the radar see it?"
That's what puzzles me. I don't get how overhead road signs can be a problem, and it couldn't detect something at car height. Its just really odd.
As a side note, radar in cars and semi trucks are a difficult match. The height of tractor trailer trucks makes it difficult to figure out if they are actually obstacles or if you're really seeing a bridge or something. Normally there's enough hanging down in back that it's not an issue, but in this case the truck was perpendicular to the car. It would not surprise me if the radar went right underneath the truck and either didn't see it, or saw it but thought it was something like a bridge that the car could go under. These are difficult and complex systems to engineer.
I am driving an '14 Audi A6 and I can tell you, the radars + software on it are the most advanced I ever saw. They even catch a small cat when was trying to cross the street. The car actioned a emergency break before I even realized. The only time when this wasn't working, was during a snow blizzard, when the radar sensors got iced and they couldn't see anything. Otherwise, they are pretty wide range, at least 100m.
It also keeps the car in between the lanes - this is done with a camera, but if you take your hands of the wheel, you get a max of about 30 seconds of free driving to maybe open a bottle of water, before it get's really loud.
TIL everyone on Hacker News drives a much nicer car than I do. The crash avoidance measures run in tandem with the autonomous driving features, but the system (which is mainly licensed technology from Mobileye) just doesn't get any depth intelligence from it. Presumably if autonomous driving pointed you into a brick wall, the radar stuff would still trigger the AEB features (autonomous braking).
no such thing exist ...
It's really tragic what happened but as a driver you should be aware of what's happening. I've put ~15k miles of Autopilot on our Tesla and while it helps with the mental fatigue I was still watching the road 100% of the time exactly for reasons like this.
Also I think the killer app for Autopilot is rush hour traffic. Things happen much slower and this is where I found the larger percentage of mental fatigue happens.
Agree that on straightaways, the pings are pretty infrequent.
That will require a stage of using Autopilot for five minutes without your hands on the wheel, and it won't even beep at you.
Currently Tesla's cars act as though they've satisfied that assumption even though they haven't, and someone died over it. Tesla absolutely should not be acting like they've created a self-driving car with safety features that let you ignore the road when in fact they are driving assists and you still have to maintain awareness.
Daimler actually shipped something intended to do this and beep/yell at you if it thought you were "distracted" or tired, but I honestly never knew what it was measuring. If you could point a camera at the driver it would be trivial, but people would definitely have an issue with that!
> It's really tragic what happened but as a driver you should be aware of what's happening. I've put ~15k miles of Autopilot on our Tesla and while it helps with the mental fatigue I was still watching the road 100% of the time exactly for reasons like this.
Yes, even just adaptive cruise control is a big help and wonderful in traffic. But the ACC I'm familiar with will turn off as soon as you let go of the wheel. I'm happy to hear that the features have worked well for you, and it's nice that the Tesla yells at you, but it just needs to turn off.
https://www.mbusa.com/mercedes/technology/videos/detail/titl...
Why is that a problem? An autopilot system is designed to assist, not replace, human operators. And that's exactly what Tesla's Autopilot does.
> … but the larger issue is that the car does not enforce what the company often says in press releases, which is that you need to remain alert and with your hands on the wheel.
Cars without Autopilot (i.e. every car ever manufactured in the past 100 years) also do not enforce that the driver must hold the steering wheel. I don't see anyone complaining about that. It's the driver's responsibility to place their hands on the wheel.
I'd argue it's the manufacturer's responsibility that they ensure a driver using their "autopilot" feature keeps their hands on the wheel and maintains attention so I am not hit by one of their broken vehicles.
Other cars that do automated driving tasks alert drivers loudly when they take their hands off the wheel, there is no reason that shouldn't be standard.
Pushing the responsibility on to consumers when it's a solved problem in the industry is negligent and irresponsible on the manufacturer's part.
> Why is that a problem?
The US is not in the business of allowing companies to sell products that pose a safety hazard to users or people around them.
Tesla's version of autopilot is a new feature that government agencies take pretty seriously. NTSB is now investigating the Florida crash.
There are several Youtube videos of people abusing the tech. Other drivers do not want those people driving around them. To get people to use the system safely, the government can (a) ask police to check that people are paying attention while driving, or (b) require stricter hardware controls such as hands must be on the wheel. (a) is impractical
> Cars without Autopilot (i.e. every car ever manufactured in the past 100 years) also do not enforce that the driver must hold the steering wheel. I don't see anyone complaining about that.
Other car systems are irrelevant. This is new tech that had to be approved for use in the US. It was approved last October.
Autonomous cars will have issues. But we should hold the software makers accountable for those issues the same way we hold airplane makers / maintainers responsible in the rare event their planes crash.
As other commenters pointed out, Tesla likely released a half-baked technical solution too early, and created misleading (and in this case dangerous) marketing to go along with it: calling it "autopilot" and not requiring hands on the steering wheel. Tesla screwed up here, and shame on them.
This is not a binary world: I'm not saying that they should be bankrupted or the autonomous car industry shut down. When a plane crashes due to poor maintenance - I want that company to suffer market+legal consequences, I don't want the entire aviation industry to be shut down.
White trucks on a sunny sky are not rare enough to call this an anomaly. If Tesla had installed sensors on their cars to collect data but not enabled autopilot yet - would this man still be alive today?
The MVP model is great, but not for things where life is on the line. Google seems to be doing this right.
The problem is that Tesla's marketing here is (borderline) fraudulent. There should be and need to be legal consequences for that.
An autopilot is a system that assists, but does not replace, the human operator in operating a vehicle. Tesla's system does exactly this.
> … and nowhere near autonomous driving…
Nobody ever claimed it was autonomous.
In a Tesla it would be irresponsible to take your hands of the wheel or not to pay attention to the road for even a minute. Tesla's "Autopilot" is nothing like what people imagine an autopilot to be like.
In the former, people sometimes accidentally stomp the gas instead of the brakes, then report the opposite.
In the latter, Tesla markets a feature as "Autopilot". It pilots a car into another vehicle. The feature had apparently been demonstrated at events with key safety features disabled: attendees are able to use it without their hands on the wheel. "Autopilot" fails to account for a common real-world traffic occurrence and its driver has an inadequate mental model of the feature, which is made up of a complicated mesh of sensors and software.
There may be something to the argument that self-driving car features will reduce traffic fatalities even as they become responsible for a distinctive (but hopefully small) subset of those fatalities. But this article doesn't make that point.
The problem people have with features like Autopilot is that it may not be reasonable --- at least, as those features are currently designed --- to expect human drivers to model their behavior accurately enough to use them safely. This is an HCI problem that people in the computer industry should be very familiar with. I'm surprised to see it dismissed so easily.
Did you intend this to be a double-negative? It's pretty confusing as written, but would make sense as a single-negaitve.
The article is describing how a past issue with Audi caused a huge sales problem for the company, and is suggesting that 'autopilot' style technologies could cause the same style of event.
My bet is it will quickly fall apart. Ask your average person what "autopilot" implies and they're going to give you the sci-fi definition of a completely autonomous driving system, not many people know about the mechanics and limitations of autopilots in airplanes.
In short: as automation is better able to handle easy-to-predict scenarios, that increasingly leaves only rare, complex scenarios for the humans to deal with which, by definition, they are far worse at handling.
It wasn't a freak accident. The AI failed. The human driver was distracted and failed to respond in time. Two failures Tesla has to answer for. IMO the second one is the gravest. They shouldn't have deployed the technology, primitive as it is.
"The Design of Everyday Things" talks about this, that sometimes it's the designer of something's fault, rather than the user of the something. Also called "Human Error" by statements.
Search for "https://archive.org/stream/DesignOfEverydayThings/DesignOfEv... here https://archive.org/stream/DesignOfEverydayThings/DesignOfEv...
Point being, unless Tesla makes it safe to be used as autopilot, you probably shouldn't say it's autopilot. Otherwise it's not good enough and you would fault Tesla for the design rather than the users of the autopilot.
Incidentally I'm carrying that exact book in my backpack right now. I think the problem here is much deeper, and even opposite to what Norman discusses in the book. An autopilot is the epitome of "don't make me think" - I want to go from A to B, no need to worry about the driving. The problem at hand is that the technology is not mature enough to fully take over from humans, and we are not good at staying alert in this half-focused state - I don't own a Tesla but it must be incredibly boring. It's an interesting conflict of tech vs human nature.
Camera looks up from bumper, see a overhead sign, continues on.
Later it looks up from the bumper, and a trailer is down about the same area, continues on.
Nope... Need radar. Give me a break.
As far as I can tell, the only rare circumstance here is that Tesla's autopilot would be enabled on a road where there's cross traffic.
For most of my life I have lived somewhere with lots of snow, and where the sun hangs out close to the horizon during the entire day during the winter, and I have never understood why white cars and tractor trailers are legal on public roads. In some conditions you are lucky if you see them at 1/2 the range of a vehicle of just about any other color. Even compared to ligt grays/silver metallic it's a huge difference.
Maybe something with the faint silhouette not looking very vehicle-like making the brain ignore it, I don't know? While white vehicles can be harder to spot only with the sun being close to the horizon, add some drifting snow and they essentially becomes ghosts.
So while the accident clearly show Teslas autopilot is not good enough for hands free driving - of which it has made no claim to be - it might also be telling us a very simple fact: That white is a particularly bad colour for tractor trailers regardless if it is a human or a computer interpreting the visual field ?
the point of this tech is that it can be better than human, specifically wider viewfield at wider wavelength band. Humans have only narrow wavelength band ("visible light") only in reflected/passive mode in a 120 by 60 viewfield. The tech can do 360 by 360 viewfield in human visible - cameras(passive) and lidars(active), infrared - cameras and lidars, centimeter - radar, ultrasound radar. Most of this tech (except may be non-stereo low quality camera) would see the trailer. Tesla's main failure here is that they did viewfield smaller than even the human's one.
I know I'm not going to be one of Tesla's beta users. Fuck that.
Any system that controls the fate of human lives should have several fail-safes. i.e They shouldn't only have one way of detecting objects. Maybe video + sonar + laser. I don't know the details of their architecture but even a Microsoft Kinect could have detected a white trailer against the bright sky using it's infrared + video camera. Gravely disappointed in Tesla and even more for articles like this who don't hold them accountable for inferior technology.
We're very rarely well served by phrases like "this article seems like propaganda".
"Self-Driving Cars Will Thrive With More Regulation"
Level 2 automation is not really a self-driving car, and from the driver’s perspective, sitting there staring at the road and waiting for something to happen is probably worse than just driving the car. Moreover, since people don’t actually pay as much attention as they’re supposed to when using these features, these systems will probably also cause some accidents, even as they prevent others.
And another
Getting to level-three automation, where the car mostly drives itself most of the time, seems imminent. But the actual advantages of level-three automation over cruise control and early warning systems seem dubious. If people are distracted when the car needs them to take over, then level-three automation could end up being more dangerous than the older systems. And if people aren’t distracted, which is to say they are grimly staring at the road with nothing to do for hours at a stretch, then it seems worse than actually driving the car.
http://www.bloomberg.com/view/articles/2016-01-15/self-drivi...
http://www.bloomberg.com/view/articles/2015-12-01/winning-th...
Between the systems, the SOPs, and the training, there are layers of redundancy (slices of cheese) in almost everything. All have weaknesses, but not the same weaknesses. Still, occasionally the holes in the cheese line up and you have a crash. Usually after the investigation another layer of cheese is added.
In this case the Tesla had only two layers: the autopilot, and the driver. The driver wasn't paying attention so when the autopilot made a mistake there was nothing to save the situation.
"Underreaction"[1] is much more dangerous for society than overreaction. I would cite digital privacy as an example of this.
[1] i'm not sure underreaction is a word or if i invemted it. Is underreaction so underreacted to that it is not even a word?
If you're accounting for micromorts then we might be at the point where TSA checks destroy more human life-hours than terrorism and normal plane crashes combined. Simply reverting airport security to pre-9/11 levels could save more lives than infinite spending increases for airplane crash safety.
So yes, at some point it becomes an overreaction when resources would be better spent on other ways of saving lives.
I'm not saying that we're there yet with self-driving cars. We probably are with planes.
If you look at AWS and the like, or services that tout "5 9s", you could make similar arguments. But reliability is itself a feature of a system.
If airplane crashes were more frequent, less people would rely on it (if you think it's hysterical now...).
Public Transit and package delivery are other things where reliability is super important. Not just public perception, but in a real way.
If you have to change trains twice, you can't have both be late right?
Also, death isn't just "subtract the man-hours from this person's society"....
On one hand, it's great that if I get in a crash I get crap loads of money for physical therapy and what not. On the other hand I wasted 2h every day of high school waiting and sitting on the bus and so did everyone else. Hours I'll never get back. Would rather have taken my chances :/
Eventually, the National Highway Traffic Safety Administration got involved, and wrote up a report which found that … yup, these drivers were stepping on the gas instead of the brakes, often with horrific results. That didn’t save Audi, for which sales collapsed and which almost pulled out of the U.S. market.
Despite thousands of hard working engineers, QA, Marketing, Sales, Legal and design people earnestly working to put out a safe, quality vehicle, Audi was almost undone by histrionic consumers. Really depressing when you think about it as a founder.
That said , if we look at the statistical likelihood of death in a car autonomous systems may very well be much better than humans, but whether we as a society are ethically okay with coming to terms with a piece of software that kills 1 / 100M is another question. A drunk driver that kills someone or themselves is obviously to blame. In this case who's fault is it? The drivers , tesla or the safety board for allowing autopilots to be sold?
Obviously the insurance. In Europe you need to have insurance to pay whenever is your fault.
With autopilot the only difference will be that the event of having an accident will be less probable. This means that paying the normal driver fee the insurance company will be making money if you autodrive.
But most of the time the fault will be on the other side, because the autopilot is not going to drive like a drunk or slept person. It will be deer or dogs or children on the road when they should not be there.
But an interesting situation would occur if the incentives from the insurance companies would be directed towards automated driving because of reduced risks and costs to the driver. That would be an interesting scenario
From that search, here's an interesting article from WIRED warning about the dangers (and lawsuit proneness) of the technology:
https://www.wired.com/2015/10/obviously-drivers-are-already-...
A mechanical, electrical, or hydraulic system used to guide a vehicle without assistance from a human being.
"I set the autopilot to due south, so I could get some rest."
Normal consumers aren't interested in technical details or complicated explanations. They just want what the marketing promised them.
- Google has collected 2 million miles of data in 5 years
- others have collect 0 million miles of data
One might assume that each 10x of data doubles safety. So, a billion miles of data is perhaps 8x better than a million. It was bold of Tesla to ship autopilot, but it gives them a significant data lead while Google's caution means they may be squandering their once-thought-unassailable technology lead.
My thinking is that first you eliminate the 1% problems, then the 0.1% problems, then 0.01% and so on. So each 10x of scenarios tested is roughly one step of improvement.
Tesla has added some driver assist features which are far from unique to Tesla...
It's an apple to oranges comparison.
Project those data numbers forward, do you really think Google can maintain a technology lead with such a disparity? Maybe they can, but it's a reasonable question.
Tesla is implementing existing technology. Google is working on solving the next problem.
EDIT: Tesla has.
EDIT: No, they haven't... They have shipped industry standard driver assist features. By your logic, I assume you think my Kia Rondo has 'outshipped' Google, since it has cruise control.
Google isn't a car company, they don't need to ship a car. They need to ship a working AI.
It's beginning to look like Tesla saw there was a niche where existing hardware they could license would quickly give them a more "advanced" "Autopilot" system simply by being less conservative wrt. safety than other carmakers. Will this come back to bite them? Time will tell.
The other reason for the caution is that people's lives are at stake. Google's self-driving cars are significantly more advanced in terms of the sorts of situations which they can handle ocmpared to Tesla's. But one of the things which Google has found is that there is an uncanny valley in terms of driver safety as self-driving cars get more capable. People will tend to trust them more, even shouldn't, and that paradoxically, makes them less safe.
The first generation self-driving cars were lent out to Google employees, where they would be told that the driver would be video'ed for research purposes. And people were found reading e-mail, newspapers, etc., even though they were told to be ready to take over at any time. After all, what do you do if you lose a tie rod? Or you suddenly have a catastrophic tire blow out at highway speeds?
That's why the second generation car has no steering wheel, because it's been shown that it's hopeless to depend on the human being. Instead, you need to have dual redundancy for everything (much like in aircraft design), so that even if you have a hardware problem, that the car can always bring itself safely to a stop at the side of the road, even if the driver is asleep or totally not paying attention because they are too busy watching Harry Potter or some other movie.
Caution is good, when human lives are at stake. Even if they are doing something that they were emphatically being told Not to do, if they do it, maybe the designer of the Autopilot has at least some moral responsibility, even if the lawyers have carefully protected Tesla from any legal liability?
https://www.youtube.com/watch?v=iSasPVYzSQ0&t=150
It's a fancy cruise control, not self-driving. It doesn't understand anything but how to follow lanes (sorta, it doesn't work everywhere) and it has some ability to recognize obstructions in front of it and stop (again, sorta).
Google is teaching their car how to navigate city streets, deal with human drivers and pedestrians, navigate intersections, handle construction zones, deal with emergency vehicles and cops directing traffic, etc. All with a much better sensor package than in a Tesla Model S. Their system doesn't even need lane markings on the road, as they compare LIDAR data with a pre-marked reference map. Tesla's system, and every other system that doesn't do this, breaks down when lane markings are faded or non-existent.
Google's system would not have been prone to this type of accident five years ago, much less today.
The interesting thing is that no matter how much one anticipates situations a priori and builds them into the AI/model, there will likely exist situations not anticipated by the engineers.
For example, it is not clear to me how exactly a situation like https://en.wikipedia.org/wiki/Driving_direction#Sweden will be handled. Will this be "patchable" (never mind the logistics and security issues of providing updates to a nation's fleet), or will it require a full retraining of the AI? Or more generally, how robust/modular will the designed AI be to these kinds of situations?
The reason I like this example is because it is a classic instance where humans have inertia and thus have difficulty in consistently applying the "switch driving side" rule - I have seen international travellers requiring at least a few minutes at the wheel to reorient themselves.
In principle one would hope that a modular software solution can handle things more consistently.
Google has a lot more data than their autonomously driven miles (e.g. their street view data and presumably more) and they're still not comfortable enough with what they've got. Their initial results looked very promising but it seems they've become more cautious over time.
I don't think Google is squandering their lead. I think they've just gotten to that part of the curve where they realize this is a harder problem than what they initially projected. The others haven't gotten there yet.
They are using different models for their data, different types of data and so on.
Data itself eventually stops helping; although finding more edge cases is certainly helpful.
If Tesla gets its system right, Google will have a really hard time. Their desired lidar data monopoly would become a big pile of useless data if simpler systems could do the job at a similar safety level.
At any rate, there isn't much room for error in those numbers, and considering the above this almost looks like evidence that Autopilot is less safe than normal driving.
Well no, that would constitute an immediate and utter failure. More importantly though it's not significantly better than a human, which is disappointing, especially since they'd already limited the system to the simplest possible use case.