Study: Tesla Autopilot misleading, overestimated more than similar technology
mercurynews.com
mercurynews.com
The Tesla page on it (https://www.tesla.com/autopilot) takes a while to get to describing what it does, but then says, among other things:
> Autopilot enables your car to steer, accelerate and brake automatically within its lane.
And:
> With the new Tesla Vision cameras, sensors and computing power, your Tesla will navigate tighter, more complex roads.
So it's steering... but you're supposed to leave your hands on the wheel anyway? But not actually steering, just resting there just in case? I'm not sure how realistic this is in terms of human behavior. To their credit, the page, at least as of this moment, also says:
> Current Autopilot features require active driver supervision and do not make the vehicle autonomous.
Reading just the web page, if you asked me "So, do you need to leave your hands on the wheel when autopilot is engaged on the freeway", I'd be unsure. Does "active driver supervision" mean "hands on the wheel"?
In general, it is very hard for humans to keep paying close attention to something they aren't involved in actually actively controlling, "just in case." I'm not sure if there's a way for this technology to be safe, if that's what it depends on to be safe. (not unique to Tesla).
In reality, once you are used to it, it’s actually quite easy to keep a hand on the wheel while using Autopilot.
...while you do something other than pay attention. Like take a nap or watch a movie (just search YouTube for Tesla nap).
Yesterday a guy was trimming his toes. Bare foot up on the dash and just going at it. Can’t. Unsee.
Approximately half of drivers are holding a cellphone on their ear, or yawning with a look of utter exhaustion on their face, occasionally rocking out to some great tunes, which is fun when they catch you watching.
None of it is OK, but it is very clearly a universal issue much bigger than AutoPilot.
I had that guy as a Lyft driver once.
The difficult part is to stay focused when doing that. If you spent 30 minutes driving to work on a highway, and your car was able to drive itself every day for 2 months, how likely are you to take any reasonable action on 3rd month when something unexpecting happens?
The computer focuses better, but isn’t yet as capable. When things get dicey and the computer abruptly hands control back to a zoned-out human, they won’t have the same situational awareness as if they had been driving all along. OTOH, driving also tires humans out, which leads to errors and accidents too.
Figuring out the right balance is non-trivial.
I can imagine that being able to relax on the highway makes subsequent high-stress city driving a little more bearable, but people are really bad at going from zoned-out to action mode quickly.
It detects minute levels of torque on the wheel, so resting a hand in the wheel is enough to keep it happy. If it doesn’t detect any torque on the wheel it prompts you, and you can touch the wheel or move one of the scrollwheels to let it know you’re paying attention.
> In general, it is very hard for humans to keep paying close attention to something they aren't involved in actually actively controlling
I hear this a lot without citations, and I would guess it’s not specifically studied in terms of Tesla AutoPilot.
But what I can say is that AutoPilot takes the strain away from driving and allows me to focus on higher level planning and situational awareness. I watch the cars around me, I look ahead down the road and check mirrors more frequently. AutoPilot makes driving significantly more pleasant while allowing me to be more aware of potential risks around me.
I have no doubt some people choose to disengage. But it is by no means difficult to stay engaged in monitoring the drive, or at all boring.
On the contrary, while I’ve many times “zoned out” while driving as simply maintaining position in lane is extremely tedious, I’ve never zoned out while on AutoPilot, because watching the absurd things that the drivers around you are doing never gets old.
I havent tried tesla autopilot yet so I don’t know how much easier it makes driving, but if steering[1] keeps you from doing these things ... should you be on the road at all? If you’re too tired to both steer and maintain situational awareness you should probably take a rest :/
[1] steering, more like keeping the wheel steady on the freeway. Afaik Tesla can’t autopilot on the twisties yet anyway
People have a finite ability to multitask especially with similar actions. The only possible approach to driving to to not pay attention to most things as you focus on something specific and then cycle through each activity. Further, when you are forced to pay attention to something specific you must spend less time focusing on other things.
Suggesting a net benefit exists relative to the base model without self driving capabilities.
I don't know how these people fall asleep behind the wheel unless they've done something that mimics an active hand on the wheel. Maybe with the model 3 which has a driver-facing camera, they will enable it and check the driver's eyes.
I look at it this way, how many have been driving on a long trip only to glance in the rear view mirror and see a bridge behind you that you do not truly remember driving under?
I find AP reduces stress of the long drive while adding safety. I do not allow it to change lanes on its own nor do I use navigate on autopilot. Many cars have automatic cruise control, the type that keeps distances from other vehicles, some have lane keeping assist which does as the name suggest, AP is all of that with active steering. Plus the car can drive in some of the worst conditions that are tiresome for human drivers, like at night in the rain with the headlight glare of oncoming cars does not even phase it.
About the article, all those things they asked if people felt safe(r) doing with autopilot, well they actually are safer. People do it already with plain cruise control and regular driving with no overriding system doing the active driving. Do I suggest being inattentive, no, but to suggest it isn't safer with AP is silly. However saying AP replaces a human driver is just as silly.
Attention and memory formation/retention are not the same thing. The mind tends to discard detailed memories of routine tasks that have been mastered. If I ask you to tie your laces, would you remember which hand touched the laces first? It doesn't mean you weren't paying attention - only that it wasn't memorable.
Humans are not capable of suddenly be required to take control of a vehicle with a split second notice.
Hopefully the laws mature enough to prevent these for being road legal.
But Tesla doesn’t seem to push this specific use-case resticted angle of it, Musk is focus on full self driving.
If they were serious about full self driving there wouldn't be any reason to bother with autopilot.
Do you think a level 4/5 system would be done already if they hadn't made a level 2?
You can have automatic braking without having such an unsafe system.
Other brands do...
It isn't ethical, not that I expect them to care. But I don't think a sane company would play so loosely with peoples lives when they are fully dependent on complete trust when they launch something with full automation.
I have an Acura, so no autopilot, but it allows my car to steer, accelerate, and brake automatically within its lane. Also, if I don't apply control to the steering wheel (let alone have my hands off of it) for more than N seconds, it will warn me that it will disengage. The value is then more safety (the car keeps the lane by itself if you don't bother steering for some reason), which does reduce some fatigue, I guess.
It definitely reduces the cognitive load, letting you drive better in my opinion. But there isn’t an idiot-sensor in the car, so like every technology, there will be people who abuse it. The interesting question is whether those same idiots would be safer in a regular car, or if, on balance, the “autopilot+idiot” combination is safer - that seems like an empirical question to me.
Even at the current level, it’s a great advance on a regular car (I have no idea how it compares to other expensive cars with various assistive technologies, I usually buy super cheap cars...). I also think that if fully autonomous driving is technically possible, then Tesla will be the one to do it - they have several hundred thousand drivers giving them training data in all sorts of weird edge cases every day.
It really is worth watching the whole of the recent investor event at Tesla and then deciding what you think of their technology: https://www.youtube.com/watch?v=Ucp0TTmvqOE
I do this too. It is actually really good at holding the lane center, probably better than I am unless I'm really focused. But, it is more serene about the barrier than I am so sometimes I'll flinch away and end up driving again.
I was really impressed in the heavy rains we had a few weeks ago. At night in the storm I could barely make out the lanes but the car had a solid picture of not just my lane but the adjacent lanes as well. What ever filtering they are doing is impressive.
None of the autopilot accidents affected other people? I guess if by affected you mean fatalities, but 2 of them smashed into tractor trailers, so that definitely affected others. There are multiple reports of Teslas on autopilot running into stationary rescue vehicles as well, thank goodness they haven’t been fatal.
As far as risk aversion, I would say this group and Tesla owners are near opposites. Add in the completely different torque and breaking behavior. A completely different cockpit experience.
I do something like that with cruise control. I enable it but I keep my foot resting on the accelerator just in case I have to break suddenly. I'm afraid that if I keep my foot somewhere else, when my reflex kicks in, I won't hit the break pedal.
As an American who spends considerable time driving cars in various performance events (even some fun wheel to wheel endurance stuff) I think I can safely estimate at least ~25% of drivers on the road today, probably shouldn't be. Tesla is making progress, and regardless of whether they survive these various courts of public opinion, they're helping to lay the ground work for safer roads by forcing the industry to integrate more and more automated safety systems.
There will be problems. Computers will fail, mistakes will be made, and people will die, but I'm confident the break even point on those human costs is already pretty far behind us.*
*That said, we already have onerous and terrible auto regulations in this country. If we (as a society) want to force companies to be more cautious in this area, well, its entirely within our regulatory structures power to do so.
They focus more on driver-aid systems that encourage not paying attention. Their automated safety systems aren't anything special compared to what most manufactures offer.
Which other vehicles on the road actively steer your vehicle away from an imminent collision when it's safe to do so?
Tesla's blind spot monitoring doesn't just detect what's in your blind spot, like other vehicles, it also detects whether a vehicle is slowing down or accelerating into the "blind spot" adjacent to your vehicle.
Their autopilot or automated emergency braking doesn't just brake when a vehicle in front of your slams on it's brakes, it slows down when someone in front of that vehicle hit the brakes.
There's a lot of intelligent engineering there, and a lot of improvements that are pushing the industry to be safer and more responsible. There's a lot of negative press out there, but much of it is not based on fair statistics.
When I'm trying to figure out which of two really vocal, opposed camps might be more right on some issue, I look for the really hyperbolic statements that are easily refuted but still repeated.
2016: A Tesla's Forward Collision Warning System warns the driver about an impending collision he can't even see yet and starts applying the brakes. https://techcrunch.com/2016/12/28/watch-teslas-autopilot-sys...
2016: A Tesla automatically alerts the driver and swerves to the right to avoid a collision. https://www.inverse.com/article/14393-tesla-autopilot-avoidi...
2018: A Tesla's automatic emergency braking system makes it the only car in an intersection that isn't hit by a someone running a red light. https://insideevs.com/news/342781/watch-this-tesla-model-3-s...
2019: A Tesla gets rear-ended and then swerves around the vehicle in front of it and pulls safely off to the right. https://electrek.co/2019/05/06/tesla-autopilot-maneuver-avoi...
2019: A Tesla avoids a stopped vehicle on a busy road in icy conditions with extremely poor visibility. https://www.teslarati.com/tesla-model-3-automatic-emergency-...
There are so many different reports like this that searching for them is boring.
So which manufacturers do you have in mind when you say that "most" of them are offering comparable safety systems?
https://www.iihs.org/topics/advanced-driver-assistance
"study found that systems with forward collision warning and automatic braking cut rear-end crashes in half, while forward collision warning alone reduces them by 27 percent. The autobrake systems also greatly reduce rear-end crashes involving injury."
I recently did a test-drive and the guy maneuvered the car out of the dealer lot by driving backwards and looking out the rear window, completely ignoring the fantastic surround camera and sensor view that popped up on the huge display in the car... Great job selling that feature, buddy.
Another reason is that Teslas are "cool", and get a lot of press in non-automotive circles, which means that a lot of people think they were first with a lot of technology, which simply isn't true.
And a possible third reason is that quite a lot of tech people have been researching and toying with the idea of getting a Tesla, but those same people have never researched similar vehicles from other manufacturers, so they have no idea how they compare.
I had a 2014 Mercedes E-class, its self-driving capabilities was on-par with a 2014 Tesla. However, Tesla offers over-the-air updates of their vehicles, so existing Teslas got better, while my vehicle stayed the same, so that's a point they should have credit for and something they actually do better than most other manufacturers.
They're also wilfully misrepresenting the quality and safety of their technology to their customers, presenting intentionally misleading statistics to the media about it, and attempting to suppress all criticism by insisting it's part of some big oil / short seller conspiracy.
Some of the most common mis-conceptions I've seen include:
1. Level 5 Full Self Driving tech is right around the corner, with government regulation being the biggest show stopper. Elon promised "feature complete" of FSD by end of this year, and claims that within 6-12 months of that full level 5 self driving will be ready to ship to consumers.
2. Tesla is unique in their usage of computer vision in their FSD stack, and the rest of the industry only uses LIDAR. In reality almost everything Tesla does Waymo and others also do and very likely do it better, and on top of that they use LIDAR.
3. Since companies like Waymo employs pre-mapping and LIDAR, what they are building is a completely hardcoded solution to driving with very low degree of versatility and flexibility. I've seen Tesla fan suggesting that Waymo handles Stop Signs by hard coding the location of each stop sign in the map.
4. Because Auto Pilot, a level 2 driver assist system is being shipped in production cars and are racking up millions of miles, Tesla does not need any actual comprehensive testing of their FSD stack. I've seen many Tesla fans arguing this is why Tesla has done more test and gathers more "data" than companies like Waymo, when in reality Tesla's FSD system has seen almost ZERO actual testing anywhere, and puts them far behind competitors.
5. OTA updates makes Auto Pilot safer than competitors. In reality OTA very often enables less stringent QA and less rigorous regression testing. The Mountain View driver who was killed at the 85/101 on-ramp was the result of a terrible AP regression that failed to handle concrete barriers that it used to handle.
6. Auto Pilot is already safer than human driver because it has lower accidents/mile than the national average. This one is especially egregious since Tesla used active AP safety data compared against all motor vehicle accident data, which includes motorcycles and cars of all ages, driven by drivers of all ages, employing safety techs (or lack there of) of all levels, driving in all conditions. Where as active AP data is gathered by Tesla cars that's newer than 5 years old, driven by older, wealthier drivers, employs crucial safety tech such as automatic emergency braking, and only engaged during favorable conditions (mostly highway driving, clear weather, etc). When compared against data from cars with AEB, Auto Pilot has very comparative safety record.
Elon always had a tendency to be fast and loose with facts/predictions/promises, and usually I view them as entertaining at best and disappointing at worst, but with safety critical features like this what he's doing makes me feel borderline angry. A few major mistakes will set back the industry by years and as of now, Tesla is literally the only driver assist system that has a fatality count. I honestly cannot understand how any engineers with good conscious can still work on AP/FSD at Tesla and sleep well at night.
Elon says he thinks development cars will have FSD by the end of the year and you take it as a "promise" that they will have it and a "claim" that it will then be in production cars shortly thereafter.
He couched his predictions with a lot of caveats that you are ignoring while you engage in disingenuously calling the predictions "promises" and "claims."
If he "thinks" it is achievable with that short timeframe, he must have a project plan with milestones being predictably achieved in accordance with that schedule.
If instead there's no such plan, or the progress is not in line with prior internal expectations, he doesn't "think" it is achievable, he wishes it, and it is unlikely to be achieved by that time.
Whether it is a promise or not isn't the point!
Some of that difference could be accounted for in the types of miles customers choose to activate autopilot on vs drive manually, but it's certainly a positive sign.
People use Teslas cruise control in inherently safer situations. A proper comparison would compare identical road s - and would be done by a neutral body not involved with Tesla.
Uber had a fatal self-driving accident.
If I could warn anyone, I found our fellow 'highly educated' nerds, fall for marketing, similar to how we see the uneducated masses fall for populist politicians. I believe this is because these 'nerds', don't work in automotive and have limited understanding of 2019 automotive technology.
> 3. [..] I've seen Tesla fan suggesting that Waymo handles Stop Signs by hard coding the location of each stop sign in the map.
I'm sure you'll see uninformed individuals behaving uninformed. But in my experience does this not represent the average of e.g. the r/tesla community. It's surprising to have you specifically mention Waymo, since it was mostly others (especially German car makers) who usually bring up how important high precision maps (or more recently "5G", while we're mentioning buzzwords) is for self driving. My first google result right now was from Lyft [0]. I can totally understand how people less informed on the topic can get that wrong.
> 4. [..] when in reality Tesla's FSD system has seen almost ZERO actual testing anywhere
What? The usual argument I remember is Tesla outsourcing the testing to its customers and the many good reasons against it. But how can you in good faith represent the tesla AP as untested after it has driven millions of miles? Or are you specifically talking about whatever final L5 FSD solution Tesla might come up with one day? Since this obviously cannot be tested yet, since it doesn't exist.
> 5. The Mountain View driver who was killed at the 85/101 on-ramp was the result of a terrible AP regression that failed to handle concrete barriers that it used to handle.
Can you source that being caused by a regression? The best I've found [1] mentions someone else saying it got worse in a comparable spot, but I also remember reading the killed driver having specifically complained to Tesla about the location the crash happened, but not if AP behavior there had gotten better in between.
But yeah, a lot of Tesla's communication around that case was horrible and I also don't feel they do enough to stop their fans and customers from being aware of the technologies current limits.
[0] https://medium.com/@LyftLevel5/https-medium-com-lyftlevel5-r... [1] https://arstechnica.com/cars/2019/03/dashcam-video-shows-tes...
According to Elon they will reach feature complete by end of this year, but somehow that's achieved without any real world testing and validation, and they are supposed to be shipping to customers by next year.
If their "Autopilot" needs any sort of human supervision, either it is faulty, or tesla marketing is misleading. Either way, in case of an accident, it's tesla's fault - the name of the feature strongly implies that no human intervention is necessary, and talking about it with many pilot friends they all agree.
Seriously? This reads like internet trolls. But it's from a public company.
My best attempt to understand this cryptic comment is as saying that Tesla intends "Autopilot" to mean a system for "piloting" an "automobile", unrelated to the common use of the word in aviation, where it's an "automated pilot(ing system)".
Even if that is the case, they should be aware of the potential for confusion with "automated pilot" and for the risk that this confusion entails.
> “Tesla provides owners with clear guidance on how to properly use Autopilot, as well as in-car instructions before they use the system and while the feature is in use,” a Tesla spokeswoman said Friday. “If a Tesla vehicle detects that a driver is not engaged while Autopilot is in use, the driver is prohibited from using it for that drive.”
The perceived definition is far from the real definition.
People hear things like "planes can basically land themselves" which is sort-of-true, and they think the planes can actually land themselves when the reality is that the pilots need to set up ILS properly, make sure the engine thrust is at a reasonable level, do the proper communication with Air Traffic Control, and only then (maybe) can the plane land itself.
Note: I'm not actually a pilot, so my details on landing with ILS might be a bit off.
That's why a relatively simple mechanism can autopilot a plane, but the same mechanism will crash a car in an instant.
(for instance, I couldn't help but think the "sex" bill clinton was grilled about could not have led to pregnancy)
How autopilot actually works in airplanes is unimportant, the public perception and Tesla's intentionally ambiguous play with the word is what counts.
It’s unfortunate that there’s a disconnect between what autopilot means colloquially vs what it actually does on a plane.
> The person in the driver's seat is only there for legal reasons. He is not doing anything. The car is driving itself.
It's the first video on their "Autopilot" page. There's absolutely no mention of features that don't exist yet in the video.
You have to scroll all the way to the bottom of the page for any mention of full self-driving. I'd believe your claim if the video was under the headline of "Full Self-driving", but it isn't.
When you pull up the video in vimeo, the title is
"Autopilot Full Self-Driving Hardware (Neighborhood Short)"
...Flies the plane safely with zero input from the pilot (hands off the yoke) indefinitely? Lands the plane on its own [1]? I'm well aware that autopilot systems need input directions for desired altitude and air speed, but the semantic distinction people try to draw between "real" autopilot and Tesla's autopilot capabilities have never made sense to me.
The goal of Tesla's Autopilot is to take you from your garage to your parking space with zero input. Therefore it's exactly the same as a plane's which takes you from runway to runway.
Just because it's a work-in-progress doesn't mean it's named wrong.
Tesla autopilot is theoretically capable of indefinite highway driving and emergency 'landings' (on the shoulder).
Pretty close in actual capability. The difference is the environment.
It is very possible to hit people in the air if you aren’t paying attention
>Lands the plane on its own
Autolanding is not autopilot, it’s autolanding.
Typical airliner autopilot flies along a list of 3D waypoints and manages airspeed to avoid falling from the sky or overstressing the airframe. It's a fairly trivial system since it does not have to find its way around a complex, constantly changing environment like a car does. It will happily fly into mountains, other airplanes or dangerous environments like thunderstorms.
All decision-making depends on pilots. Autopilot itself is nothing more than a simple cruise control that relieves pilots from things like manually maintaining constant altitude over six hours on a transatlantic flight. Any hobbist with a Raspberry PI can built a similar GPS-based waypoint following and speed scheduling for a car, but it's useless for road traffic, because roads are not straight, long empty stretches that can be navigated by driving from one waypoint directly to another waypoint 100 miles away.
Car autopilot must be able to make decisions (follow the road, react to obstacles) to be useful, and that makes it fundamentally more complex than any existing airliner autopilot.
So while the technology may be the same, calling it "autopilot" is not; in the one case it can run on auto, in the other it can not.
You have to actively disable safety measures in order to get the car to drive for any kind of extended period without applying force to the wheel.
Anyone caught doing this intentionally should be cited for impaired driving same as a DUI.
Then there are the people who legitimately fall asleep at the wheel and AutoPilot saves their lives, which is a different matter.
Rotating the wheel just a nudge with one finger is enough to let it know you’re there. You can also rotate the volume or cruise speed adjutant wheels —- sometimes I will tweak it up and down a click (meaning you end up with no change in setting) instead of rotating the wheel.
They're worse. People who drive drunk don't necessarily make a fully conscious decision to do that. Their judgement is already impaired.
But if they think that sleeping with autopilot on is reasonable, that's 100% on them. It's something they decided ahead of time.
I think a better way forward would be to do something fun with the 'Autopilot' name, e.g.:
"Zanzibar (formerly known as Autopilot)"
Nobody need know what "Zanzibar" actually is other than some place a long way from California. You could easily talk about your Tesla to a non-owner saying 'yeah, it's got Zanzibar...' or to a fellow owner 'did you spec the Zanzibar level 5 option?'
I picked the name 'Zanzibar' randomly here, it could be any place rather than any thing. If Autopilot was rebranded as 'celery' then someone would expect that in the glovebox. A brand name that means nothing and can be nebulous would be best.
* now that I look at it again, it is actually more 'auto' than autopilot since they do not require your hands to be on the wheel but they do track your eyes for attention to the road.
I think this is misleading, Autopilot never told it's safe enough to let hands off the steering wheel. To this extend of course there is no difference than other solutions.
What regulation needs to catch up?
On the Model S page one feature is called "Autopilot: Future of Driving".
The Autopilot page then leads with a still shot of a driver not touching the wheel. If you press play it says "He is not doing anything. The car is driving itself".
Sure, about 2/3 page lengths further down, hidden in the middle of a paragraph, it caveats it. But unless you read the fine-print, it's advertising hands free driving.
> The person in the driver's seat is only there for legal reasons. He is not doing anything. The car is driving itself.
Both the Center For Auto Safety and Consumer Watchdog[1] have filed claims of deceptive advertising with the FTC because of this Autopilot video. It's really reaching to claim that the video is demonstrating anything other than Autopilot.
[1] https://www.consumerwatchdog.org/news-story/consumer-groups-...
I could completely understand a reasonable person thinking this was a demonstration of Autopilot, given it’s the primary content block on the /autopilot landing page.