MIT study finds Tesla drivers become inattentive when Autopilot is activated
techcrunch.com
techcrunch.com
That should be the question.
"MIT study shows folks drive 5MPH faster with seatbelts on" - doesn't mean we should get rid of seatbelts if overall damage and accidents / injury rates are reduced.
Im serious - do the study. Put someone in a 6 hour drive or simulator for an SF to LA drive or something, one in a car with tesla's drivers aids one without. Then have something happen (sudden braking / someone cutting in front of you etc) on drive in half of the trips.
Who crashes? The "inattentive" tesla driver? Or the other driver.
Based on my own observation, there are drivers in regular cars texting, looking up directions, playing with their phone maybe for music? I've seen people putting on makeup, sitting at a light for 5 minutes on their phone (when it is green) etc. I've seen people drift out of their lanes crazily. These are folks in normal cars. I've seen folks speeding and swerving in and out of traffic, some without using blinkers etc.
The question you ask is completely different and thus, not brought into the article.
So if attentiveness is measured as “staring at the license plate in front of me”, my attentiveness went down. But I felt safer since I was looking hundreds of yards down the road for situations I might have to handle or avoid, at a longer timescale.
I do think that it is very easy for me to spend longer looking at a screen or button when the autopilot is driving for me, since there is no immediate penalty (lane line, rumble strips, etc) for looking away.
I've always had the opposite problem, if I'm not maintaining situational awareness to the horizon, I'll inevitably fall asleep after an hour or two out of sheer boredom. I prefer to drive long distances in manual-transmission vehicles (there's more to do!), and commercial vehicles with loads requiring careful management of engine RPM are even better. Those I can drive for 10-12 hours in a day without issue. I never saw myself as anything other than an ordinary driver.
In my case, a Tesla autopilot is the worst possible compromise. It removes enough attention requirements that it feels much harder to stay attentive, but without automating the process completely. Other people I've talked to that operate large pieces of mechanical equipment have often said similar things about automation that it's best to either be a) fully manual or b) fully automated for some duration of time.
I was taught to do this while driving. It was a basic lesson in "Driver's Ed".
Making it a habit though is a different thing. For me that was easier because it was one of the things that pro racers mentioned they did as well.
That is certainly a key to avoiding accidents. Others would be always keeping a safe distance between you and the cars in front of you and what's going on behind and around you.
I made a habit when I first started driving to always stay focused on the task of driving. I almost never turn on the radio or play music. That's because I want to hear what's going on around me.
The best advice I ever got was from a California Highway Patrol officer who told me "Drive like everyone is trying to crash into you".
Where I live now almost everywhere I drive it's on curvy two lane mountain roads. Most have a 55mph limit with curve signs as low as 25mph. We also have deer by the thousands here, and they forage at night. You really have to pay attention to just to stay on the road and not hit deer.
There is no way I'd trust a Tesla to drive me around here (or anywhere for that matter).
That advice (for cyclists) was also in one of Neal Stephenson's early novels, Zodiac... I've found it useful. Now I just get annoyed when drivers insist on yielding to pedestrians and cyclists when they don't need to, it would be faster if they didn't and they aren't blocking any other traffic. The driver feels like they've helped, I guess, but the pedestrians are not safer.
You have to compare the worst Tesla drivers to the worst drivers in regular vehicles. Average Tesla drivers to average drivers of regular vehicles, etc.
And I'm not sure a simulator is the best comparison.
Then, there's http://www.trb.org/Publications/Blurbs/171327.aspx, but I haven't given them my address. (https://pdf.sciencedirectassets.com/271664/1-s2.0-S000145751...)
I rode a commuter bus for years and saw the exact same thing you describe, and no one would seriously suggest 50-75% of drivers are (equivalently) drunk.
Or drunk driving isn’t as dangerous as previously thought.
Or there’s many more drunk drivers than previously thought.
I don’t necessarily agree with either of those, but the point is there are many possible explanations and we should be careful about jumping to conclusions.
A better way of refuting those claims is to find flaws in their methodology
Inebriated is much different. They are less likely to notice something in the first place because their senses are deadened. If they do notice, their full attention while intoxicated is much less as is their ability to rationally make decisions. There’s no way to un-distract if something gains.
With texting and driving there are other major daily hassles. A common scenario for me as I walk my dog is that I no longer assume a car will yield on a crosswalk - even the ones with a stop sign or a signal. May be the impact is that few people bike or walk? Or walk elsewhere like a trail or a park instead of sidewalk and neighborhoods? End of the day, it’s about what we want as a society. If we decide that people should be allowed to focus on their phone, we have to start redesigning the streets now.
Not just as a pedestrian, even as a driver I don’t assume I can go in green light anymore. People seem to miss red lights on the regular now
I'd also guess most people are not constantly on their phones as they are checking notifications/texts and that kind of thing, so you get more of an on-and-off effect rather than just being consistently drunk for the entire duration.
Not that it makes the practice any better, but it does seem like a complicating factor when trying to frame the problem in terms of drunk driving.
I think this ignores the fact that people can’t actually pay attention to more than one thing at a time, generally. Meaning, they aren’t simultaneously looking at their phone and being vigilant of “weirdness” That switching overhead from one focus to the other may be enough to delay reactions to cause accidents.
So while they’re not offline, the sensory input cycle is slowed enough to be much less useful.
Yep. About 10 years ago, I took a bus from Boston to Hartford and I spent most of my time looking down into cars along the highway.
Aside from drivers looking at their phones, I saw people block their entire field of view by unwrapping burgers and lifting them up to eat, fixing their clothing, makeup, etc.
I suppose you'd have to compare against only manual driving in good conditions to make a fair comparison. Not sure that data exists anywhere.
Shouldn't Tesla create that data - research the issue - before giving people autopilot?
Its findings were that drivers paid more attention before engagement and after engagement than when AP was engaged.
> That should be the question.
Why should that be the question? If the feature is dangerous, removing it will improve safety. If there are more dangerous features, we can remove those too.
In medicine there is a concept of measuring disease-oriented outcomes instead of patient-oriented outcomes, because it's easier to show an intervention affects the disease-oriented outcomes, and much harder to show a change in patient-oriented outcomes.
So for example, a disease-oriented ourcome for hip-replacement surgery would be "range of movement after 2 weeks", patient-oriented might be "can go up a flight of stairs"
The analogy here seems relevant.
That's bullshit.
I'm sure it's safe to assume both crash more than very attentive drivers, which is the only question that matters when it comes to Tesla's autopilot.
What you're really asking is "who'd you rather have smash into you, someone in a Tesla using autopilot, or someone texting on their phone while driving?
No. That is NOT the question... because... Autopilot and FSD software suite users are required to remain attentive and ready to intervene in case o a software failure (often seen in many YouTube videos). But because they obviously, become inattentive, Autopilot usage is actually much more dangerous than having the drivers drive their cars by themselves.
It creates complacency because the moment a driver physically removes feet from the pedals and hands from the steering wheel, his or her mind cannot maintain focus on two things at the same time.
What are those two things an Autopilot user needs to perform simultaneously? First is watching the car's surroundings, the traffic around the vehicle. Second is watching and trying to understand software behavior in those constantly changing traffic situations.
No average Joe, college kid, grandma or grandpa is capable of focusing on two things at the same time in constant life and death environment.
Airplane pilots are not average Joes and they are constantly trained to understand flying Autopilot software limitations and capabilities in a lot less stressful and busy traffic environments, two of their main job requirements being discipline and responsibility. That is not the case with the average driver on the road.
Let's agree that a normal driver without autopilot (ND) is mostly attentive, with occasional attention lapses (frequency depending on the individual).
That same normal driver according to MIT, is less attentive when using an autopilot system - let's call this autopilot distraction (AD).
However, the Autopilot system, despite its weaknesses, likely offers some extra safety benefit (ASB) - especially if it can somewhat counteract the attention lapses that a normal driver would have.
So the question becomes, does the autopilot's marginal benefit outweigh the autopilot distraction?
Is: ND > (ND - AD + ASB)?
However, determining this would be difficult without real-world testing. In the real world, the factor which matters most is whether accidents happen (kind of the Overall Survival of this experiment) - which is why testing the real-world accident rate is important - and is ultimately the important question.
But the fallacy stands in the numbers where Autopilot “safety report” is presented by Tesla and compared to average drivers performances.
Why a fallacy? Because the human driver drives everywhere, at any time, in any weather conditions and on all drivable roads. The Autopilot goes on and off based on how drivable the environment is by the Autopilot system detection done by the sensors involved. In addition to that, up to this point, almost all of its errors where caught and corrected by its users, so no one could actually estimate what would happen if Autopilot owners would let the software do the driving for 100%, same thing the humans do.
Also, once disconnected from the driving system (feet off the pedals and hands off the steering wheel) the driver would need up to 20 seconds (or even more) to reconnect and fully understand what the car does and what the traffic around is like. That is a recipe for disaster while every second the situation changes inside and outside the car.
Based on my own observation, Tesla drivers, despite being supposedly inattentive, catch a fair number of errors in autopilot. So they are part of the system.
That factor (human involvement, even if imperfect) coupled with the overall safety of the tesla approach (which includes safety benefits in a whole range of areas) may mean that despite this supposed "expose" of the "inattentive" driver the system is safer overall.
One thing folks "exposing" tesla drivers seem to assume is that non-tesla drivers are very good drivers. They are not. It's takes minutes looking into cars to see this. I was just next to a non-tesla road rager who nearly caused three accidents trying to get at this truck.
If places like MIT did some more actual studies on system effectiveness I'd be REALLY curious at fatality rates etc for folks driving Teslas in a normal way, yes, in driving in ANY kind of weather.
Same thing with emergency vehicle accidents which made the news. How many first responder fatalities were caused by Tesla's vs non Tesla vehicle? It could turn out that many / most are caused by inattentive gas powered no auto-pilot cars that are the ones actually killing first responders.
> Autopilot creates a false sense of safety, as long as the people that payed for it and are eventually using it, are (by their own standards that made them buy it) not good at driving and not interested to get better at it.
Or... there are people that are good attentive drivers who buy it as a driver aid - to minimise tiredness on long journeys?
> But the fallacy stands in the numbers where Autopilot “safety report” is presented by Tesla and compared to average drivers performances.
Yes, the methodology of the safety report is probably flawed, but that's a totally different subject.
> Also, once disconnected from the driving system (feet off the pedals and hands off the steering wheel)
...which isn't how Autopilot currently works - you are asked to maintain contact with the steering wheel and have to prove it by providing torque on the wheel every few seconds. (Yes,
> the driver would need up to 20 seconds (or even more) to reconnect and fully understand what the car does and what the traffic around is like. That is a recipe for disaster while every second the situation changes inside and outside the car.
...really, just... what?
a) where does this (frankly ludicrous) 20 seconds statistic come from? b) you're not accounting for (the many?) people who might not be actively driving when Autopilot is functioning (that's the point, of course) but are still watching the road, mirrors, maintaining situational awareness, etc.
Parent comment's purely unsubstantiated hypothesizing?
Ask them, or find out yourself. The search engine is that way.
I will respond with sources to claims made with sources.
>We can test this hypothesis by looking at fatalities per mile driven, aaaaaaaand humans are still better drivers at this point.
What makes you think humans are better based on fatalities per mile driven? Is that just a guess or do you have any sources?
As I said, I respond with sources to comments that cite theirs.
All you can really say is that you _feel_ more attentive.
Because Tesla requires people to do that, as there are no guarantees that the autopilot won't [1].
[1] https://www.paloaltoonline.com/news/2020/02/25/ntsb-teslas-a...
This is pure speculation, but it feels like humans have a very robust model to determine when something is about to go wrong (i.e. this car is about to crash into an obstacle). If it takes less attention to determine that the car steers correctly as opposed to steering the car, this would leave more attention available to, for example, monitor other cars behavior.
Maybe other people aren’t like that, but I certainly am. I find myself wishing there was an FSD-like depiction of what’s going on in the driver’s brain, to answer the “you _are_ planning to stop eventually, right?” question.
There is a lot of research explaining why it's true, particularly when it comes to piloting airplanes.
The TL;DR is that doing something with your hands, involving touch, engages more of your brain.
>This is pure speculation, but it feels like humans have a very robust model to determine when something is about to go wrong
Yes, this is pure speculation, and also incorrect.
Again, if you were to look things up instead of speculating, you'd quickly find plenty of information.
Knowing where the tires are going is also important, like in that case where a Tesla accelerated into a highway divider.
Yup. Legacy. Humans are legacy equipment, required up until very recently. Constant forward-looking may seem beneficial since most things come at you from the front. You might think they are the most important class of problems because you're forced to focus on them constantly.
However, when you're on the 9's tail of problems you encounter on the road, very few of them are things approaching you from the front. Many of them are unexpected things on your sides, rears, and blind spot.
The implication of alot of these comments that assisted drivers HAVE to pay full attention to all minutae of driving just as if they were driving is really debased from the reality of self-driving i experience.
In terms of statistics right now AP is 4x safer than average human per mile travelled.
>very few of them are things approaching you from the front.
Yeah, most of them are stationary, like that highway divider that Tesla's autopilot crashed into, killing the driver.
Most things are approaching you from the front in driver's frame of reference.
And the autopilot still sucks at detecting them.
The driver was doing 68 in a 55 around a corner while playing "Three Kingdoms" on his phone and crashed 10 seconds after engaging autopilot (not FSD).
Using that as your basis of reasoning about AP safety is self-delusion. Watch some FSD beta 10 videos.
Irresponsible driving is irresponsible driving.
Because the profile of the average autopilot useris: poor, overworked, young, inexperienced driver, driving a 10 year old Honda civic without any active safety features?
Yeah, that's a fair comparison.
Also, citation needed.
My father has driven through a switchback mountain path without markings, during hailstorm, and wasn't always sober. On a car that has hasn't seen a mechanic in years.
Autopilot doesn't even work in those conditions, it gets all it's miles on a straight highway in good weather.
You play the game on easy mode, you get a better score.
Attempting to characterize the system through describing its exceptions is not really accurate.
The question isn’t if people reading a book are capable of using auto pilot safely, the question is if people can supervise level 3 self driving systems or if they need to evaluated assuming generally inattentive drivers.
Counterintuitively, the less people have been paying attention when AP is engaged the closer it is to being a level 4 system. Aka if people are catching 90% of potentially fatal mistakes then in level 4 it would be 10x as deadly. Alternatively, if their catching 10% of potentially fatal mistakes it would be almost as safe as a level 4 system.
Finally someone in the thread points out that we are pretending that the agreement "you are responsible for intervening at a moments' notice to avoid a crash" is anything but a cover-your-ass legal fiction. This is like allowing an airline to pass off responsibility for engine failure to it's customer.
It should not be legally valid unless we can prove that this method actually works. It's not just the driver's life at stake, it's also pedestrians and other people around them that did not get a 'choice'
Visual behavior pattern changes are a microbenchmark that may not even be measuring attention at all. Regardless, the pattern changes don't seem to increase crash risk as compared to not using Autopilot in any study that I'm aware of.
If there is data supporting a claim that Autopilot actually increases crashes, I would be curious to see it. "A Tesla crashed on Autopilot once" is just an anecdote; humans crash cars constantly, and the question is whether Autopilot results in greater or fewer crashes.
Source please.
Also a comparison with autopilot-off miles in Teslas would be necessary here to control for Tesla owners not being representative of average driver in the US.
Poor young overworked mininum wage job drivers aren't driving Teslas.
Nor is Tesla with autopilot off comparable to 1999 Honda Civic in terms of safety.
More food for thought https://www.herrmanandherrman.com/blog/what-does-new-data-re...
Also a comparison with autopilot-off miles in Teslas would be necessary here to control for Tesla owners not being representative of average driver in the US.
Luckily, they also release crash rates for Autopilot-off miles, as per your desire; Autopilot-on beats Autopilot-off by about 2x. Disabling all active safety features is of course even worse, losing to Autopilot by roughly 4x — although still beating the national average handily, since as you hypothesized, Tesla cars and drivers tend to not be similar to 1999 Honda cars and drivers.
Quarter-by-quarter breakdowns: https://www.tesla.com/VehicleSafetyReport
You then argue that self-assessment is inaccurate. You're right, but it just doesn't seem relevant.
On fully rested, deep sleep days, I can drop the tetronimoes at 1.5 to 1.6 pieces per second (pps), while, sleep deprived states (4-5 hours), no matter how hard I try (and it "feels" like I'm performing very fast) drops to 1.2-1.3 pps.
There's also a bit of a qualitative measure on how difficult it was to sustain focus for the 1 minute to 1.5 minutes that gives me an idea of my ability to focus and perform.
One thing that I find interesting is how i can see the differences in my performance through the day, as i hit a lull in the afternoon, and come back in the early evening.
This honestly sounds exactly why it's a bad thing. It's not just cars to worry about ahead of you.
Have you never driven anywhere that people didn't just spontaneously cross the road outside of a crosswalk? Especially at night? Not wearing anything remotely reflective or easy to see?
I've seen this happen on the freeway too. Guy wearing beige/dark green jacket, tan pants who should not have been there, yet was being dangerous to any drivers out on the road. Broad daylight. If I hadn't been looking ahead I would've missed him and at least thankfully in my scenario he didn't enter my lane.
While it's extremely important to (as my dad labeled it) "drive for other drivers" (as in, observe cars in your surroundings and anticipate any dumb actions they may make), that does not mean you should take focus off everything else, not least of which things you'd see by "staring forward" as much as possible.
If I had something like autopilot my car might've stopped like theirs did.
Honestly that's a terrible argument.
We haven't reached any kind of semblance that Autopilot or similar systems are capable of that.
Not only that, but it's meant as a supplement; it's not one or the other. We're simply not there yet, and may never be.
Oh, I was not suggesting that AP would more intelligent overall than humans, rather the vision stack might (or might not) be more sensitive to such pedestrian/background contrast at night than human eyes, simply due to the sensory differences.
I’m not sure whats stopping them. It can be done locally to avoid privacy concerns.
It also has better UX than the weird wheel touch sensors on some vehicles where you have to be touching the wheel at all times when activated (I haven’t decided if that make sense as a requirement, leaning towards no).
Cars already come with microphones for voice commands.
Like I said it could be local and isolated.
Next up, researchers discover that people like ice cream.
FSD and autopilot on highways are still bad enough that sober people are mostly going to be paying attention. I think this approach is doomed though, there is going to be a dangerous time where it's too good to pay sustained attention but not good enough that you don't have to pay attention. the way Cruise and Waymo are doing it seems much much safer.
If Tesla is indeed reporting lower crashes per mile with autopilot engaged, it suggests to me that the fatigue from paying constant attention is a bigger overall risk?
There is less sunshine at night?
In contrast, one of the challenges with FSD is that the driver often acts as though there's no distraction at all, and that their inattention to the road (apropos of detecting 'attentiveness') is a non-issue "because the car is driving and paying attention".
That's what this study is about - that markers that Tesla and other manufacturers are using to measure attentiveness may not be particularly accurate.
Isn't that kind of the ultimate point of an "autopilot"?
It's a misnomer and I'm still unclear just how useful one is which isn't developed enough to let you achieve that.
LOL. Of course they are!
But I guess the days when academic study (time, funds, energies, raw intellect) was spent on more pressing issues are behind us and we'd have to deal with these types of studies going forward. I'm so looking forward to the MIT study that finds that most of HN comments are not read.
>> the only significant conclusion this paper had was a driver was more than 22% likely to look at the "down and center stack" when on AP
>> the paper was based on data found on this https://ieeexplore.ieee.org/document/8751968 paper author by none other than my favourite podcaster D Lex Friedman . the study is conducted on 25 cars all either model S/X from 2016-2019 AKA pre fsd hardware and driver monitoring.
>> now tesla has a big screen on the center console and i would find it natural to see the navigation when im not driving. BUT all other aspects of driving were not that far from normal ... turns out people do all sorts of dumb stuff with or without AP
>> this took me less than 15 mins of googling and i was baffled by how something so stupid could go past even the most bare bone peer review . and then i found about "The Advanced Vehicle Technology Consortium (AVT)"https://agelab.mit.edu/avt the sponsor of some of the co-authors . Which curiously has major car companies and suppliers listed as partners except for tesla . i dont have the inside info on how money changes hands in academia ... but the whole thing looks pretty bad
personally i am biased towards Tesla (aka tesla fanboy) . But i feel there is enough wrong here to justify my case that this was a academia sponsored hit piece not to different from climate change deniers or tobacco companies.
Precisely. I said this before and now this study supports my previous points even further [0]. Using the wheel to monitor driver attention is not even close to good enough to determine if the driver is paying attention or not. Comma.ai seems to be able to implement eye-tracking for their driver monitoring system.
I'm quite surprised that Tesla continues to lack such a system to monitor the drivers eyes on the road when they are behind the wheel especially when the user is using autopilot or not.
You're this close to getting my point
I'm not sure if it's documented anywhere but I think the 2021 Tesla models do this [1]. That said, I'm not sure if this is enough - this is the same issue Google faced which led them to believe they had to go L4 of bust.
[1] https://www.theverge.com/2021/5/27/22457430/tesla-in-car-cam...
How is a regular Joe supposed to know?
But I think we all know this problem hasn't materialized in crash data, because if it did, that's what we'd be talking about. Instead it's just more FUD about the names of features, some of which are explicitly not yet delivered.
It absolutely did.
Why we're not talking about it is another issue.
https://www.paloaltoonline.com/news/2020/02/25/ntsb-teslas-a...
The lack of that data is why, despite years, hundreds of thousands of vehicles, and millions upon millions of miles driven, people are still blaming the names of features.
It's time to move on. Over 100 people die in the US per day in traffic accidents. How many years need to pass before a handful of anecdotes and speculation doesn't pass as valid criticism?
There is statistics, and then there is addressing obvious design flaws. The latter is worth doing a lot more than the former.
The answer is yes [1].
>It's time to move on
That time will be when the self-driving cars can safely elf-drive -- which is not now.
[1]https://www.natlawreview.com/article/dangers-driverless-cars
Natlawreview links to Carsurance which links to https://www.govtech.com/fs/first-driverless-car-crash-study-...
I'm not the one making a claim that self-driving cars are safer. The burden of proof is on whoever does.
Here are your damning statistics: self-driving cars have a higher rate of accidents and fatalities per mile driven than human-driven cars [1].
[1]https://www.natlawreview.com/article/dangers-driverless-cars
Because before regulators told them to fix it, Tesla believed the driver only needed to be checked on every _quarter hour_.
Perhaps we shouldn't expect that of people? If you've taken the vast burden of driving off their minds, we should accept that human nature means they'll stop paying attention to driving.
That, however, explicitly implies that automation needs to be much better before it can be used.
You are surprised for no cause. Tesla uses exactly that system.
https://www.notateslaapp.com/software-updates/version/2021.4...
https://www.notateslaapp.com/news/565/tesla-rolling-out-driv...
I was expecting this to be more advanced than Comma.ai, but after seeing this, I'm even more disappointed with Tesla. What if the driver is using autopilot when they are driving at night? It appears that there is no night vision in the car cabin cameras of this system. [0]
That means that it can't determine the driver's attentiveness when you are driving in the night; the most dangerous time to drive. Therefore, this system is unreliable and very dangerous and it already can be fooled. I'm afraid that this implementation is still not good or safe enough to use.
[0] https://insideevs.com/news/529788/tesla-driver-monitoring-ca...
not sure what that has to do with me explaining that the system the commenter was requesting is in fact implemented.
The system that you showed to me ‘as evidence’ may qualify as ‘basic eye tracking’ but it does not qualify as ‘safe’ when the tester in the video not only tricked it easily (just like the wheel torque tricks) but it can’t track the drivers attention at night; most dangerous time to drive, even with autopilot or FSD on.
Once again, I’m surprised that this so-called driver monitoring system came from Tesla given that it is still unreliable.