Third fatal Tesla Autopilot crash renews questions about system
reuters.com
reuters.com
The technology Tesla is using in the autopilot is pretty cool, but the feature definitely has a wrong name. It should be called something like Driver Assist Plus etc.
Oh, and there is no way in hell they'll have functioning robotaxis outside predetermined geographic areas in one year's time. I'll print this post and eat it if they do!
There is precedence that "it's up to the user" is not sufficient as a societal standard for what is legally allowed. (For example seat belt laws). It is reasonable to ask: "Should we allow such a feature in cars on our road?" For example, what if the car had struck a cyclist rather than a truck and the fatality went the other way?
You are not legally free to do all dumb shit. For example, you are not free in many states to ride around without a seatbelt on. Car makers are not free to make cars without seatbelts.
That's the precedence.
At 9:58 there’s a behavior I saw a few times two or three versions back, where Tesla didn’t want to make room for a car cutting you off. Since about 2019.8 it will back off even as soon as it sees a blinker to make space.
The biggest challenge which this gen AP has is crossing intersections which are on a curve and have no lane markings. It’s not supposed to be able to do that, and not surprisingly it doesn’t do it so well.
But as a counterpoint, MA was nice enough to grind off all the lane marking on the Mass Pike westbound out of Boston a couple weeks ago and, I have no idea how, but AP keeps in lane nicely with just the grinding marks remaining on the road.
It seems to me that a feature which randomly veers into objects is not in fact a form of assistance, but a hindrance which makes driving more difficult than baseline.
It's been said before, I'm sure, but telling people to keep their hands on the wheel doesn't solve the problem of determining when to take over or resist the algorithm, since you don't know what it will do next and the time between normal driving and disaster is generally very small. A crash could be caused by the combination of actions, just like when two people both try to avoid each other in a hall, but collide anyway.
It’s pretty good on freeways. It was never intended for small streets.
And yes you’re supposed to keep your hands on the wheel.
I think the point parent was making was that Tesla was playing fast and lose with what they define as what is full auto pilot and what is merely driving assistance, especially by changing their wording depending on who they're facing, but it's mostly been able to go along because so far they're still in enhanced driving assistance and it's a gray area despite all the existing rules.
But robotaxi is much clearer: at no point can you make it a requirement / way out that whoever was having a ride should have driven the car 100% of the time at 100% attention. And that's basically how they got out of every single one of their major crash.
Potentially quite difficult to detect them if the car doesn't have them marked in its GPS-map, or?
As far as taxis, Elon said they would have supervision drivers at first. This fact continues to escape the notice of people misquoting him and claiming he “promised” things that he didn’t.
Tesla keeps saying this, but as far as i know, they can't actually detect that (because it's a torque sensor and can only whether they are moving the wheel).
Does anyone have different info?
You can even find plenty of reports that even light touching of the wheel causing it to issue warnings that hands are not on the wheel.
They never have let one of the lawsuits get past discovery (AFAIK) before settling, and this would almost certainly come out there if true.
When I was driving, holding hands on a wheel all the time, the car kept giving me notices to put hands back on the wheel. After the third time it refused to reengage the autopilot.
I guess my grip wasn’t strong enough for it to notice.
On the way back my friend did a few short stretches letting go of the wheel completely (still fully alert and hands an inch away), and the car failed to notice.
(This based on /r/teslamotors so it could be out of date or wrong. >.> )
I believe the detection relies on torque + angular position (the model S manual says it will sense resistance to wheel motion as well as motion itself), and the latter should be more more sensible because those mechanisms are usually made of encoder-motor-high ratio transmission-load, so a small turn on the load gets multiplied 10-100 times by the transmission before being sensed by the encoder.
I guess the lack of precision comes from Tesla wanting to avoid false positives
That's how dumb this argument is.
If a driver ignores the requirements of their driver license AND the vehicle's clearly worded warnings, then the driver is entirely responsible. Claiming that the driver is not at fault because they interpreted the word "autopilot" to mean "I can disregard my responsibility as a driver and ignore the vehicle's very clear warnings" is deeply absurd.
Tesla's own Autopilot page video has this disclaimer in bold at the very start of the video: "THE PERSON IN THE DRIVER'S SEAT IS ONLY THERE FOR LEGAL REASONS. HE IS NOT DOING ANYTHING. THE CAR IS DRIVING ITSELF". Then there are multiple videos of Elon Musk driving hands-free on Autopilot for extended periods of time, like this one:
https://youtu.be/MO0vdNNzwxk?t=123
So Tesla can bury whatever carefully worded legalese they want in their terms and conditions, but they know that these videos will get shared and watched millions of times, and that they strongly convey the message that "Tesla's drive themselves on Autopilot". These videos set the expectation of how Autopilot should function, and I don't blame Autopilot users for not understanding this.
> All new Tesla cars come standard with advanced hardware capable of providing Autopilot features today, and full self-driving capabilities in the future
What's more, when you activate AP in a Tesla, a message appears to keep your hands on the wheel and be ready to take over at any time.
The park-parkway analogy is about as good as cruise control vs. cruise ship. It’s a coincidence of words that nobody will ever confuse.
So, no problem changing the name then - because in your opinion the name isn't effecting the purchaser or driver expectations.
They could name it Jeff.
Boy are you going to be in for a shock when the investigations start. "Autopilot", with the way Elon has promoted it, shouldn't slam into stationary objects.
Has Tesla ever said that Autopilot will stop for stationary objects?
Has Tesla ever said that Autopilot is intended to allow the driver to take his focus off the road?
I do think that Tesla can do more to emphasise that Autopilot isn't a synonym for full self driving, but that isn't an excuse for people who are supposed to be acting in accordance with their driver license.
We have a serious problem if "autopilot" cannot avoid an object in plain sight. Everyone interprets "autopilot" to be self-driving capabilities. Some people understand that the performance of this system is tied to the conditions of the road. If you cant handle the most perfect road conditions then you shouldnt call it autopilot. It is gross negligence. Tesla can add whatever warnings they want -- they will not win that case in the long run. Either they will lose in the court of law or in the court of public opinion.
I wonder how Tesla feels about lidar now.
What the hell is the point of it then?
If a warning makes a feature less than advertised, or is impractical in general, the company has still come short of their duty. The problem with Autopilot is that basic human nature will make you less attentive if it's taking that level of control. You can't warn that away.
The warning message says "Be prepared to take over at any time." That is clearly the exact opposite of "You may now focus your attention to that laptop on the passenger seat."
Yes it is a concern that Autopilot is failing in some circumstances. But whatever Tesla says or does, the road rules say that the human driver is always responsible in cases like this.
Seems that would be easier to me.
>In 1947 a US Air Force C-54 made a transatlantic flight, including takeoff and landing, completely under the control of an autopilot.
Airplane autopilots are already at the fully autonomous stage. They have the advantage of air traffic control removing obstacles from their path, but IMO this just means ATC is part of the autopilot system. An autopilot is something that gets you from point A to point B with no input from the pilot/driver, and if it's not capable of doing that reliably then it's not an autopilot.
You can get autopilots for boats as well and they can even do collision avoidance using radar, but its non obstructed 2D space with no real lanes in open water and lanes measured in 100's of feet in or more in channels.
Cars share such close proximity with obstructed curved paths that makes a safe "autopilot" orders of magnitude more complex which should be obvious. There is just no comparison due to the space constraints cars must navigate actively.
Perhaps a PR battle is better than no PR at all.
Perhaps they should rename it to assisted driving or something like that. I think I saw something about someone sleeping while on autopilot. There's only so much parenting the manufacturer of the car can do. The ethics of all this are much more complex of course - but perhaps the messaging needs to be clearer that autopilot is more akin to advanced cruise control as opposed to you can go to sleep, the car can drive itself.
You mean like every other car company, and the exact opposite of tesla's entire marketing strategy?
Citation needed. Do you have any examples of Tesla’s marketing strategy indicating that “you can sleep at the wheel”? (Not what they predict of the future of the system, but about it’s current state)
The term autopilot comes from the avionics industry. Correct me if I am wrong, but I am pretty sure a pilot cannot go to sleep while auto-pilot is engaged either.
Elon musk takes hands off the wheel. https://www.businessinsider.co.za/elon-musk-breaks-tesla-aut...
Tesla very much makes it sound like that you just need to sit back and enjoy a coffee while your car drives. Many imagine that the car will give them 5-10 seconds of heads up before takeover is needed.
> Tesla very much makes it sound like that you just need to sit back and enjoy a coffee while your car drives
Again citation needed. Although this might sound like a fact in the HN forums, it is far from the reality.
Coast to coast autonomous drive, cross country summon mode, full self driving on highways from entry to exit, the fsd fake demo video with "driver is there only for legal reasons", million robotaxis that earn you 30k$ a year from 2020, lidar is a fools errand etc etc.
You need more? Here's Elon musk in 2016 proclaiming self driving to be easy and solved. Showing entire lack of understanding what driving is like and why computers fail at it. https://youtu.be/wsixsRI-Sz4?t=1h17m57s
What does the header say right above that video? In a really big headline font, that’s really hard to miss?
“Future of Driving”
Tesla (and more specifically, Elon) seem to continually go out of their way to make that less clear, not more.
A better angle would be that, some people who were gonna fall asleep anyways, had their lives saved.
Warning labels are not an effective way of communicating to people what the limits on a product are. Especially so when the company's CEO heavily implies that those warning labels are exclusively there for regulatory reasons.
On the one hand you have a boiler plate warning, on the other hand you have the mulit-billionare genius CEO of Tesla doing interviews with 60 minutes making a point of taking his hands off the wheel completely and bragging about it.[1]
>A better angle would be that, some people who were gonna fall asleep anyways, had their lives saved.
I don't buy this, people are more likely to not pay attention when they think they've been told they can not pay attention.
[1] - https://www.cbsnews.com/video/tesla-ceo-elon-musk-the-2018-6...
emphasis added
Maybe. Here's the thing. The feature, by its very nature, does not punish inattention like normal driving. If you let go of the wheel and focus on your phone -- even using your knee to hold the wheel, chances are the longer you are distracted the more likely you are to have a minor wakeup that says "don't do that." Maybe your car starts to hit the bumps between lanes. Maybe your peripheral vision picks up something and you get an adrenalin jolt as you refocus attention on the road.
AP is not like that. 99.9% of the time it does a pretty good job on the expressway. (95% if you are in traffic and have defensive driving tendencies where you are projecting ahead 10-15 seconds and moving lanes accordingly.) So you literally can look at your phone for 15 seconds look up and find your car at reasonable distances from other cars and properly in the lane. All of the normal "don't do that" feedback loops for inattention are removed.
But then there's that unusual case, like this one, where AP gets it wrong and the consequences are extreme. Is this different from normal inattention? I think yes because normal inattention often leaves you feeling "I just took a risk" due to the natural, and usually small, consequences.
My point is this: It may not be that the labeling of the feature is dangerous. It may be that the feature itself, by its very nature of removing the less serious consequences of inattention, is dangerous.
This reminds me of sports car reviews where they talk about how the combination of extremely grippy modern tires and traction/stability control mean that there is no warning before losing control, whereas older cars had gradual feedback.
Ultimately, I'm skeptical these are a good idea regardless of messaging.
I can see the appeal. Drivers get to experience cool new tech. Tesla/bmw/etc get a ramp that lets them develop autonomy gradually.
But at least in my uneducated opinion, autonomy-with-human-on-standby is a bad idea. Uncanny valley and whatnot.
Human in the loop is a terrible idea. We love doing make-up, texting, driving drunk, etc. We love road rage and silly egotistical lane changes or blocks and reduce efficiency of the roadways. It was fine at speeds of horses but not anymore. Especially with smartphones in the loop.
Partial autonomy and then transitioning to full autonomy is the future of transportation.
A car should be more like an elevator in functionality. Press a button, and take me there.
What? Because Elon wasn't able to achieve a milestone you consider it not possible at all? That's just absurd.
> Still a long way from full autonomous driving.
Yeah, still a long way from full autonomous driving, but having that training data make it that much closer. Theses cars have driven more miles than they ever could on test roads in a century.
The question is, does the path to full autonomy necessarily lie through partial autonomy? And related to that - is partial autonomy actually more dangerous than no autonomy at all?
Personally I think that full autonomy might be easier to hit from scratch on prepared roads, with sensors built into the environment, and the roads/lanes not shared with pedestrians/animals/animals driving cars.
I would consider making the equivalent of e.g. the London low emissions zone into an autonomous only zone, with the roads, environment and static cameras providing assistance to the cars, possibly centrally guided, and extending it out from there.
Not on my 2015 S 80D. There it is a small stalk on the left of the steering column. Up to go faster down to go slower. Double pull toward driver to enable Autosteer.
I am surprised there are not laws to have cars alert you to the fact your speeding for the systems that can read speed limit signs in addition to have current maps.
Well, there are systems that do this. Many new cars have speed limit sign recognition built in and they will display the current limit on the dashboard(and sometimes show you an icon indicating that the vehicle thinks you are speeding).
The reason why this is not more aggressive is because it's only seemingly a simple problem - in reality it's far far from it. In my experience of driving around the EU - I had a Mercedes CLS(brand new 2019 model) with adaptive cruise control and speed limit signs recognition, and while amazing when it worked, it worked maybe 50% of the time. Issues I noticed:
1) it kept detecting speed limit signs on roads parallel to the one I was on [0]
2) it kept detecting speed limit signs on side roads - leading to actual dangerous situations, because I was on a 50mph road, going past a 20mph side street and the car would suddenly hit the brakes because it detected a 20mph speed limit. That's 10000% unnacceptable in a production vehicle. [1]
3) it kept reading speed limit signs on the slip roads leaving the motorway - so you're doing 70mph on the motorway, and it suddenly reads the 50mph speed limit that's for the motorway exit. Same problem as with point 2.
4) In situations where there are no speed limit signs, it would use the map data - and even though this was a brand new, March 2019 vehicle, it had no idea that a stretch of motorway next to my house is not limited to 50mph anymore, and hasn't been for the last 2 years.
In practice, I had to keep overriding the speed limit chosen by the car constantly, and after few days I switched it fully off as it led to me nearly shitting my pants as the car hit the brakes for seemingly no reason whatsoever few times.
And then I have concerns about countries where the speed limit is only valid to the nearest intersection - and the definition of an intersection varies. There are paved roads merging with the main road which are not intersections because the side road leads to private property, and then there are dirt tracks which are intersections because technically it's a public road open to motor traffic. There are no signs to indicate either - the driver is just "supposed to know". It's idiotic, I know. And in some countries(Poland) the speed limit can change as often as 4-5 times in a span of a single kilometre. What's even worse is that there are situations when the speed limit can actually increase(!!!!) the speed limit set in a certain zone(built-up zones have a 50km/h speed limit, but this can be increased by a regular speed limit sign - and then yes, it goes back down(!!!) to 50km/h after an intersection). All of this stuff would be incredibly hard to code into a system like this, and it's one of the reasons why I suspect the proposal made by EU to have automatic speed limiters built into cars will either fail, or people will just keep disabling it altogether, making the whole idea pointless.
[0] Here it would think the side road has a speed limit of 50mph, because it would read the wrong sign:
https://www.google.pl/maps/@54.9929676,-1.5788994,3a,75y,78....
[1] Here for example the 20mph sign is placed at an angle to the road, so the car would read it going past it - and set the adaptive cruise control to 20mph which is incorrect for this street:
https://www.google.pl/maps/@55.0074774,-1.615172,3a,41.3y,90...
Same here:
https://www.google.pl/maps/@55.0071611,-1.6214135,3a,75y,240...
My sister has an insurance policy with a tracker(because she's quite young and there was no other option for her) and one day we noticed that her insurance score has fallen down through the floor, hugely negative points for....speeding. Of course the website doesn't tell you where or when, just that you're being penalized for being naughty. Ok, so we call them up, and they say they will send us the exact coordinates where the "offence" happened. Ok, cool - and turns out that they thought she was driving 70mph on a road going next to the motorway - which had a 20mph speed limit. Which was literally physically impossible, as previous GPS points were showing her on the motorway and there was no direct exit to that smaller 20mph road. So the GPS was off by maybe 10-20 metres at most, but the insurance has automatically penalized her massively for speeding. What's stopping the same system from failing when used for automatic speed limit adjustment? Especially in some extremely dense cities where you literally have roads of varying speed limits overlapping each other.
The insurer did correct that score penalty manually after I complained. But it's still a joke that it could make that mistake in the first place, especially since they can literally cancel your insurance due to speeding.
Google maps was (dangerously in the winter) directing people to an old road, into mountains after a new one had been built. It took national news, and several months of attempts to get them to update it.
The just build a big very accurate database approach is the one I would absolutely guarantee will never work.
If it's a proposal by the government, they could do better: they could mandate not only automatic speed limiters, but also something embedded in the roads like balises (https://en.wikipedia.org/wiki/Balise) to reliably indicate to the car the correct speed limit.
Tesla Model 3 has this feature and, while autopilot is engaged, will decrease the speed of the car when the speed limit decreases.
The core idea is to continue to grant enough agency to the operator that they are held responsible as long as the vehicle faithfully executed the intent.
The tongue and cheek bloodsport throttle in the real world would instead be a proactive user choice of how much human guidance and interaction is required (solving the handoff problem, for example, 1 minute automatic, 1 minute manual or an automatic system that you'll need to irritatingly press a button every few minutes to stay engaged).
If a user options for "zero engagement" (let's pretend they're drunk or just got out of surgery), then there has to be a visceral experience of the underlying risks; a constant beeping and flashing, some real sense of urgency and warning.
I know it's a luxury product but cars are machines of carnage where death is never more than a mistake and mere seconds away. That reality should supercede.
This may be eventually removed, but a system like this would honestly solve all the current issues
Just like trains! There is an alarm system just as you describe. After a certain amount of time, there are audio and visual warnings for the engineer. The engineer must engage the switch to "snooze" the alarm, and in the event that the engineer does not engage, the train automatically cuts power to throttle to brake.
I'm not sure flashing lights and a klaxon are what we would want in a car, but maybe beeping akin to lane assist that quickly increases in volume + a light on the dash would be a good start.
Europe is mandating this (combined with limiting) for new cars soon. It will be possible to override 'temporarily'.
1) I don't believe we'll go "autonomous cars only" for predictable future. I can see that happening on limited set of streets(say, strict city centres), but not overall. Mostly because I think requiring everyone to buy an autonomous car is ridiculous, and then there are cars made even right now that I can totally see people driving in 20/30/50 years from now. We'll always allow manual cars on the road, just like we still allow horses - it just became so rare that in practice it's not an issue to have an animal with no safety features on the road next to modern motor vehicles.
2) there will always be users of the road that cannot be made automous - motorcycles. Unless you believe that motorcycles will be banned, but I don't think that will happen.
So yes, I think any speed limits of the future will be set for humans first and foremost. I do look forward to those speed limits to be revised up though - cars are far safer and more efficient at higher speeds than they were few decades ago, and yet speed limits have never been revised to reflect that. US is particularly bad at this, with empty stretches of straight road having a speed limit of 50mph for instance. That should have been fixed years ago.
Where anyone who actually drives a Tesla would get the idea that you could sleep while the car drives I can't imagine. The configuration screen where you enable it makes it perfectly clear that this is a driver assistance function that requires the driver to be able to take control. And every time you enable the function with the steering column stalk it reminds you in the dash to keep your hands on the wheel.
Yachtsmen sleep while on autopilot but that kind of autopilot is considerably less capable that Tesla's and is only safe while in the open sea away from busy shipping lanes, ditto aircraft. As far as I know commercial aircraft require at least one pilot to be awake and in the seat in the cockpit at all times whether or not autopilot is on.
Only a few short weeks ago Elon proclaimed Tesla will have 1 million fully-autonomous, value-appreciating robo-taxis driving the country within a year. Way too late for them to admit their technology is nothing but "assisted driver".
I believe that the time between each crash is lengthening, even as the number of cars with AP balloons. This would speak to the regular improvement.
https://www.theverge.com/2019/5/16/18627766/tesla-autopilot-...
Accident happened on March 1, 2019, and the car was a 2018 Model 3.
> Tesla Inc’s Autopilot system was engaged during a fatal March 1 crash of a 2018 Model 3 in Delray Beach...
Since Tesla doesn’t say how many time drivers had to intervene and avoid a crash then we can have no relevant statistic.
Both Tesla and Musk do everything in their power to overhyped the AP at any cost (for drivers). Between insisting it's overall safer than human drivers and Musk suggesting the driver supervision is only needed due to pesky regulations we can assume they already have all the needed supporting data. This means they know who was responsible for saving the situation each time. Which makes them not publishing any kind of data (beyond a report that was already refuted [0]) that much more telling.
> The consequences of the public not using Autopilot, because of an inaccurate belief that it is less safe, would be extremely severe. [1]
Supported by comparing the number of deaths between a fleet of new, modern cars driving almost exclusively on freeways in good(ish) weather and a fleet of random cars even decades old, in every possible road, traffic, and weather conditions.
In the meantime Tesla AP is the only system that killed multiple drivers. An infinite times more than any other system if we make the statistic. Don't you think it's reasonable to assume it's because its capabilities are overhyped to the point where drivers are so confident they actually put their lives on the line?
This isn't trying to say the AP is bad, just that Tesla and Musk created unrealistic expectations at the expense of people's lives.
[0] https://arstechnica.com/cars/2018/05/sorry-elon-musk-theres-...
I find it truly remarkable how on the one hand HN can rage for days over Boeing introducing software with flaws that killed people and on the other hand when it's Tesla's software flaws killing people in a similarly repeatable manner, the general consensus veers towards deaths from poor software implementation not mattering nearly as much as the potential of the technology...
[1]https://medium.com/@MidwesternHedgi/teslas-driver-fatality-r...
In the Boeing, the pilots were aware and trying to mitigate the problems, but the system prevented that and ended up crashing the plane.
If the Tesla kept accelerating and steered straight into the truck, after the driver disengaged autopilot because he saw the truck, this would be a fair comparison. We don't know whether that happened, but it seems highly unlikely.
I'm not convinced that distinction makes a tendency to accelerate into static objects a comparatively acceptable teething problem for the autopilot software though, even less so for people who support Tesla's stated aim of full autonomy asap...
Tesla is a virtuous startup. Boeing is an evil defense contractor. What do you expect.
”Tesla Industries Inc, New Castle, Delaware, has been awarded a maximum $17,056,347 firm-fixed-price contract for battery power supplies.”
Also, they're shorting Tesla. They say that their position on Tesla is due to their research (as opposed to the outcome of their research being driven by their attitude), and that's fair. Nonetheless, I'd be as skeptical of their claim that Teslas are significantly more dangerous than average as I am of Musk's claim that full autonomy will be available and safe before 2020.
And you can replace Boeing with banks, car companies, oil companies or any other legacy company. Fraud, worker exploitation, shoddy product...these things are acceptable to some here, as long as the company has come from Silicon Valley.
It seems the numbers of deaths are low enough to make stats difficult, but 'car crashed badly enough to make the airbag deploy' might have more data, and might also help separate out the effects of autopilot vs crumple zone design.
To be fair, how many of the other new cars in the same price bracket and category class (midsize sedan, large sedan) can be ordered with factory 700+hp and sub-3.3s 0-100km/h times?
I haven't seen numbers but I'd suspect that the vast majority of Tesla fatalities are due to the fact that the cars are stupidly much faster than anything new Tesla drivers will have driven before.
edit: I reread your post and realized you are talking about midsize sedans and larger. Mercedes amg e63 is a better counterexample. most of the cars I mentioned don't have 700hp either, but you don't need it when you start with a 500-1000 lb weight advantage.
Also a P3D is much cheaper for similar price (https://forums.tesla.com/forum/forums/p3d-base-price), and the 3.3s 0-100 mentioned was for a P3D+ not a S100D.
I think your point still has some merit though. it could be that fast and quiet EVs don't provide as much sensory feedback to say "hey guy, you're actually going really fast". it could also be that people who are not usually interested in performance ICE vehicles might go for a performance EV for the novelty.
I assume that this is mostly pr in terms of how autonomous the ai will be once again, but many people may buy it and then get blamed for it when accidents happen.
Do we really accept a known flaw just because it's still better than the alternative?
Airline flying is safer than driving even though planes have known issues too.
Really struggling to find a logical point in here. Maybe some nuance I'm missing?
Would you really choose an overall-worse alternative because the better option has a single known flaw?
If you're that worried about Autopilot, just don't buy or use it. The rest of the car is still best in class.
Edit: Be less inflammatory.
Elon: Let's call it "Autopilot," also "Full self-driving capability is there" [2], and "We already have full-self driving capability on highways" [3]
Tesla's driver assistance software is extremely cool, but the Elon-driven hype is ridiculous, especially when compared to how Tesla PR has to describe it.
[1] https://arstechnica.com/cars/2019/05/feds-autopilot-was-acti... [2] https://www.theverge.com/2019/1/30/18204427/tesla-autopilot-... [3] https://www.teslarati.com/tesla-elon-musk-full-self-driving-...
Theranos never had a functioning product, they used tradition equipment secretly behind the scenes.
Is this house secretly using solar panels?
Tesla wants to iterate on Solar Roof a few more times to increase yield and performance before they install it on 10,000 houses with a Lifetime glass warranty and a 30 year weatherization warranty.
Musk promised that their roof tiles will be cheaper than regular roofs. I have seen zero evidence of that. The roofs on houses you mention cost WAY more than regular roofs.
And that’s before factoring in the energy generation.
It’s always going to be a premium product for people who want solar but also want a roof that looks amazing;
> “The key is to make solar look good,” said Musk at the time. “We want you to call your neighbors over and say, ‘Check out this sweet roof.’”
I'd call that a huge exaggeration.
That’s my opinion, but factually it is the best selling luxury car in the US.
That said, even the existing partial self-driving is extremely useful. I would be hard-pressed to consider another brand of vehicle until the others catch up with this.
> Tesla said in a statement that after the driver engaged the system he “immediately removed his hands from the wheel. Autopilot had not been used at any other time during that drive.”
It was obvious from the crash that autopilot was engaged and the driver was distracted. It’s heartwrenching to think the system had only just been engaged mere seconds earlier.
It seems like the driver had something happen in that 10 seconds and thought they could attend to it with their eyes off the road.
I hope one day they get the system to the point where it is intended to be used like that.
I find jiggling the volume wheel or the cruise speed wheel on the steering wheel up/down a click is more reliable because that won’t ever disengage AP.
I have no idea what they mean “did not detect hands on the wheel” because it doesn’t detect your hands on the wheel at all. It detects steering input on the wheel.
Wow! Really?! That sounds like one hell of a lie on Tesla's part... cynical part of me wonders if they deliberately avoided trying to detect hands for liability reasons?
That said, Tesla's PR often seems misleading at best.
And let me pre-empt the replies by saying yes, what matters in safety is the statistics, not a single data point. And let me pre-empt the replies to the reples by saying yes, but the statistics on self-driving cars are not at all encouraging.
I suspect Autopilot's running-into-semis rate is worse than humans', for three reasons:
(1) Tesla has been cagey about their Autopilot safety stats. If Autopilot was clearly safer, I think they'd be less cagey. Absence of evidence is evidence of absence. This is just speculation, of course.
What concrete data they have shared is summarized here: https://www.tesla.com/VehicleSafetyReport
Teslas with Autopilot on crash roughly once every 3 million miles. To their credit, that's really good! However, they don't say: (1) how often Teslas crash on freeways with Autopilot off, or (2) how often drivers have to take control to prevent Autopilot from crashing.
(2) The fatal crash in 2016 was also with a semi trailer. 2/4+ fatal crashes having this failure mode suggests this failure mode is especially high for their vision based system. Not a good look.
(3) Pretty much all self-driving companies have disengagement rates way worse than humans. Waymo is leading the pack - by far - at something like once every 10,000 miles in California. There are tons of caveats in interpreting these numbers, and they should probably be treated as an upper bound, since it's at the discretion of the company to only self-report disengagements that would have led to crashes, I believe. But human crash rates are something like once every 300,000 miles. So even lidar based systems are an order magnitude away from humans, by one super rough measure.
Lastly, a small correction to the title: This is the third known US autopilot crash. I believe there was a suspected autopilot crash in China some years back as well, and I'm not sure about other countries.
Fatal crash in China: [1] Rammed into street sweeper at full highway speed.
Non-fatal crash in California.[2] Rammed into fire truck stopped on freeway.
Tesla's system does not detect big solid objects in front of the vehicle. It's "recognize rear of car", then "brake for car". No recognition, no braking. Machine learning at its finest.
I know, I've posted this before, but I keep seeing denial.
[1] https://www.youtube.com/watch?v=fc0yYJ8-Dyo
[2] https://abcnews.go.com/Technology/utah-driver-slammed-tesla-...
I'm aware that would be creating an exception for an exceptional situation and not really a root cause solution, but still.
That’s why other companies use lidar, which is much better at object detection, but has its own problems (like fog or heavy rain, for example)
I agree that when you can't handle this kind of situation in a tesla it really challenges the idea that we'll have total self driving in a year. There must be a lot of software that is under testing that could be more reliable than what they have today, yet the issue is still there. The last part (5 or 10%) of regular software dev is really hard, and it should be even harder for self-driving, never yet approved or safe enough self driving software.
In case of the first trailer crash it apparently looked similar to an overhead sign to the sensors, also approaching a stopped object would be AEB territory, not ACC, which doesn't work well at higher speeds.
Tesla later pushed a software fix for the AEB to trigger even if only the radar alerted.
https://electrek.co/2016/07/01/understanding-fatal-tesla-acc...
Step 2: if stopped object is immediately not in front of the car, otherwise, ANALYZE IMMEDIATELY because a stopped object is right in front of the car.
It's not rocket science, and it's something so ridiculously basic that Tesla's failure to do something this simple borders of gross criminal negligence. It's something their competitors already do; in fact Waymo's problem is that it's too sensitive to stopped objects in front of the vehicle, hence the Waymo's constantly stopping and starting or driving very slowly...but that's the better and safer way to do it, especially when you're testing on real streets with third-party vehicles.
The car always has still-standing objects in front of the car (the road, anything around the road, barriers, anything on opposite sides of barriers, things in the direction of the ar but the road is turning etc.
The car has to know that the road doesn't slope up over the truck or around it. I'm guessing that's where the split-barrier fails occur.
It really needs to create a real-time 3d map of the entire environment (including what's visible from eye-height, not only front of hood) and map a route through that.
Once they are sophisticated enough to have an actual world model (something which, to my knowledge, only Waymo actually has ATM) they can stop ignoring static objects. Until then, they would keep slamming the brakes every 30 seconds for all the stuff they don't understand.
Of course this seems like a really hard thing. Probably with enough layers and with the right learning tasks it might be able to do it eventually. But the probability for noise and various learning artifacts is enormous, when you want to learn a sort of model of everything road related.
Then it should be properly marketed as autocrasher, not autopilot.
An autopilot that requires that you have your hands on wheels and pay attention at all times is missing the auto part of the autopilot (reminder that auto means self). So such a system can't be called an autopilot.
How many times did the driver have to intervene to avoid the crash? Tesla provides no data about this. But next time you hear “AP is so safe it only crashed 3 times” just remember that student drivers also almost never crash. As long as it needs constant attention, not considering this in the statistics makes them irrelevant.
It seems this is a classic estimation problem, and we need performance data points to fit a curve. Do we have anyting? Do we even know if this is a problem where we have a chance at accelerated compounding (so exponential improvement rate)?
I think the self-driving car efforts are dramatically underestimating the effort required to close those gaps to zero at least in the hype machine but also perhaps in the daily working cycle. (That's all well and good; I'm sure that any audacious engineering program should discount some of the hardest parts lest everyone get discouraged and never begin.)
Well, there are just a lot of them, outside the lane markers, and those never kill you. And while the systems surely try to classify parked cars as well, detection quality isn't under much scrutiny. Then when that one exception is sitting in the lane, good luck, your life depends on a second class feature.
Anecdotal evidence suggests that a similar perception vulnerability exists in long distance cyclists who spend many hours on the own road: lanekeeping and pothole/junk avoidance get more and more delegated to subconscious processes which work with eerie (and enticing) reliability. But that "mental autopilot", tuned to the open road, knows the back of cars only as something that pulls away from you, the simple model knows them as something not worth "raising an interrupt" over. Similar learned routines certainly exists with highway driving as well, but when driving, there isn't a class of objects which is routinely there but almost universally not a concern, like backs of cars are for a cyclist. Reports of that cyclist hitting a car parked in the open road always earn a lot of ridicule, but it's not quite as surprising as it seems at first glance (the very majority certainly succeeds at "disengaging" in time, but the exception make the news, same thing in every field)
I also wonder if this statistic includes situations where the autopilot had disengaged immediately beforehand and the human had failed to respond.
What’s the human disengagement rate?
As it happens, my uncle drove for a living and I recall he had a plaque for a million miles without an accident.
- Nearly 1.25 million people die in road crashes each year, on average 3,287 deaths a day
- An additional 20-50 million are injured or disabled
Preferring to die by human error instead of shitty software doesn't make logical sense to me and feels like your trying to justify something irrational. Autopilot either drives safer than me or doesn't. And I wouldn't let a drunk friend drive me either.
> but independent analysis shows there’s just not enough data to say
Do you link a trustworthy analysis? I can't remember reading one that wasn't from a highly polarized source.
The poster, camjohnson26, did not say die, the poster said crash.
Camjohnson26's point as I understood it was to compare two situations:
1. A crash that occurs because one of the drivers was extreme drunk,
2. A crash that occurs because of an autopilot in one of vehicles did not make the current action.
In the first case most people and the legal system will likely conclude that the drunk driver was at fault. In the second case it will be more difficult to determine who was at fault. This second case may add legal, insurance and ethical complications that camjohnson26 wishes to avoid. For instance what if your insurance decides you are at fault for activating the autopilot under unsafe conditions?
Seems reasonable to me. The driver is always responsible.
It certainly does if both auto pilot Tesla’s and standard vehicles are resulting in death, but the law will provide a remedy to your survivors and heirs in one instance and actually turn around and hold you liable in another instance.
After all the post was about liability and clearly nothing to do with how the poster prefers to die, which you somehow read into it.
That being said, I'm on board with the self driving cars, we should approach it with a logical head and really try to figure out what we should do, not what feels good.
I beg to differ. https://en.wikipedia.org/wiki/Transportation_safety_in_the_U...:
”Comparing motorways (controlled-access, divided highways) […] the United States had 3.38 road fatalities per 1 billion vehicle-km on its Interstate-type highways, often called freeways”
I think that number better reflects where Tesla’s autopilot is used. Also, if you remove motorcycle deaths from that number, I can’t see how the Tesla could be significantly safer, let alone four times less deadly.
I think you should also correct for vehicle age (older cars and lighter cars are less safe). You could (and, IMO, should) also correct for vehicle size/price, as Tesla drivers won’t be interested in whether their car is safer than other cars on the road, but in whether it is safer than what they could have bought instead.
And of course, there’s the elephant in the room. I expect that Tesla drivers prevented at least half the deadly accidents that would have occurred if Autopilot did all the highway driving.
They don’t make fraudulent misrepresentations to the public regarding their technological capabilities to self-drive which people rely on leading to fatal accidents.
But when those car manufacturers are at fault for the accidents vis-a-vis some manufacturing defect...they or their suppliers are liable.
20% of those are from motorcycles where collisions are far more dangerous. Of the remaining cars, a number of them happen in stronger weather conditions (where autopilot does not function yet) or cars that are old and unsafe. The number of fatal accidents for luxury cars with modern safety ratings under good weather conditions is much, much smaller.
This data is all available from the NHTSA at https://cdan.nhtsa.gov/tsftables/tsfar.htm , though it might take a bit of finagling to get the queries you want out of these tables. If there is a better source, please let me know.
So for a successful rollout they need to be much safer than human drivers. This is something that looks achievable. But if you try to push them out too early you might get a backlash that delays progress and costs more lives in the long run through slower adoption when they're ready.
Though I'm not sure you could compare them really at all in this specific case regarding the autopilot. I'm not sure how many of the traditional vehicle fatalities involve someone flipping on their cruise control and taking their hands off the wheel and ignoring the road in the same manner that the autopilot fatalities seem to have occurred.
It's like trying to compare Rotweilers to Pitbulls. Both are dangerous but one of them is 20x more dangerous.
That means Tesla should have driven 0.3 billion miles. Might be at the same level. Definitely not orders of magnitude different.
Automated systems are held to a higher standard. Telsa's Autopilot apparently has trouble spotting trailers and decapitates the occupants. Cite all the accident figures you want; it won't stop the reporting about Autopilot wrecks.
It was my first experience with lane departure warnings as well which was pretty neat but soon became a small distraction. I could see myself turning that off, if it was my daily driver.
The best feature of adaptive cruise control is how it keeps you a safe distance from the car in front (about 4 seconds) - whereas if I was using my own judgement, the spacing would probably be halved.
"The person in the driver's seat is only there for legal reasons"
https://i.imgur.com/74XHPiL.png
In fact, the entire video shows the driver with their hands off the wheel.
I think you can rightfully give them shit about calling the current gen "autopilot", and think their full of shit for claiming FSD is as close as it is, but this video is not intended to suggest you can operate the existing car in that manner.
But, if you're a newbie, you just Googled "tesla autopilot" and clicked on the first result. You see that tesla has the "future of driving" - that must be autopilot, right? You scroll down and watch the video. There's a notice that people are optional and a guy being driven around hands free.
The messaging is painfully there - autopilot does not require hands and the feature set of FSD vs autopilot is conflated in a very misleading way on that page in general. If you scroll down (albeit a heading that describes FSD), you see "The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat."
Tesla's autopilot landing page is clearly designed to mislead unsuspecting consumers into believing that autopilot is a self-driving capability.
To the viewer, that would just look like a person driving a car.
The only other context where we currently use "autopilot" is in aircraft, and Tesla autopilot demonstrates a significantly higher level of autonomy than the autopilot of your average aircraft.
In a recent release note adding the ability to turn off lane-change confirmations when in 'Navigate on AutoPilot', they have the following;
> With confirmation turned off, lane changes will only be made once we've confirmed that the driver's hands are on the wheel. Disabling lane confirmation does not abdicate the driver of their responsibility to keep their hands on the wheel, be engaged at all times and carefully monitor their car's surroundings. [1]
Here's what Elon actually thinks;
> [C]an’t make system too annoying or people won’t use it, negatively affecting safety, but also can’t allow people to get too complacent or safety again suffers. Latest update should have a positive effect on latter issue especially. [2]
Fundamentally, as a TM3 driver, you just absolutely must be looking out the damn window when using AP. Hands on or off the wheel is a proxy for "are you taking responsibility for your vehicle" which first and foremost means; looking at where the car is going.
I'd like to see Tesla add gaze tracking on the driver facing camera, even if it doesn't work when wearing sunglasses. If it is able to track your gaze, then it should use that. Fallback to wheel torque detection as a secondary. I couldn't find a sample video of what that camera captures, but on the PCB it is apparently labeled "SELFIE".
[1] - https://imgur.com/a/gxePQ75#6cklbYQ
[2] - https://twitter.com/elonmusk/status/1005879049493725186
https://www.wired.com/story/elon-musk-tesla-autopilot-60-min...
Every other other Level 2 system advertisement I could find shows the driver with their hands off the wheel, at least to some extent;
[1] - Nissan - https://youtu.be/HkL67DgleQY
[2] - Ford - https://youtu.be/Z_9218dTXXY
[3] - Audi - https://youtu.be/nUlK6fpveXg
[4] - BMW - https://youtu.be/xsQvq4WlUYU
Tesla can't market its system in a way that indicates hands free use is ok, then turn around then blame the driver when they crash driving like that. Like Elon!
Other car companies need to be held to the exact same standard.
Audi says you can take your hands off in traffic jam assist in its ad, the car monitors alertness and tells you when to take over.
If a customer takes their hands off as directed and the car accelerates suddenly in to the car in front of them, Audi is at fault. If they then turn around and say the customer isn't reading the fine print, then they've crossed the line in to being a bad actor just like Tesla.
(translated from Dutch to English)
https://translate.google.com/translate?hl=en&sl=auto&tl=en&u...
A student driver never crashed because the instructor is there to take over. Would you say student drivers are good drivers based on this?
There are plenty more news articles, especially in local news, about fatalities involving non autopilot cars, but they're not as interesting so don't get shared as often.
Statistically speaking 100% of people are able to pilot a plane with their eyes closed and not crash it... as long as a qualified pilot is ready to take over at any time. Such a stat is only valuable if you want to show the standby pilot is useful.
What would you base your assumption that AP saved even 1 life on when that data is not provided? All we know is that it crashed a number of times exclusively when it wasn’t supervised as required. The pragmatic conclusion is that it works because it’s supervised. I imagine Tesla would release more data if it supported the performance of the AP.
[0]: https://www.reddit.com/r/teslamotors/comments/bl5pa9/tesla_m...
I do believe their approach with a camera focuses system will work. however if video on youtube showing all cameras in action and that of our available dash cam is representative of the color richness it might be an issue where there is not differentiation to properly see all objects in some lighting conditions.
And if not, why not?
Unless Elon is going to release Real Soon Now super-special spectacles that let you really see those reptiloid shortsellers from Nibiru that are conspiring in their reptiloid lairs against all that is good (for Elon)? Maybe bundles them with those flamethrowers?
Actually, don't they already do those things in beta for some users?
I'm negative because Tesla has a track record of killing people willingly.
AP1's entire sensor suite was in other cars at the same time it was in Teslas.
Other companies rightfully used those sensors for Lane Keep Assist, Adaptive Cruise Control and Automatic Emergency Braking.
Lane Keep Assist does EXACTLY what it says on the tin. It ASSISTS in keeping a lane. It will not automatically center in a lane. It will not let you let go of the wheel.
It WILL save you in 99% of situations where one may naturally be distracted, like drowsy driving.
Adaptive Cruise Control, does EXACTLY what it says on the tin. It is cruise control... that adapts to traffic conditions. It will not stop the car by itself. It will not lull you into a false sense of safety by trying to claim it detects stopped objects. It will not give you the confidence to stare at your phone in traffic.
Automatic Emergency Braking, does EXACTLY what it says on the tin. It's a backup that will only activate in a real emergency. It will precharge the brakes opportunistically, but it won't activate until the last moment possible. It instead alerts the driver BEFORE action is taken.
AEB from Mobileye alerted me to an overturned box truck on an interstate in pitch darkness and heavy rain before I could visually confirm it.
-
Now you'll notice if you just take these building blocks, and increase how often they engage to continuous, you can lie and say you have a rudimentary self-driving car. I say lie because this level of autonomy is not what people have in mind when they say "self-driving car"
And of course no one would do that.
And certainly no one would do that and then call it AUTOPILOT.
Except Tesla did. And people died.
And they're still keeping up the charade.
And that's (part) of why I went from a Tesla fanboy to a staunch hater of Tesla and what they represent.
They've fostered this culture of setting up a fundamentally inadequate system, getting the user to use it with criminally optimistic language, then blaming the user when the inadequacies, which they knew about, result in death.
Before the accident Musk will say:
"Your hands are only on the wheels for regulatory reasons"
To imply your hands are not on the wheel for safety reasons.
After the accident Tesla will say:
"Their hands were not on the wheel as they should have been to safely use our product"
Rinse and repeat.
-
Honestly what's impressive is how many people on here condone this.
Imagine a plane's autopilot suddenly banked into a mountain. It wouldn't matter if both pilots had been sitting in the back of the plane chatting up passengers, no one would hesitate to ground hundreds of planes until the cause was fixed.
The NTSB would not release a report saying "As long as autopilot kills less people than manual flying it's fine!"
Yet every time this happens apologists come pouring out of the woodwork to parrot this stuff. It's honestly getting a little disturbing.
They would be just like all the consumers who expect that misleading advertisement is illegal.
Nobody is reading the small print, especially on the "accept this long text to use your thing" kind. Not you, not me, not anyone, really; and everyone knows it, including the courts. In fact, for that reason EULAs aren't even enforceable in all legislatures (like Germany, IIRC).
And putting critical information into the small print that you have to accept just to use the device is plain sleazy.
People take risks when new tech comes out. If they wanna be the Guinea pig then let them do it. Everyone sees musk as two faced and playing both sides but honestly - if you're an adult paying for a tesla then I'm fine with it being in your hands to decide if you're willing to roll the dice. Marketing or not, no one should be stupid enough to think ANY device has a 0% chance of killing you, let alone a computer driving.
Why should I "roll the dice" with me and my family?
And no, they weren't overtaking, etc - if you need more details I can provide them.
Coming towards me is a Tesla model 3, followed by a Peugeot.
The Tesla neatly lane changes into my lane and drives straight at me. Closing speed is around 160km/h. I hit the brakes and brake to a halt. The Tesla driver seems to take control at some point as the car first heads back to his side, but doesn't change back into his lane (maybe due to the Peugeot), then moves across my lane to the narrow shoulder on my side of the road, while braking fairly heavily (as was I).
We both reduce speed quickly enough that we aren't going to crash. The vehicles behind me have also done so. As we pull level with him (facing the wrong way on my shoulder) I can't see the driver as I am in an SUV and he is low down, opposite side of the vehicles of course. He has stopped. I continue. The car behind me stops to check on him.
The teenagers in my cars saw the whole thing and are exclaiming about it.
I guess maybe his car was seeking the white line and couldn't find it in the snow, so it hunted into my side. I don't know though.
It also has a chance of killing me, a non-Tesla driver. Where's my informed consent?