An Update on Last Week’s Accident | Hacker News Reader

An Update on Last Week’s Accident

tesla.com

349 pointsrunesoerensen429 comments

I really hate the whole Tesla angle of "technically we are not lying but we know people are going to mis-remember and misrepresent what we are writing".

Take for instance: > The driver had received several visual and one audible hands-on warning earlier in the drive...

What this means is that during this incident there were No visual or audible clues that a crash was about to happen

What they are saying is that while he was driving there was an earlier point at which the car did not crash, where the car gave a visual and audible clues. And you have to ask? Why the hell is that relevant? It isn't. They are stating it as a fact because they know some people will incorrectly claim that the car warned the driver prior to the crash.

With their eager to try to explain what happened why aren't they talking about what actually went wrong? Why did the car drive straight into a barrier. If their claim is that it was cased by the driver not having his hands on the wheel for 5 seconds, then they need to fold the company and give up trying to create self driving cars. That is Not acceptable of a self driving system.

The fact that the barrier was damaged intensified the accident, but that does not in any way excuse their system driving straight into it. And no, you can't get out of culpability by claiming statistical superiority. That's like a gang-member drying to get out of jail after killing a rival gang member because his gang statistically kills fewer people. Tesla has put a product in the hands of consumers, when it kills those consumers they need to step up to the plate and be honest about their fuckups, not just blame the drivers, blame the infrastructure and point to statistics.

Yeah, it's pretty crafty PR. The CPC must be green with envy, while Uber is still ruing their reputation. Such a difference in conversation; so many people laying the blame squarely on the driver(he is ultimately the PIC tbh, but Tesla and particularly CalTrans messed up bad), however Uber is being raked through the coals when somebody literally walked in front of their car..

Just Say'n

EDIT: While I believe Tesla and the driver are the ultimate source of the accident, I believe CalTrans messed up worse with that barrier. At least Tesla owns up to the limitations and warns drivers to stay in control, but why the hell was that barrier not replace d?! What systemic mediocracy is playing out there that they have road work signs with dates 3 years old on them, and a critical safety barrier is just missing with traffic flow as usual?!

somebody literally walked in front of their car..

The analysis I've seen suggested that the self-driving system in the Uber vehicle had 4 or 5 seconds to detect the pedestrian crossing the road in front of them. Driving into a pedestrian on the road in that circumstance is /not/ acceptable for any driver, be they human or automated.

Don't be fooled by the video footage Uber released from the in-car camera - the quality is appalling. Compare it with the dashcam footage taken by people driving the same road at the same time & you'll see that visibility should have been perfect & a human driver would easily have avoided the pedestrian who was doing something that all of do every day - crossing the road.

It is acceptable though, most people wouldn't even get involuntary manslaughter which still requires criminal negligence. People are really spouting off here and project in their own superiority onto the situation. It's a situation similar to when a child gets left in a car or jumps a zoo fence, "I never take my eye off my kids even for one second even when I'm sleeping. I would never allow that to happen to my kids. Worst parent/person who has ever walked the earth".

Not only did the pedestrian instigate that, the failsafe(which are supposed to be very simple to back up complex and more fallable systems) to not getting hit and killed by a car was entirely in the pedestrians area of responsibility; don't walk into traffic.

I disagree with your implied belief that cars should have priority on the roads & that in every collision with a pedestrian it's implicitly the pedestrian who is at fault unless it can be proven otherwise.

We all owe each other a duty of care. Excuses about 'she just walked out in front of me' are just that: excuses. The driver in this case had plenty of time to slow down and avoid a collision & simply chose not to do so. That's inexcusable.

The fact that the driver was a pile of self-driving code doesn't change that one iota.

> It is acceptable though [to drive into a pedestrian]

No. Children wander into roads. Drivers have a responsibility to keep an eye out for the unexpected. They are faultless only if they were driving responsibly and had no opportunity to react — neither of which were the case with the Uber.

At least here, the driver of the vehicle hitting anything (another car, especially a person) is /always/ responsible, because you're supposed to be attentive for unexpected events. And that's how it should be. Why do you think otherwise? If you don't see clearly in front of you because of scarce illumination, you should go slower.

Unless assisted driving is close to near perfection (that is, it becomes a true auto-pilot without the need of assistance), we're going to see much worse numbers of accidents on the streets for assisted-driving cars.

Assisted-driving requires the same, if not more attention, as it essentially becomes like supervising an inexperienced driver that is prone to make stupid mistakes in otherwise normal circumstances. If you ever had a kid, you should be pretty familiar with how stressful this is.

But thanks to the PR spin, the driver is led to think that he can keep less attention, until he pays none at all. Assisted driving is even more boring than regular driving, lowering the attention span.

What could possibly go wrong?

I think CalTrans owed up to its limitations: you should expect the road to be flat and not end abruptly without appropriate signage. But you are fully responsible to control the vehicle at all times and maintain an appropriate speed - so that when the road does inevitably end you can react safely. At no point are the crash atenuators a guaranteed component that the user of the road can expect to be in place and should rely on to save his life.

That is not to say CalTrans did not break some contractual obligations to replace that infrastructure in a reasonable timeframe, and shouldn't be held responsible in a commercial sense. But they are not in any way liable for the accident, the missing hardware was not a reason to stop traffic and all such devices are offered on a best effort, good to have, basis.

While I believe Tesla and the driver are the ultimate source of the accident, I believe CalTrans messed up worse with that barrier.

When reading something like this in sci-fi, I always thought that the stories where over the top and tongue in cheek, to make some other point. Now I see they were real.

Companies are allowed to create robots that kill people. Then we sit around here and discuss how much of the killing is on the the robot's manufacturer, the robot's owner, the street lights and the victim.

Replace "robot" with "car" and it suddenly doesn't sound sci-fi at all.

> Then we sit around here and discuss how much of the killing is on the the robot's manufacturer, the robot's owner, the street lights and the victim.

You forgot one entity and I think it is a major one.

The people of the world who just stood by and let sdv's on the street with no sort of regulations and tests.

I posted this [1] after the Uber incident, Look at the awesome response from HN community that I have received.

[1] https://news.ycombinator.com/item?id=16696681

> Such a difference in conversation

in both the driver was supposed not to let the car drive itself unsupervised but only in one the driver was paid to do so

Perhaps they don't know (yet?) what exactly went wrong? The car was heavily destroyed, I am surprised they got any logs retrieved from the car at all. They might be able to reconstruct more information about the accident, but unless they have a complete log including video (unlikely), it will take time to reconstruct. As there is an ongoing investigation about the accident, it is understandable, that they are not making any statements beyond anything they can prove without a doubt at the current point of time.

Sure, but the ONLY reason many of us can figure for mentioning the previous warnings in the way they did is because the way it would.get interpreted as a sound bite. Could be wrong, but this isn't conspiracy theory territory. These companies have teams of highly paid communications experts(spinsters) engaging in social engineering.

It's widely known that RADAR systems are not able to detect stationary objects, only moving ones. Actually, they are are to detect them but just not where they come from, so a barrier in the middle of the road and a traffic sign on the side of the street, they would look like the same to the RADAR system.

But to be honest, I am more worried with the markings of the road rather than the inability of the autopilot of not foreseeing the accident: https://imgur.com/a/hAeQI

What's wrong with the US road administration? Why is this even look like a driving line? Where are the obvious markings? It's a very misleading road layout, I am curious the amount of accidents that happen there every year.

This is how I expect this kind of thing to look like: https://i.imgur.com/dfZehmd.gif

Given on how the road looks like, it makes more sense why Tesla is reinforcing the fact that the driver wasn't paying attention to the road.

Edit: Since people are curious about the limitations of the RADAR, the manual of the car mentions this limitation:

"Traffic-Aware Cruise Control cannot detect all objects and may not brake/decelerate for stationary vehicles, especially in situations when you are driving over 50 mph (80 km/h) and a vehicle you are following moves out of your driving path and a stationary vehicle or object is in front of you instead."

You can also read here that Volvo's system faces the same problem: https://www.wired.com/story/tesla-autopilot-why-crash-radar/

> It's widely known that RADAR systems are not able to detect stationary objects, only moving ones. Actually, they are are to detect them but just not where they come from

That doesn't make any sense. Radar systems do detect stationary objects just fine. In fact, from the point of view of the moving car, almost nothing is stationary.

I wouldn't be surprised if you're correct about the markings on the road, I almost crashed into a temporary barrier driving at night, because there were two sets of lane markings - the old ones, and the ones going around the barrier. Construction workers simply didn't bother to erase the old ones.

It is mentioned in the manual of the car:

You can also read here that Volvo's system faces the same problem: https://www.wired.com/story/tesla-autopilot-why-crash-radar/

IIRC the radar can detect the object but it has too many false positives, like a small soda can that has a lucky alienation and shape that make it appear as a big object. So the system ignores the static signals of the radar, because randomly stopping each time an small object is confused with a big object is also dangerous. And the system uses other signals to identify and avoid static objects.

This is not a RADAR problem, it’s a world modeling problem.

So you’re sending out pulses and listening for echos, which tells you how far away something is in a particular direction. You correlate subsequent pulses to say whether the object is moving toward you or away. If you have a car 50 m ahead of you every time you ping, everything is good. Now that car suddenly swerves around a car stopped in front of it, and your ping off that object says 60 m. A crash is less likely, your model thinks! The object in front of you is rapidly speeding away! By the time it realizes it isn’t, boom, crash.

That’s not how it works in my limited understanding. Radars don’t just detect the positions of object, but also their relative speed. For this, they take advantage of the Doppler effect. When a pulse gets reflected, its frequency rises if the object is moving towards the radar, and falls if it’s moving away.

You could be right, either in this case or other cases. I'm entirely inferring what autos are doing based on their failure case (not detecting stationary cars when the car in front of you swerves around them); that sounds a helluva lot like mistaking the new car for the old.

Again, not saying it's a limitation of RADAR; it sounds like a deficiency in the way they're using it.

Although there's an inherent trade-off between detecting position and speed. Longer pulses give more accurate frequency/speed detection, while short pulses give more accurate time/position detection. This is the same phenomenon that underlies Heisenberg uncertainty.

> It's widely known that RADAR systems are not able to detect stationary objects

What the fuck? That's not at all how radar works. It has no such limitation.

My understanding is that RADAR systems can detect stationary objects just fine: What's going on is that in-car radar systems can't differentiate between, say, a metal sign on a gantry or at the side of the road and a parked car on the road, because their spatial resolution is poor to non-existent. Hence in-car RADAR systems (often) ignore radar returns that are non-moving relative to the road surface. Otherwise the car would panic brake every time you went past a road sign.

The effect of this is that current in car RADAR systems are great at avoiding collisions with vehicles that suddenly brake to a halt in front of you whilst at the same time will happily let you drive a full speed straight into the back of a parked car.

This is why I believe current self driving vehicles (apart from Tesla) are all using LIDAR for object detection.

(The above is my interpretation of my reading on the current state of the art. If anyone knows more detail, please correct me.)

The problem for all self-driving car systems is modeling the road. Let’s say you are climbing a gentle hill, and there’s an overpass ahead. The nose of the car is aiming up, so it will see the overpass dead ahead and not moving. This looks the same to the car as driving down a flat road with a semi perpendicular to the travel lanes and dead stopped. This is true for all imaging systems - RADAR, LIDAR and camera.

Same if you are about to enter a right-hand sweeper, pedestrians standing on the sidewalk in front of you would look as if they are directly in your path.

Musk mentioned this in one of his blog posts. Tesla is attempting to build a database of obstacles like this and geotag them, so that the car can filter them out.

> It's widely known that RADAR systems are not able to detect stationary objects, only moving ones.

Uh, what?

I think you're thinking of Doppler radar systems. You can do static scene imaging with radar, no problems (I've written the code to do it, even!).

There's no way in hell a car is going to be using a Doppler radar. In all likelyhood, they're using a plain-old FMCW system, and it can absolutely detect stationary stuff.

Answered here: https://news.ycombinator.com/item?id=16722617

Model S likely uses https://www.bosch-mobility-solutions.com/en/products-and-ser...

(It's for sure a Bosch radar, but maybe a different one)

Yeah, there is so much wrong going on with that road. The divider warning sign is also lower than a standard car.

While I find it deplorable that several companies are essentially using people, although in this case apparently willing, participants in testing auto pilot or autonomous driving software. I also find the lack of prominent road markings, signage, or any significant shock absorbing zone simply astonishing!

It makes me wonder, is that layout actually on purpose? I truly find it hard to believe that a road or traffic authority in this day and age could design that on purpose! It's quite well known that human attention is a tenous thing, and that the attention given to an object is primarily proportional to its size. But here we have a barrier end that is signed as being about as dangerous as a a bit of debris.

What irks me is that they think it's OK for them to advertise it as auto-pilot, with an asterisk pointing to the fine print that says "aw shucks, not actually, haha".

Well all that an actual autopilot (on a plane) does is maintain your velocity and heading, so if anything the name is underselling it.

autopilots do a bit more than maintain course, they can basically execute the entire flight-plan in between takeoff & landing, which includes plenty of heading adjustments. They also increasingly do a fair bit of heavy lifting on landings as well.

A plane is much easier to navigate because there aren't things like lane-dividers floating about. That's handy, but alas it does not apply to driving a car.

Just because an in-air navigation aid that allows for essentially pilot-free navigation deserving of the name "autopilot" happens to be relatively trivial to build, does not mean any navigation aid of similar complexity for other modes of transport should be called autopilot simply because they're at least as complex. If you're not automatically piloting, the name autopilot is pretty disingenuous, is it not?

Also, planes have a bunch of other navigation aids, including collision avoidance systems and ground-based air traffic control, and pretty obvious stuff like predictable flight paths and huge safety margins between nearby aircraft. An autopilot for an airplane works in that context. I kind of doubt it would work if there were thousands of planes nearby, many of which are on paths separated by distances the vehicles in question cover in tiny fractions of a second, and some of which follow almost invisible rather unpredictable paths (that drunk guy without lights over there...). Context matters.

I think what I'm taking issue with isn't anything about comparing them to self driving cars, it's that you characterize them as "trivial to build". They are not. They coordinate a significant number of life-critical systems, react to minute perturbations in the air stream with microsecond precision, and do so with such a high level of precision that human pilots are often forbidden from overriding them outside of specific situations. I don't know what gave you the impression that these systems were basic or trivial, but they have been at the cutting edge of systems automation for decades.

Compared to a self-driving car system, they are trivial. Your assertion that "they have been at the cutting edge of systems automation for decades" kind of indicates that - this stuff is simple enough to make that it was feasible to do even that long ago.

Trivial is good mind you because it actually works, and works reliably. The same is not yet clear of self-driving cars.

Well my kap 140 sure doesn't do any of that stuff.

Absolutely.

It's a marketing term, and they are intentionally committing murder by the act of not calling it "LaneAssist" or some other similarly boring name.

Radars do just fine detecting stationary objects, what they can't do is tell whether the stationary object is blocking the road, overhead, or to the side of a curve in the road. As most radar reports of stationary objects fall in the latter two categories, the adaptive cruise control system filters them out of consideration.

They're marked 'optional' on your diagram. Optional things can be excluded.

"Optional" doesn't mean "exclude it if you feel like it". It means "think about it and decide whether it's worth adding". They can be excluded if doing so doesn't present a danger (e.g. it's a small road).

From the one photograph we've seen of the slip road in question, I'd say the lack of chevrons poses quite a serious danger. It looks just like a normal lane. I've never noticed a slip road branch off like that in the UK, i.e. a big crash barrier up the middle but no chevrons.

(I'm not saying whoever designed this section of road is guilty of some sort of crime, or is somehow responsible for the crashes, I'm just saying it should have been designed better).

I think at least 50% of the blame should be placed on the maintainer of the road.

The crash attenuator was 'used up' from a previous accident and might have prevented death if it was in working condition.

I see this all the time where I live. Crashed attenuators or disfigured guard rails go months and months without repair.

If we are moving towards a future with self driving cars there needs standardization of road markers so that cars can recognize what's going on.

This crash is looking to be the autopilots due to the strange lane markings.

Have you ever driven down a road the lines grinded away and replace by new ones? Sometimes you can still see both sets, it can be pretty confusing for a human.

Won't help. There will always be surprises. Debris on the road, or snow drifts, or something.

Any system that works by not interpreting the world outside is bound to encounter nasty surprises.

And no, you can't get out of culpability by claiming statistical superiority.

Quite. Tesla is deliberately comparing their modern vehicle design & wealthy driver base (statistically one of the safest cohorts) with the entirety of the U.S. driving population. This is bad statistics: We expect Tesla vehicles to have far fewer accidents than the mean US vehicle per mile, because Tesla's are expensive modern vehicles with (relatively) wealthy drivers who can afford to keep them maintained & will themselves be in better health than the mean US driver.

Don't tell us how safe Tesla's are compared to the entire US driving population: Tell us how safe they are compared to equivalent vehicles from other manufacturers. My working assumption is that Tesla doesn't do that because it would be far less flattering to the 'A Tesla is totally safe, honest' PR boosterism that runs through every Tesla press release.

> Quite. Tesla is deliberately comparing their modern vehicle design & wealthy driver base (statistically one of the safest cohorts) with the entirety of the U.S. driving population. This is bad statistics: We expect Tesla vehicles to have far fewer accidents than the mean US vehicle per mile, because Tesla's are expensive modern vehicles with (relatively) wealthy drivers who can afford to keep them maintained & will themselves be in better health than the mean US driver.

You're going to have to explain to me how the wealth of a driver has any bearing on their ability to pilot a self-driving car.

Statistically if you're wealthy enough to own a Tesla, you're also way less likely to have to drive when tired/under the influence for example.

All sorts of constraints are correlated with wealth that put you in way more risky situations if you're poorer.

Do you have anything to back this claim? It seems to not make sense.

For example: https://academic.oup.com/alcalc/article/46/6/721/129644

It's not exactly controversial to claim that lower income correlates to crime though.

There's lots of different kinds of crime, each very unique in its circumstances. Just try comparing the stats for securities fraud vs. drug possession, for example.

DUI is a tough one, people of all socioeconomic backgrounds like to get drunk, and people of all socioeconomic backgrounds like to drive. Off the cuff, I might guess that the numbers would show DUI to be the most egalitarian crime, actually.

How about you look at facts before spewing PC-motivated 'guesses', like http://www.thelancet.com/journals/lanpub/article/PIIS2468-26... and http://injuryprevention.bmj.com/content/8/2/111 - both from the first page of my first google search, I'm sure it's trivial to find many more.

Firstly, wealthy drivers can afford to keep their cars in better shape and can afford better cars in the first place. Neither of these factors depend on self-driving technology.

Secondly, Tesla themselves makes it clear that their self-driving can't be relied upon to driver the car unsupervised - the driver has to be paying attention at all times in case the Tesla self-driving code decides to do something catastrophically stupid. Hence the same factors that correlate with driver attentiveness in the ordinary population will also affect the Tesla population. Wealth correlates with health correlates with better reaction times & general fitness.

The Tesla autopilot tends to turn itself off precisely in situations where driving is difficult and complex - which in turn implies that these human factors will remain just as important as they are in the general driving population.

(Of course it may simply be that the reason wealthy people have a lower death rate on the roads per mile is that they can afford cars that are safer in crashes; the above is my supposition, not evidentially backed.)

What I'm looking for really is anything specific to auto-pilot technology to suggest that the safety benefits won't also be met, or exceeded, when extrapolated across the whole population.

All other factors taken into account, driving has become much, much safer in recent years, and is now safer than it ever was. When you start to use technology to eliminate driver errors in that self-same error-prone cohort, there's no reason to think that the safety dividend won't be even more pronounced than it is today.

If you want to link crash rates to wealth, you would also have to account for countries like Germany, that has a similar median wealth figure to the US, but a vastly lower motorway death rate [1]

It seems you can't both say 1) Tesla drivers are less likely to have accidents because they are wealthy and error-prone, and 2) Extending self-driving technology that eliminates basic errors to less-wealthy and/or more error-prone drivers won't have a net benefit on driver safety.

[1] https://www.sciencedirect.com/science/article/pii/S240584401...

I’m suggesting that Tesla drivers are both relatively wealthy & /less/ error prone!

Clearly /if/ Tesla self-driving code is safer than the average driver then extending that to the general driving population would be a net good. But if the apparent safety is an artefact of the statistics due to Tesla cars themselves being intrinsically safer just as practically every other modern vehicle is a safer car than one built 20 years ago then extending self-driving to every vehicle might not be a net safety benefit

Likewise, if the driver cohort is different (and we know that it is different) from the general population & that cohort is required to drive the vehicle precisely at the times that the self-driving code can't cope which are probably the times when driving places the most demand on the driver, then that different in driver cohort matters: the statistical difference in death rates could easily be a result of differences in driver populations.

You can't tell from the data as given & the fact that Tesla keeps repeating the aggregate figures in their PR is suspicious frankly.

(NB. that paper you link simply underlines my point - In both countries, the lowest wealth quartile has a vastly greater death rate on the roads than the rest of the population. It doesn't matter whether Germany is worse or better than the US. If Tesla draws disproportionately from higher income deciles, then we should a priori expect them to have a far lower death rate than the mean. Quoting the mean as a benchmark to compare the Tesla death rate against is outright misleading.)

>Wealth correlates with health correlates with better reaction times & general fitness

I agree with your general point, but you can't just join up 2 correlations like that, that isn't always going to work when there are other confounding factors (and there always are). Wealth also correlates with age which anticorrelates with reaction times and general fitness.

Agreed, hence my parenthetical note at the bottom of the comment.

According to insurance adjusters, age correlates with lower accident rates.

And having a hotmail emailaddress with higher accident rates.

Lower accident rates, or a lower percentage of injury or fatal accidents? I find it hard to believe 80 year olds are getting into fewer accidents than 40 year olds unless it’s because they drive less frequently.

Which would be fine for an insurer of course. Heck, even if they got into more incidents, but the incidents were cheaper, that would be fine for an insurer too.

And not all markets are equally competitive. Just because insurance rates differ doesn't mean risks necessarily do the same way. And let's not forget that while insurers are professional risk assessors that doesn't mean they're infallible either, or immune to other distortions (e.g. pass-the-buck subprime mortgage kind of issues).

In case anyone's wondering, here's a simple example of 3 variables so that the first is positively correlated with the second, the second positively correlated with the third, but the first and the third have a negative correlation: simply take 3 independent standard normal variables, X, Y, Z, and define our 3 variables to be u=X+Y, v=Y+Z, w=Z-X. Then correl(u,v)=0.5, correl(v,w)=0.5 and correl(u,w)=-0.5.

> Wealth correlates with health correlates with better reaction times & general fitness.

But at some point between wealthy and rich the person hires a (non-wealthy) driver and now they're back to square one.

Wealthy people can better control the quality of the driver.

Wealth of driver - converts to potential good lawyers per damage per crash converts to fear of god in Tesla software departments management converts to actual safety practices being followed and corners not being cut?

That's an awfully long risk compensation feedback loop to close! It sounds like a plausible motivation for Tesla to care more about the quality of their self-driving code than they otherwise might, but it would be a bit difficult to run the counter factual experiment to actually prove it one way or the other :)

Hell, wealth correlates with age. How many 16-20 year olds are driving Tesla’s?

> We expect Tesla vehicles to have far fewer accidents than the mean US vehicle per mile,

We do? Is there a cite for that? It seems to me that driver of new luxury vehicles are disproportionately aggressive and unsafe, actually. Has this been studied?

If your going to make an argument for bad statistics, you need to practice good science yourself.

The simplest argument here is that Tesla's are new vehicles, the total US fleet average age is much older than any Tesla on the road today and driver/passenger fatalities tend to be higher in older cars because they have fewer safety features.

Yeah yeah, I get the argument. The point was that you can't ding Tesla for "bad statistics" by using an argument that asserts "tend to be" without evidence.

In fact I personally doubt (i.e. I have no evidence and am not actually making this argument because it would be "bad statistics") this is true if you compare Tesla's to the BMWs being sold into the same demographic. Those "safe new vehicles" you're positing include a whole lot of crossovers and minivans driven in much safer ways.

Lots of data here:

http://www.iihs.org/iihs/sr/statusreport/article/42/4/1

higher in older vehicles

This seems intuitively that it would be true, but are there any stats on car age and accident frequency? Also where is the line on old? Vehicles made in the last 5-10 years will share most safety features, with all but a small minority of newer cars having some cutting edge. I'll admit I had been thinking of fleet age from the state of maintenance, but safety feature is a fascinating angle as well.. And the article you link in another comment was a fascinating read too, thanks!

For one there is the presence/absence of safety features (that one is obvious).

Then there is the actual design of the vehicles, cars that were made in the last couple of years fare much better in collisions with cars that are older because they are a lot stronger, and are better able to channel the energy of an impact to areas away from the passengers. Many newer vehicles will for instance shift the engine under the passenger compartment in a frontal impact.

Older airbags had 'best before' dates associated with the igniters being somewhat unstable and that reduced the certainty of them going off as they got older.

Then, finally, the bodies themselves, once cars start to rust you can patch them up but they won't be the same afterwards. There is quite a bit of redundancy in the body but it definitely doesn't help if there is significant corrosion.

>> You're going to have to explain to me how the wealth of a driver has any bearing on their ability to pilot a self-driving car.

It doesn't. But autopilot is only supposed to be used on the highway. Not in New York City. Not on dirt roads. On the highway. That's the easiest place to make a system like that work - adaptive cruise control and a simple lane follower go a long way there (I hope Tesla has something a lot more advanced than that). To compare stats on their limited use autopilot to the whole country is truly disingenuous.

Wealthy people tend to be older, which directly correlates to less impulsive or stupid behavior on the road.

People who buy a super expensive car have more to lose than some dude in a 10 year old hoopty.

Maintenance standards vary. We can presume that big brother Tesla maintained the car as they would have castigated the driver for not doing so. They are also the only game in town.

For the same reason those drivers will pay less for car insurance for almost any car they buy: they are actuarially safer drivers.

> You're going to have to explain to me how the wealth of a driver has any bearing on their ability to pilot a self-driving car.

One important nit: it isn't self driving. It's sometimes self driving.

About cohorts: how about this: young (thus inexperienced and more careless) drivers generally can't afford Teslas. So yes, wealth is an indicator for how accident prone someone might be.

Do you expect 370% above average for any cohort? That's amazingly amazing. That must mean that some other cohorts are correspondingly far below the average, what are they?

Thank you. Do people really want Tesla to use another company's failure / worse #'s via statistics in their review of this accident?

At least they are responsive, seemingly open, and seemingly empathetic. They are obviously handling things in a business first manner as corporations will but it hardly feels as inhuman as people seem to be making it out to be.

My question is honest. I found GP's assertion and its implication astonishing, but if that's how it is...

You forget the key rule: "Move fast and break things", including people and expectations of human decency

There's always someone who expects more. If you make a doodah that reduces (name something bad) by 30% someone will expect 40%, if it improves (name something good) by 50% someone will expect 60%, and by the way it costs too much, whatever it costs.

Name your expectations up front, please.

The expectation is simple: value human life. Don't try to do beta tests and simultaneously market it as if it's production quality. Accidents happen, but own up to them; don't immediately and fervently insist that you were not at fault before the investigation even starts.

It may be easy to casually update software and break builds, but when human lives are at risk a healthy dose of conservatism is appropriate.

But you're right -- it's probably unfair to expect a SV company to prioritize anything over profits (we saw that with Facebook, right?)

I thought about this over the weekend. About that you want, concretely.

Tesla says their cars have drive past that spot 85000 times so far. So you may be asking for some testing that's better at uncovering problems than 85000 whitenoise test instantiations are. Or you may be asking to test, and test, and improve until the software meets the developers' intentions: If their intention is to improve safety by 40% (or 400%, or 40000%) then you may be saying they shouldn't release a version that's improved over the baseline by 20% (or 200%, or 2000%) because that still doesn't realise the design.

Seems a bit absolutist to me. You could equally well make the opposite absolutist argument that having inhouse software that's better than the current released is putting people's lives in danger by inaction/negligence/whatever. That fixing a bug or improving a feature confers a moral obligation to release immediately, lest people's life be put in danger which would be avoided by the new code.

Less rich people, less healthy, driving smaller, older, less well maintained, less safely designed cars in more dangerous/remote locations. That’s a bunch of factors contributing to safety or lack thereof. Vehicle age alone is a huge factor in safety. Even the likelihood the driver and passengers are wearing a seatbelt is likely vastly different between Teslas vs all cars on the road. All those factors add up.

But that’s not even the point. The point is that the numbers Tesla gave are meaningless. Tesla’s stats would only be meaningful if you compared them to similarly sized/priced cars of the same age cohort with the same market segment. The fact that Tesla compares its luxury high-priced large sedans to all cars and not its own segment is itself very telling. They have those stats, and they would use them if they were compelling.

I've heard that before and it still sounds puzzling... Elon Musk has an ego the size of Mars, right? So he wants to sell autopilot-equipped cars to everyone, and his basis for comparison is everyone.

(Last posting from me today.)

Have you seen some of the drivers out there? Next time you’re in a parking lot take note of all the cars in terrible condition with scrapes and dings all over.

> Don't tell us how safe Tesla's are compared to the entire US driving population: Tell us how safe they are compared to equivalent vehicles from other manufacturers.

I'm having a hard time to find an exact number comparison, but:

- Tesla claims 1 fatality per 320M miles

Compared to

- A 2015 study of 2011 model year vehicles showed, for instance, Volvo XC90 had never been involved in a driver fatality (1)

- There are about 10K 2011 XC90s in the US (2)

- Avg. US drivers cover 13,400miles / year (3)

So:

- 4 years studied * 10K vehicles * 13,400 avg miles -> 536,000,000 fatality-free miles

This is obviously super hand-wavy, but I think it's fair to state as a hypothesis that Tesla could be twice as deadly as the safest high-end vehicles.. you'd need to design some experiment to attempt to falsify it to feel confident this is correct.

(1) https://www.freep.com/story/news/nation/2015/01/29/study-cha...

(2) http://carsalesbase.com/us-car-sales-data/volvo/volvo-xc90/

(3) https://www.fool.com/investing/general/2015/01/25/the-averag...

Probably should also note, that the average Tesla driver is likely male, has money (i.e. more risk taking), etc. Essentially, compare to sports cars as opposed to high end vehicles.

They we're originally advertised as sports cars really. 0 to 60 faster than any other car, etc.

Eh, you can't cherry pick specific (model, year) combinations after the fact without overfitting. You expect some variation in deaths per (model, year) and by searching for the least deadly ones, you'll dramatically influence your estimate simply because you used a maximizing search instead of a broad survey of a-priori safe models.

This is totally true, and I agree; I tried to guard against this at the end with the "I think this is enough to state a hypothesis, which would then need to be tested.." paragraph.

However - the same is true for 2009, 2010, 2011, 2012 model XC90s, and they are just one of 10 models that had no fatalities in those four model years in the study. I just picked that year and model to get one data point for mileage to compare; the other model years of XC90 have similar sales, so the same applies.

So, while you're right, I don't think it's an entirely useless comparison; it indicates there could be something here, which could then be used to state a falsifiable hypothesis, something like: "Similar cohort drivers would be safer in <one of these ten models> than in a Tesla.".

At a minimum it indicates that Tesla's numbers do not seem exceptionally safe, even if you can't say they're less safe than comparable competitors either.

Given the low number of incidents, you probably want to be looking at something other that fatalities to get a large enough pool of incidents to reduce the impact pure chance; at least in addition to actual fatalities. It's not going to be an easy comparison, that's for sure.

I'm not sure how much of that is about the car versus the driver. Volvo drivers are some of the most slow, excessively cautious, annoying (to others) drivers on the road.

They should (on closed tracks) test self driving cars with drunk and deliberately stupid drivers. Do they do this? Because if you mass market such a car you will get people using it to drive themselves home drunk.

They can be far more specific: just tell us how Model S drivers without (or not using) Autopilot do compared to those with and using it. This would control almost every variable except Autopilot. I find the fact that they don't release this number as extremely damning.

Somebody has tried to do this [1]. It would be nice to see this analysis from Tesla themselves. As you say, we can probably guess why we haven't.

[1]: https://www.greencarreports.com/news/1107109_teslas-own-numb...

This is by far the most important point in this discussion. According to this article (https://www.greencarreports.com/news/1107109_teslas-own-numb...), Tesla's statistics are deliberately wrong, and Tesla compared its Autopilot crash rate to the overall U.S. traffic fatality rate—which includes bicyclists, pedestrians, buses, and 18-wheelers.

You could argue that Tesla should be compared to luxury vehicles (which have more safety features), but even if you just compare their crash rate against U.S. drivers of cars and light trucks, the Tesla Autopilot driver fatality rate is almost four times higher than typical passenger vehicles...

And look who fell for it (techcrunch):

"However, it seems the driver ignored the vehicle’s warnings to take back control" [1]

Followed by the quote from Tesla that at some point in the history of the ride a warning of some sort was given.

[1] https://techcrunch.com/2018/03/30/tesla-says-fatal-crash-inv...

yeah that's pretty bad. I wonder if they'll correct it.

Also, ever notice how in none of the Tesla Autopilot accidents, Tesla finds itself guilty?

I don't even read these posts from Tesla anymore because I know each one of them is just justification for how Tesla did nothing wrong, and it's the fault of the stupid human driver for using its Autopilot "wrong" and dying in the process. Meanwhile, Tesla is laughing all the way to the bank selling the usage of Autopilot as one of the primary features of its cars.

It almost reminds me of how Coke markets its sugar-filled soda "Drink as much as you can at every meal and at any event...but you know, in moderation (because that's what the government makes us say with its dietary guidelines)".

Tesla is kind of like that. It tells people to use Autopilot because it's such a great experience and whatnot, but then stuffs its terms and conditions with stuff like "but don't you dare to actually lay back and enjoy the Autopilot driving. You need to watch the car like a hawk, and if you happen to ignore any of our warnings, well it's your stupid-ass fault for ignoring them, even though we wrote everything there in the 11-point font." It's very irresponsible for any company to do stuff like this.

I don't know what the exact process is now, but I wish the authorities would be able to get access to un-tampered logs of the accident after the fact from the self-driving car companies.

I don't care if it's a car company that has "Elon Musk's reputation" (who I like) or Uber's reputation. These accidents, especially the deadly ones, should be investigated thoroughly. No company will ever have the incentive to show proof that leads to itself becoming guilty. It's the job of the authorities to independently (and hopefully impartially) verify those facts.

I'm a big fan of Tesla and its electric vehicles, big fan of Tesla Energy, big fan of SpaceX. But I've always criticized them for the Autopilot because it was obvious to me from day one that they're being very irresponsible with it and putting the blame on humans, when they were marketing it as a self-driving feature (they did for years, even if they don't do it as much now, but they still use the same name), while its system is far from being anywhere close to a complete self-driving system, and thus putting anyone who uses in danger of dying.

It reminds me of South Park's drink responsibly ad [1]. You could almost replace 'drink' with 'drive' and it would sound like the current autopilot spin.

[1] - https://www.youtube.com/watch?v=j3osSJSGInQ

Tesla's huge success in people's mindspace of self-driving vehicles is baffling to me. Correct me if I'm wrong, but these are the self-driving features in Teslas:

- Cruise control maintains speed, will automatically decelerate if a car in front slows down, speed back up if they speed or go away, etc.

- Collision avoidance: the car will automatically emergency brake if it believes a collision is imminent.

- Lanekeeping: the car will stay in the lane without driver attention, although if it doesn't detect your hands on the wheel with enough frequency, the system will disengage.

- The car can park itself.

A quick look at Wikipedia tells me it can also change lanes and be "summoned."

Most of these features, especially considering cruise control, collision avoidance, and lane-keeping as the core featureset, since at least in 2017, have been in wide deployment among most car manufacturers, down to even cheap economy cars, that people are buying right now without getting on waiting lists or anything of the kind. You can buy a car right now with this core featureset for around ~18k.

Not only that, people seem to dramatically overestimate what kind of auto-pilot Teslas have. Speaking with my doctor about it, he assured me that "you just tell the car where to go, and it goes there." How did Teslas get so hyped up in the common man's imagination?

Yeah, I'm going to call BS on this one. More specifically, on the lanekeeping claim. The current non-Tesla implementations are worse or much more limited at the price point you mentioned (and frankly, at most price points).

All I can tell you is that the Honda lanekeeping does everything I would expect or want, and its the only one I have a lot of experience with. The only thing that annoys me is that it requires you touch the wheel every now and then.

Can you elucidate on what tesla lanekeeping does that others don't?

> It's very irresponsible for any company to do stuff like this.

It's very irresponsible for regulation to allow them to do that. And statistically for us to vote for politics that are not against that kind of behavior. Big companies rarely give a shit about killing people in the process of making money, as long as they are not punished for doing that. (And we, btw, mostly don't give a shit when the people being killed live in other countries, even when the process to do so is far more direct than the 1st world problem that interest us here.) People who used to organize those kind of criminal behaviors used to be held responsible for the consequences, but it is more rare now and the laws in various countries have actually been changed so that people managing companies are far less accountable than before for the illegal and otherwise bad things they order. As for companies being accountable, it is usually in punitive money that is order of magnitude less amount than the extra profit they made in doing their shit, so why would them stop? The risk of the CEO being sent to jail would stop them. Ridiculously small fines against the company, in the astonishingly rare cases where they even actually fined, not so.

Totally agree, which is why I refuse to automatically be hyped about everything that Musk does.

Tesla always blame the "driver" (and publicly so, nonetheless) when they are killed by their "autopilot" (which is actually driving, that is fulfilling the purpose indicated by its name, despite being not very good at it...) -- either directly, or by carefully stating misleading statements that are both technically correct and incredibly dishonest in the context of both the events and the surrounding text.

Why people are still using their shit is beyond me. Why they are still authorized to sell it under this marketing (like the name given to it, or the mismatch between the restriction and the practice), I also can't understand: it should be better regulated by the authorities.

This is probably the worst part about driving a Tesla: Your car is going to testify against you. Because Tesla can and will make a statement like this, and they will always throw you under the bus. Which is a terrible way to treat a customer, as a general practice.

> Your car is going to testify against you.

Presumably Tesla/the car is telling the truth as it saw it. I don't think you're suggesting they're falsifying the logs.

> Which is a terrible way to treat a customer

I'd assume that Tesla sells you a vehicle, not PR or legal services that protect you in case of an accident. Your expectation that Tesla should take the blame is unreasonable by any standard.

I'm not saying Tesla is lying. But I am saying that I could spend six figures on a car from a company that is ready and waiting to stab me in the back. Or, I could spend five figures on a Toyota, knowing that Toyota is not going to sell me out if my car crashes.

I mean, look, it makes sense Tesla has to defend itself, especially legally, but there's a big difference between pulling out the logs when you need to for the lawyers and the regulators... and making a public blog post blaming the customer.

I think you're absolutely correct, and moreover that the difference comes from that it's in Toyota's best interest to think about the long term, play good guy and dip into their rainy day money for any costs associated with that.

Whereas with Tesla, they're leveraged out to here with stock valuation based on promises and projections, and if their key tech is perceived as seriously flawed, they're not going to exist as a company in the long term.

As far as I can tell, Toyota (or other car manufacturers) would be just as inclined to argue driver error in crashes (especially once involved in litigation). See e.g. https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_... for a discussion of some Toyota crashes.

I think there's a significant difference between a company who will defend itself reasonably in court, and one which is actively collecting telemetry on your car, and will proactively write a blog about how badly you drove.

There is a nuance here though: unlike a "classic" car, tesla and other modern cars clearly have access to a wealth of data about your driving and your crash. The scope of lying by omission, of even simply misinterpreting the wealth of data is much larger.

I do think it's reasonable to ask of them to either keep completely silent; or to be completely open - perhaps by giving complete access to all data to a truly independent third party, and not picking which facts they think are fit for public consumption, and which are not.

I admire what Tesla has achieved - but this is a company under considerable pressure. They may well go bankrupt; and every incident matters. They are under terrible pressure to massage those facts to make them look good. I can't help but treat any statement like this no differently to any other ad - it's easily possible to be misleading even when there is some possible interpretation that isn't an intentional untruth.

Everybody knows people aren't good at sitting around doing nothing and maintaining focus. Distracted driving isn't a new risk. So the question is: are the safety benefits to a driver aid really greater than the risk they cause by encouraging distraction? Even if the driver aid catches 99% of all risks, that may yet be a net negative. Personally, I don't think it's reasonable for a system that encourages distracted driving to then claim it was the driver's fault they were distracted. No amount of small print or big fat warnings can ever excuse ignoring reality. If the autopilot is contributing to driver inattention, then the driver's inattention is no longer just the driver's fault. Not that I'm at all sure that's what's happening here, but there are some ominous indications, that is for sure. Without an autopilot, who takes their hands off the wheel for more than 6 seconds on a drive like that?

You know how many plane crashes are not the fault of human error? Very few.

I have a friend that investigates planet crashes. He’s told me that between pilot error, human error, hubris, flying when conditions are too dangerous, poorly trained or sloppy mechanics only about about 1% really can’t be classified as being caused by humans. He said the stats are a bit skewed because like any organization that reports on human performance they’ve been influenced to be a bit “flexible” when assigning blame.

Some of this is a quirk of how the NTSB reports accidents. For instance if a small plane suffers engine loss in flight, the cause of the accident will be listed as "failure of pilot to land the aircraft safely without power". Which is technically true, but most ordinary folks would blame the engine failure for the crash, not the pilot. It reflects a culture where the pilot is absolutely responsible for the aircraft and its passengers. That's the exact opposite of where the self-driving car industry is going.

> planet crashes

I want this job. Although I'm pretty sure all planet crashes would be the result of gravity.

Human error is distinct from driver/pilot error. It's possible (though quite unlikely) that there 0% pilot error in all plane crashes, and 99% human error.

Given that plane auotpilots are roughly comparable with Tesla's autopilots, one wonders how many plane crashes have occurred while the plane was in autopilot, and what fraction of those crashes were attributed to pilot error.

I think the regulations will come soon enough. Tesla, Uber, etc are trying to get as many miles booked as possible before this happens.

Don't you think being honest about their fuckups in situations involving death is just a massive lawsuit? You can't do that in America.

I also dislike how Musk is synonymous with Tesla, except in cases like this.

The defensiveness, narcissism and lack of true leadership stamped all over this press release scream Musk to me.

>> That is Not acceptable of a self driving system.

Yes. But as we all know autopilot is not the same as self-driving.

>> And no, you can't get out of culpability by claiming statistical superiority.

In fact no one has found Tesla culpable for the accident. You can however defend the safety of your system by referring to statistics. (In fact that is the ONLY way to measure safety level) And your analogy does not hold. It it more like a parachute company saying their parachutes are safer than other brands because it fails less often than the other brands. Which brings me to my final point. Even if it turns out Tesla was culpable and negligent in this accident, it would not be a complete judgment if we leave out the inherent risks in car manufacturing and the incidents attributed to other companies.

"And no, you can't get out of culpability by claiming statistical superiority"

Every MBA student needs this tattoo'd on the back of their hand in order to graduate from now on.

>> The driver had received several visual and one audible hands-on warning earlier in the drive

> What they are saying is that while he was driving there was an earlier point at which the car did not crash, where the car gave a visual and audible clues. And you have to ask? Why the hell is that relevant?

I see it being relevant that the driver had his follow distance set to the lowest and that several times in the drive prior to the accident he had to be prompted to maintain his hands on the wheel. This speaks to a driver that wasn't paying attention like they were supposed to be doing. That would increase the likelihood of him missing an upcoming hazard and responding in time.

Regardless of that, if he had reported and issue in that area to Tesla, why the hell was he not driving it himself at the time? I know that I personally value my life enough that I wouldn't trust AP if it had been veering into the wall repeatedly in one area.

Sadly I have to agree with you on the narrative angle. While I am very impressed with what Tesla has accomplished in their autopilot I find the careful construction and omissions in this follow up post to be done poorly.

They omit what speed and what lane the car was travelling in. They omit the time at which the car was no longer behaving in the way in which it had (when did it start heading for the crash barrier? At what point did the wheels closest to the barrier move over the line? etc)

They have previously mentioned that their cars have navigated this spot many times in the past (over 200 as I recall) did they note the number with and without the crash barrier intact? How many times was the car in the exit lane and how many times in the lane to the right of the exit lane? How many times have they navigated past it with a damaged crash barrier.

The driver had driven this route before, it was his commute to work. Had he driven it with this car on that route in autopilot before? Did it work then? How many times? How many seconds were there between when the driver's car was doing something right (in lane etc) and doing something wrong (straddling the lane). The response indicates there were 5 seconds of visibility coming up to the crash barrier, at what point was the autopilot on a collision course? All of those five seconds? or the last 500mS ?

Reading the Tesla response I felt I wasn't getting the whole story, and was getting a whole lot of deflection. That isn't the tone that inspires confidence in me. And that makes me sad because they have done much better in the past. I cannot help but speculate that this time they feel they might have been contributory at least and that a full disclosure would be used against them in a civil suit.

I'm pretty sympathetic with PR departments doing what they have to do, in terms of putting out the official boilerplate remorse, the promises to cooperate and keep people up to date, as well as publishing the incident data, and then keeping mum until the investigation is finished.

But Tesla is shoveling in the bullshit pretty early on. Besides the fuzzy and deliberately vague stats (why is the mileage attributed to Tesla's "equipped with Autopilot hardware"?), this graf is just despicable:

> Tesla Autopilot does not prevent all accidents – such a standard would be impossible – but it makes them much less likely to occur. It unequivocally makes the world safer for the vehicle occupants, pedestrians and cyclists.

"unequivocally" has an actual meaning: leaving no doubt. And yet it's used following the paragraph that pretends Tesla's and all U.S. vehicles are directly comparable. I know it sounds silly to focus on that word but someone in Tesla PR thought it needed to be used with the pile of crap numbers. Such blatant and pointless deception -- "significantly" would work just as well for that inane sentence -- feels like a strong a signal to doubt the rest of the info given, as if the evidence weren't already so obviously dubious.

Like you’ve predicted, first headline on Techmeme: “Tesla says Autopilot was engaged during fatal Model X crash on Mar. 23, and the driver didn’t respond to warnings to take control of the car before the crash”. Although the linked article has a different title; might’ve been changed.

Vehicle notifies the driver it's having difficulty operating the driver elected auto pilot mode based on the conditions. Driver elects to continue using the failing operating mode despite being warned.

I fail to understand how "It isn't" relevant that the driver chose not to take manual control when warned that conditions were not ideal.

To me there is a stunning lack of compassion and decency in the response to these incidents by both Tesla and Uber.

Uber made sure to point out that the victim of their incident was homeless. Tesla is pointing out how the driver received unrelated cues earlier in the journey. None of this information is relevant. They’re trying to bamboozle the reader in an effort to improve their images at the expense of victims who can’t defend themselves.

I don’t understand why it is so impossible for these companies to act humbly and with a sense of dignity around all this. I don’t expect them to accept responsibility if indeed their technology was not to blame, but frankly that isn’t for them to decide. Until the authorities have done their jobs, why not show remorse, regardless of culpability, as any decent human would?

This was clearly vetted (or written) by a lawyer who is interested in changing the narrative for his future wrongful death defense rather than exercising empathy.

It reads like it was written by a company who has a lot of scientists working there is all. They are simply explaining what happened according the system log files before the incident and then reminding the public that it is still safer.

> They are simply explaining what happened according the system log files

I must have missed the part where they explained why their autopilot system drove into a concrete divider at high speed. Please can you quote the explanation you're referring to.

Yeah, but it seems that they may be, statistically speaking, right. The fears that surround self driving cars may be unfounded, and if they are unfounded, then it would be a shame to throw the baby out with the bathwater.

They're right when they say we don't speak of the accidents that didn't happen, and I bet there's a ton of them.

As someone who's been in multiple car crashes, as a passenger, I really on longer want to be at the mercy of human drivers.

Brief reminder of the same discussion, in a different context.

https://www.youtube.com/watch?v=bzD4tIvPHwE

Yet on one, and I really mean no one complains about computer assisted landings and takeoffs. I'm not sure why. Lots of passengers even sleep through them.

I guess we've come to realise that the man machine symbiosis works well for flight. But it might take time for this to get engrained into our culture when it comes to driving.

Exactly. It's very important to ensure negative (and unjustified!) publicity do not affect technology perception, adoption and growth. There are so many Luddites who take every chance to hammer another nail.. It's understandable that any tech pioneer can easily sense being attacked and has to quiveringly rebuff any hysteria attempts.

The main thing that would make me convinced of the safety of self driving cars would be if the executives of self driving car companies took criminal liability for accidents caused by their cars or software.

When the cost of them killing somebody by having a deficient QA process is minimal (and just affects shareholders), then the cars will be rolled out before they are ready.

What will happen to liability at present is still unclear but if the self driving car companies are successful in dodging most of it (and it appears they are trying) then self driving cars are going to kill a lot of people unnecessarily - quite possibly more than humans would.

If individual engineers and executives were made to be legally responsible for potential fatal accidents resulting from technology, progress would grind to a halt.

I mean, in cases where they are criminally negligent, they are already responsible for accidents.

The tricky part is figuring out what constitutes negligence.

I dont think that's the case. However, if the people who build it refuse to take responsibility for their defects killing people then I think that's the clearest possible signal that the technology isn't viable.

Obviously this would also mean that compensation would have to go up for self driving car engineers and execs to compensate for the risk. That's fair, and, if that cost makes a self driving car venture uneconomic - again, not viable. And that's okay.

People take responsibility for other people's lives every day when they get in a car. I don't think the idea that Uber engineers should do the same should be considered particularly controversial.

I agree with your intention, though isn't it inevitable self driving cars will kill people even if they are 1000x safer than humans?

If a CEO has many millions of cars running his software, their will always be a death.

In that case, nobody would want to take on that responsibility so nobody would build self driving cars.

Therefore humans will still drive cars and there will be 1000x more deaths, most preventable with self driving cars.

This is pretty much exactly the Trolley problem in ethics and philosophy. Read up on it :)

They don't need to be 1000x safer than humans. If they are 2x safer than humans, that is plenty for me to support using them. The issue is determining how safe they in fact are, to appropriately high confidence, before putting people at risk.

That is something that "tech" companies will in all likelihood never be able to do, since they seem uniquely designed to betray and lose the public's trust at the earliest opportunity.

"Tech" companies have become far too culturally accustomed to dishonest hype, "growth hacking", marketing BS, PR spin, etc. for their statements to be trusted, especially when there are life-and-death implications.

It's not always true. Waymo is a clear exception: their car was at current Tesla's level 5 years ago, but they chose not to run into production. Many Google engineers left from the team because they wanted to see their work in public, but Larry clearly wanted safe L4 driving before getting to the roads.

(of course Google did lots of ,,ungoogly'' things in the past few years, but at least they got Waymo right)

It's not the trolley problem because as yet we don't know how many people would be killed by rolling out self driving cars and, more to the point, the speed of rollout and safety are not necessarily inversely proportional.

By characterising self driving cars as a "trolley problem" you have presented what is known as a "false dilemma". Maybe you could read up on that ;)

Good point! Are there any publicly accessible crash/death statistics for cars with a similar price/safety/demographic profile as Teslas?

I feel that is the only way we could isolate for self driving and not other factors.

But is it really unjustified? The statistics we have so far suggests that with the current state of self-driving cars they are significantly (about an order of magnitude) more likely to be involved in a fatal crash.

Maybe what we need is to put the brakes on the notion that self-driving cars are nearly ready for prime time and in a few years they will be all over the road. A level-headed review of the state of the technology suggests that is extremely unlikely. Except for people hanging around /r/futurology and to some extent here on hacker news, who are perhaps too close to technology to really appreciate it's limits in the practical world.

It would probably be for the best if the enthusiasts and the companies rushing headlong towards this new driving paradigm put forth some effort to tone down the hype.

> https://www.youtube.com/watch?v=bzD4tIvPHwE

What exactly am I looking at? The video itself says it's a computer-piloted takeoff, the YouTube title says it's a computer-piloted landing, and the top-voted comment says it's a human-piloted fly-by.

edit: it appears to be this flight: https://en.m.wikipedia.org/wiki/Air_France_Flight_296

I agree with the lack of compassion bit -- I think the messaging could have been far more empathetic. However...

> Uber made sure to point out that the victim of their incident was homeless. Tesla is pointing out how the driver received unrelated cues earlier in the journey. None of this information is relevant.

I don't understand how you can equate those first two lines. Uber's observation is clearly irrelevant, but the fact that the Tesla driver received multiple "get your hands back on the wheel" notifications, as close as six seconds before the accident seems very relevant to me.

> the fact that the Tesla driver received multiple "get your hands back on the wheel" notifications, as close as six seconds before the accident

But that isn't what their statement says. It says the victim had his hands off the wheel for six seconds before the crash, and that he received hands-on warnings "earlier in the drive." It does not say that during those crucial six seconds he was being warned. Nor does it explain the fact the car plowed into a barrier.

”Nor does it explain the fact the car plowed into a barrier.”

This is critical. No matter how many notices you give, slamming into something at speed is the wrong answer. It’s almost unbelievable that they’re trying to use that as an excuse.

I'm finding it hard to believe the wording on this crucial sentence was so misleading by accident. Upon first reading it, the wording seemed a little off and seemed to me that the two statements should have been two separate sentences instead.

Even then, I fail to see how it’s relevant information.

I’m supposed to trust an “autopilot” that warns me six seconds before it slams me into a wall?

In their next statement about the next person they kill: 'The Autopilot clearly both stated in vocal synthesis and by displaying on the internal screen: "WARNING - I'll attempt to kill you in six second". Yet the driver did not manage to regain control against the rogue AI. He is clearly at fault for having died, and his family should be fined to recover the cost the the investigation.' :P

I'm willing (and almost hope) to be proven wrong on this, but I personally find it hard to believe that the ambiguity around this in the statement is accidental. A statement of this nature will have been extensively wordsmithed.

It seems that it was written in a way that confuses the issue intentionally.

>the victim had his hands off the wheel for six seconds

Actually it doesn't even say that, it says his hands "were not detected." Which means nothing to me.

@jVinc's post answers this. They didn't state specifically that he was warned six seconds before the crash.

Could you source the assertion that Uber (as opposed to the local police) has indicated the victim in the crash was homeless?

(That would be an inexcusable and cynical deflection.)

Uber never came out mentioning the homelessness of the person. The police did. This is typical HN banter on Uber, thanks for calling it out.

That's because Uber never released an official statement on the accident. At least, I can't find one on their website.

That is utter bullshit that Uber said the victim was homeless. It was the police chief, Uber never mentioned it once and their response was completely appropriate.

It's fashionable to hate Uber now so people don't object to lies told against them.

> I don’t understand why it is so impossible for these companies to act humbly and with a sense of dignity around all this.

I would venture a guess this is an outcome of the "sue for everything" legal environment in the US. Any statement that could be construed as anything other than "we did nothing wrong" could be seized on by shareholders suing the company for securities fraud and demanding class action status.

What you are saying here is: "I really wish the PR person who wrote this chose words that would cause readers to feel as though the company empathized more with the victim."

Ultimately companies don't have feelings, their employees do. PR statements and the words chosen (unless written by an individual like the CEO) don't somehow make a company into a person and don't reflect the feelings of their employees: they are a crafted tool to create a certain outcome by instilling thoughts in the mind of the reader and/or providing legal cover.

>I don’t understand why it is so impossible for these companies to act humbly and with a sense of dignity around all this.

Because humility requires you to admit wrongdoing, which from a legal standpoint is not advisable in a public statement.

Really not much additional info here aside from the number of warnings the driver had. Also, the standard "Tesla is 10x safer" metrics that get pulled out each time a crash gets sensationalized.

What I think they fail to address, especially in this case, is that the autopilot did something a human driver who was paying attention would never do. Autopilot does a great job of saving people from things even a wary driver would miss, much less a negligent one, but the fatal accidents that occur in statistics are not from people who are fully watching the road missing the fact that there's a concrete barrier with yellow safety markings directly in their path, and hitting it head on for no good reason (e.g. evasive action because of another driver, etc, oh a stupid last-minute "oh crap that's my exit").

I want autopilot to succeed, and I want Tesla (and Musk) to succeed, and for the sake of their public image they have to realize that this isn't an average accident statistic, a lapse in attention or evasive maneuvering. It's a car that seemingly plowed right into a concrete barrier while still under complete control. That's not a mistake a healthy human will make.

> the autopilot did something a human driver who was paying attention would never do.

Then how did the crushed barrier get crushed before the Tesla hit it? Clearly, the stretch of road is unsafe enough to trick humans drivers (and, clearly, Tesla should improve).

I'm obviously not privy to the details of the prior crash, but I'm pretty sure any healthy human would not deliberately drive into the barrier.

The most likely scenarios I can think of would be not paying attention and drifting into the barrier, attempting to avoid a car merging into the lane (and not paying enough attention), or being struck by another car and being forced into the barrier.

At that particular point, what I see the most is people deciding that they either need to exit or not and abruptly crossing from one HOV lane to the other. Lots of crazy shit. So I'm not sure your scenarios have much relevance -- how many times have you driven through this intersection? I've been through it maybe a thousand times since it was built.

I’ve seen people back up in the breakdown lane on the expressway because they missed their exit. But that was Beijing, and all drivers are hyper sensitive on the lookout for that kind of stuff.

On my first drive into Bucharest a guy drove on the wrong side of the divided highway to take a left into the lot of a building supplies company right in front of my car. Scared the crap out of me.

https://www.google.com/maps/@44.436552,25.9597006,18z

Can confirm, have just driven for 2+ hours on the streets of Bucharest and there’s no way for an AI-like car to drive in here for more than 2 blocks without getting involved into an accident. Unless said car becomes sentient, but even in that case it would still have to “guess” some of the crazy stuff the other drivers on the road may be up to or to expect that a barely marked road-construction thingie will suddenly show up just in front of the vehicle while driving at 60 kph. And Bucharest has got nothing on a city like Istanbul.

It's the only place where I got honked at by a car on the sidewalk while walking on the sidewalk, and when I didn't get out of the way fast enough he proceeded to attempt to nudge me with his car.

I think the answer is that the path into the barrier looks just like an actual lane, and that this was enough confusion that a human driver apparently had made the same exact mistake a week earlier. https://imgur.com/a/iMY1x

https://techcrunch.com/wp-content/uploads/2018/03/screen-sho...

The lanes aren't very well indicated, but given how visible the divider is, I don't think either an autopilot or a driver should drive off.

Reminds me of a scene in "La grande vadrouille" where a motorcyclist is killed on a mountain road because he was following the dotted line and the painter had gone to the side of the road to take a break.

Also, that's an immense pothole!

> That’s an immense pothole!

... well you certainly aren’t from southeast Michigan, that’s for sure.

CalTrans doesn't diagonally stripe spaces like that? It would seem that they should.

There has been a lot of repaving work on 101, so these may well not be the final markings.

I don't think that area has been repaved in a while.

This divider looks rather unforgiving. Is it common for dividers in US to be just plain concrete blocks without anything to prevent rapid deceleration?

If you look at the imgur link you'll see a impact barrier that had previously been crushed and not reset or repaired.

I dont see it

It's in the techcrunch image, not the imgur. The barrier with the yellow face is designed to collapse.

https://techcrunch.com/wp-content/uploads/2018/03/screen-sho...

that image also makes me wonder if tesla's system is designed to identify knocked-over cones. It's one of those things that would be easy to code for, but also easy to overlook.

It's quite hard if you take into account that machine learning needs 10000x the training data that a human needs, and Tesla is sold in 30+ countries (and used in much more).

I think 3d mapping the surrounding world correctly using multiple cameras (what humans are doing) is more generalizable.

That still looks pretty risky to me - I compared a similar junction here in the UK near to where I live (on the M90 in Fife) and it has about 40 impact attenuation barrels in a triangle.

Those are also present in many locations in the US, although they seem to be phasing them out so I think they are considered old tech. I don't know the performance specs, but new cars are a lot smaller and safer than old cars - the same is probably true here too.

Except those solid white lines delimit an uncrossable barrier delimiting the HOV-specific (during commute hours) flyover from the 101 lane, which is well documented for nearly a mile preceding that point.

Reasonable, lawful, absolutely. People break the law, people get drunk, people get sleepy.

There's a spot just like this in Houston on 610 East. HUUUUUUGE flyover as the HOV lane spends about a half mile merging into the left lane of 610 (nice fat fast freeway, they need the runway). A guy had thought the flyway would be a safe place to park his car and wait for a tow. Safe enough that he was sitting on the hood of his car. Sure as shit the Challenger in front of me got confused, cut into the flyaway thinking it was a lane, hit the disabled car, and sent the guy flipping up into the air a good 10, 15 fit. Fucked up accident.

Anyway point is after the accident I was talking to the driver of the Challenger, trying to calm him down (kept saying "holy shit I fucking killed that guy!") and other that shock he was sober as a duck. He got bloodtested and everything, clean. In court he said he just got confused and thought it was a lane.

LONG STORY SHORT human error mang, human error. Still not sure why a car was able to do it.

They have those same white lines leading into intersections and people cross that shit all the time. People speed.

I read that that driver was arrested for DUI.

Really, I did: http://abc7news.com/automotive/exclusive-i-team-investigates...

If you are building an autopilot system, I’d expect 1) identify black/yellow hazard signs and 2) don’t hit them to be basic features by version 0.5. Seems like a big miss to run straight into that.

Maybe SDC already do this, but it seems like the Agent should keep track of things that look like potential lanes and then if the potential lanes turn out to be not-lanes (have barrier in them) the Agent should flag the area that looked like the start of a lane so future SDC know to avoid it.

The entire thing is two solid white lines on either side, which are pretty strong indicators that you shouldn't pass into that area in the first place, diagonal stripes or not.

It also indicates that you shouldn't pass out of that area.

The vehicle should have at the very least stopped short of the road obstruction. But we don't yet know what led up to being in that lane, I was hoping the article in the OP would have taken us through it. Instead I'm getting the feeling this was a catastrophic software failure. Without more transparency on the issue, despite what Tesla may prefer I don't think I have any choice but to feel uncomfortable.

You would think they would at least add pylons leading up to the divider: https://ops.fhwa.dot.gov/publications/fhwahop13007/images/fi...

The divider is from the fast lane no less.

Does seem like a problem that a LIDAR based system would have been less likely to make, compared to AI on video classification

I wonder if the problem is the uneven pavement leading up to it. The white solid line starts close to it, then drifts left. In certain lighting conditions, the division in the pavement might seem more prominent than the white line, which I could imagine tricking a human or computer to follow it instead. That wouldn't cause a head-on crash itself, but would cause the car to get very close. I could then imagine a computer trying to correct its mistake by moving RIGHT (because that area looks like a lane) instead of left and going head-on into it.

Though of course that doesn't explain why it didn't recognize the barrier as something it should avoid at all costs. Unless perhaps something to the left confused it and made it think there was an obstacle there, too, thus causing it to think it was going to crash regardless. If so, maybe it did try to brake at the last second?

It's not that straightforward. It's two adjacent lanes, which gets separated. Left one becomes an exit, the lane next to that one continues. The space between them keeps growing. The two lanes in question have a solid line between them before they separate to indicate you can't cross between them.

If you step a bit further back, it does not look so much like an actual lane: https://teslamotorsclub.com/tmc/attachments/streetv1-png.288...

A piece of software that mistakes that wedge for a lane in broad daylight has no business being deployed on the road, in my opinion.

  any healthy human would not deliberately drive into the barrier

But plenty have driven negligently into barriers.

The worst such crash I've ever seen was barely 2 miles from there, on northbound 85 near Fremont Ave. There is a soundwall that comes to a connection point where a wall segment is directly perpendicular to the freeway. For some reason, the guardrail had a gap there almost exactly the width of a vehicle.

A few years ago, a vehicle veered off the right shoulder and perfectly threaded the gap, into the wall at full speed. It was compressed to maybe 5 feet long.

Prior crash was a DUI.[1]

[1] http://abc7news.com/automotive/exclusive-i-team-investigates...

Distraction, misjudged timing, sudden health problem, etc

Previous driver was drunk.

It's a somewhat easy accident to have in heavy traffic if the lanes aren't marked clearly and you don't see the barrier because of cars ahead of you. Not a factor in the Tesla crash (they had a clear line of sight) or in the original crash (which was a DUI).

it's a mistake that healthy humans make all the time, which is why those crash barriers exist in the first place, and why that particular crash barrier was damaged by a previous crash.

You’re missing the point. The crucial fact according to OP is that the car did something that a fully aware driver would not ever do. It’s at least worth acknowledging that.

"Fully aware" drivers (true Scotsmen) as proposed here statistically do not exist, or at least do not compose the vast majority of drivers. So it is kind of a meaningless comparison to draw.

It's not a True Scotsman in this context. The point is that humans tend to crash into things they're not fully aware of, short of self-harm.

Either:

A) The autopilot could not see a concrete barrier in its path. B) More likely, and as the story reports, it WAS aware of the danger but didn't do anything.

Either case is at least worth discussing, no?

In which bucket would you put the possibility that the computer observed evidence of the barrier, but the evidence that there was no barrier -- the prior that barriers don't exist in lanes of traffic -- was too strong?

Bucket A. Which could be articulated more generally as: failure to capture the world as it really is.

"tend NOT to", did you mean?

"Fully aware drivers" are all the many drivers who manage to navigate that area without driving into the barrier, minus the ones who didn't mean to get off there or meant to get off and didn't. The latter groups are aware enough to drive safely through the area but still spacing out a little.

It sounds like peering into the black box of autopilot and anthropomorphising some parts of it but not others. Why not say "autopilot sometimes drives into barriers and also humans sometimes drive into barriers."? In both cases the driver/autopilot failed for whatever reason.

That's disingenuous.

The only reason a human drives into that barrier is if they swerve to avoid something or are otherwise not properly paying attention (texting, etc.).

There's no good reason for the autopilot to have hit that barrier.

The mental gymnastics going on in these comments attempting to absolve Tesla of any responsibility are truly next-level.

A swerve is not the only possibility, as the area between lanes looks like a lane. My father has done the same thing while driving at night. He insisted he was in a lane while I yelled repeatedly that he wasn't. Luckily he changed lanes / entered a real lane just before we crashed.

Yes, bad visibility could explain it, but Tesla themselves say it was 9.30am and the autopilot had "five seconds and 150 meters of unobstructed view of the concrete divider".

Looking at a photograph of the barrier, a poorly designed median also explains it.

Nobody is trying to absolve Tesla. They're just refuting a dumb and easily rebuked claim that no human driver would make the same mistake. There is evidence that crashes happened there before, and it would be easy to imagine someone crashing there because they were focusing on trying to find a spot to merge while driving in what appeared to be a standard lane.

Again, not absolving Tesla, just being realistic about the capabilities of human drivers.

It's only anthropomorphizing in the sense that we're discussing two systems capable of propelling automobiles around. Presumably, they're both in the business of avoiding concrete barriers.

If one system is steering the vehicle into things that the other system would, in most conditions, reliably avoid, it bears some discussion.

Drivers crash at this particular point all of the time.

I've been a fully aware driver unsure of what to do when seeing a surprise in my lane. I only hit the break because a passenger started screaming. I was so confused to see a pedestrian on the freeway.

When in doubt slam brake.

Just a little nit-pick - «When in doubt, quick glance in rear-view mirror, then slam brake.»

It turns out brains can freeze in a panic.

I thought the auto pilot warned the driver to take control of the vehicle or is that incorrect? There is probably a lot more we don't know and it would be premature to place the blame on the auto pilot and Tesla. This isn't a lvl 5 system. It still requires the drivers attention and that is something we need to consider ( he obviously didn't have his hands on the wheel).

Technically tesla's statement says he was warned at some point to take control during his drive. That could have been 5 seconds into his commute, or it could have been 5 seconds before he hit the barrier. He may have taken control at that time, and then reengaged it later. He may or may not have been warned to take control before hitting the barrier.

Tesla's statement is not clear on this at all.

>or it could have been 5 seconds before he hit the barrier.

If that was the case, they would have said so. Their vagueness here is telling.

The statement mentions "hands on warning" which just means you need to keep your hands on the wheel. That's very different than giving control back to the driver.

Thank you.. you are correct. That’s an important detail I wrote incorrectly.

> human driver who was paying attention

true. here’s the location (post #62)

https://teslamotorsclub.com/tmc/threads/model-x-crash-on-us-...

but the average person in the US anyway is not a good driver. i very often see people get aurprised at the lane ending at that exact spot. they should put a rumble strip leading up to it.

So, my question is, is it fair to compare AP to a human driver paying attention or is it more fair to compare it to an average driver.

I mean that just for the sake of argument. In this specific case, NTSB or someone should ban AP. It is so obvious what’s about to happen there, an AP should do what a driver should do which is take the ramp whether it’s a wrong turn or not. So many a-holes try to squeeze in last second (oops) when they should just take the damn exit or miss the damn exit as the case may be. What AP did here is what a poor and panicked driver would do and that’s just not acceptable.

Did they release information about the path the car took? It could just as well have simply gone down the median as if it were a lane of traffic.

But this wasn't a self driving vehicle. It isn't capable of dealing with all situations, and it warning the driver for 6 seconds to take control, which didn't happen.

No, it didn't warn at all. It warned prior to this, which was just put in the text to trick people into thinking it warned.

> That's not a mistake a healthy human will make.

Humans do exceedingly stupid things all the time because they stop paying attention, even momentarily (or subconsciously).

We put big lights on the back of cars that light up when they brake. And yet, despite a driver looking direct at a huge object with two lights, that rapidly grows larger right in front of them, does not always prevent a collision. Or even a chain of collisions. I don't get on the road much, and yet even I've seen a ton of accidents that are baffling and can only have been the result of a driver not paying attention for a bit.

I'm reminded of how in quite a few places removing signs and lights actually improved safety because it forced drivers to stay aware instead of 'driving on autopilot', so to speak.

I think that's the real issue here. The more we outsource our attention to a machine, the more important it is that said machine does MUCH better than we humans do. Especially if a mistake can be deadly.

But I wouldn't be surprised if, indeed, technically this accident could've happened just as easily by a non-autopilot car where the driver had a little 'micro-sleep', got distracted by something in his field of view, mistakenly thought he was on a lane and didn't notice the (let's be fair) ridiculously bad markers that were the only way to tell that part of the 'gray' stuff in front of him was in fact a wall of concrete.

I mean, just look at the image: https://imgur.com/a/iMY1x . Half of what makes the barrier stand out is the shadow!

All that said I might sound more argumentative than I am. I do agree with most of your comment.

I’m not sure I agree. Humans run into highly visible barriers all the time.

«the autopilot did something a human driver who was paying attention would never do»

I'm sure that out of the 1.25 million annual automobile-related deaths, plenty of drivers were paying attention and still did stupid things similar to this accident.

Using global statistics greatly skews your argument. While all these deaths are tragic, many of them could be prevented through law and regulation of human drivers.

"the autopilot did something a human driver who was paying attention would never do"

"this isn't an average accident statistic, a lapse in attention or evasive maneuvering. "

"That's not a mistake a healthy human will make."

Almost every driver thinks they're significantly better than average. Few are.

If a human driver's lapse of attention causes a similar crash, how much less of a tragedy is it just because we can less ambiguously blame the victim?

My opinion is that the statistics compare just fine.

> Almost every driver thinks they’re significantly better than average. Few are.

Actually, about half of drivers are better than average.

It's easy to provide a counterpoint if you ignore words (significantly). Few people are significantly above average.

https://en.wikipedia.org/wiki/Median

Hence “about” and not exactly half, unless you can prove some weird distribution for drivers’ abilities. Go on, I’ll wait.

>That's not a mistake a healthy human will make.

This statement is meaningless circular logic. You can handwave away any incident with a human driver by saying he wasn't "healthy".

A human pilot has intentionally driven an airplane full of passengers into the ground with full control because he wanted to kill himself. The airline believed he was "healthy".

I want electric cars to succeed as I am am long bored of breathing in vehicle fumes. Automated driving is cool sci-fi tech, but holds nowhere near the same sense of necessity, as far as I am concerned.

So, I thought it wasn't all that clever in the first place to try and marry the risks of getting electric cars into the market with the risks of telling the extremely wealthy that they didn't need to hold the damn steering wheel.

In keeping with the "Autopilot" terminology, this was "Controlled Flight info Terrain".

Tesla demonstrates their usual abuse of statistics. 1.25 fatalities per 100 million miles is the average across all types of terrain, conditions, and cars.

The death rate in rural roads is much higher than in urban areas. The death rate on highways is much lower than average. The death rate with new cars is much lower than average.

The autopilot will not engage in the most dangerous conditions. This alone will improve the fatality rate, even if the driver does all the work.

Tesla cars are modern luxury cars. They appear to have done a great job building a well constructed, safe, car. This does not mean their autopilot is not dangerous.

for varying values of dangerous.

I completely agree; Tesla’s marketing modus operandi is to fudge statistics. In my view, they are consistently on the wrong side of the marketing puffery | misrepresentation borderline.

I do take issue, however, with your claim that Teslas are “well constructed” - they are not. They are poorly designed, poorly constructed vehicles that feel and look cheap. Test one side by side with, say, a BMW 3-Series or a Volkswagen Golf and the difference in production quality will be palpably obvious.

The author was talking about road safety from the cars rigorous construction. There seems to be little doubt about that non-subjective issue.

I saw the argument that a traditional motor’s noise level hides rattling sounds and similar subtle car defects, which a fully-electric vehicle’s silence does not cover.

(I haven’t seen scientific comparisons, though.)

Traditionally at least, the only thing one would hear inside a Mercedes Benz going at 60mph would be the tick of the clock.

Agreed. I don't know why they think they get away with it. Is there any independent agency providing numbers about Tesla claims?

> This does not mean their autopilot is not dangerous.

It also doesn't mean that it is dangerous. That bag of doritos that you ate, it might also be dangerous. Let's use facts and not vague worrisome complaints. So if you want to talk about the higher danger of rural roads, please give a number, and then give a number for tesla, or estimate one. Don't just say "urgh".

I mean, if Tesla is comparing their luxury car's death rates to motorcycles, which have 10x-50x the fatality rate of passenger cars, then, yes, it's dangerous.

Remember, there are several very normal cars models that have had ZERO deaths: https://www.nbcnews.com/business/autos/record-9-models-have-...

These autopilot cars are death traps.

I wonder if it’s possible to determine whether those cars have zero death rates in part because someone who would buy that model is less likely to get in a fatal accident in the first place—whether because of who they are, or how they were advertised, or whatever. I imagine a minivan owner as a very cautious driver…then again, I have been in the passenger seat when a friend was doing drifts in his mom’s minivan, so your mileage may vary.

I've had 2 of those cars - the Sorento and the Odyssey, and they both feel really tactile/agile to drive, as well as being chill, relaxing rides.

Yes, one might quip that males under the age of 25 wouldn't be seen dead in any of those vehicles.

Who doesn’t wanna sit in an Audi A4 Quattro?

I'd say the report is misleading.

Fatalities are rare events. They chose cars with 100,000 registered vehicles or more. The categories these vehicles belong to experienced around 30 deaths per million cars, so cars that have 100,000 registrations would expect 3 deaths. You could easily get a few models with 0 deaths just as a matter of chance.

I don't think these results necessarily show these cars are inherently safer than other cars in the same category.

Outside of the Tesla marketing department, conditional probability has been known for 300 years. Comparing the average fatality rate to the fatality rate of a modern car with Autopilot is an egregious abuse of statistics. How can anyone defend this practice ?!

Just as an example, click on California (it is mostly consistent all over) on the map [1]. California has a lower average fatality rate of 1.01 per 100 million miles.

Rural roads have 2.62 fatalities per 100 million miles. Urban roads have 0.70 fatalities per 100 million miles.

The fatality rate is much lower on highways, according to Wikipedia [2], freeways have 3.38 fatalities per billion km (0.54 per 100 million mile, if I managed the conversion).

[1] https://cdan.nhtsa.gov/stsi.htm

[2] https://en.wikipedia.org/wiki/Transportation_safety_in_the_U...

What percent of all driving is done on rural roads?

Note: Tesla is comparing the fatality rate of autopilot-equipped cars with the average accident rate. They are NOT only counting miles where autopilot is used, they are counting all miles driven by their cars. It could be that Tesla drivers are inherently safer than the general population, or that the cars themselves are much safer. What they are not doing (at least with the data) is making a statistical claim about autopilot safety.

Exactly. Tesla drivers will be older, more affluent, and I bet, generally safer drivers. I'm sure an insurance company could easily debunk those statistics.

That's a very good point. But it raises a new question, why didn't Tesla compare the safety record of Teslas with Autopilot against Teslas without Autopilot.

I don’t know. It could be because the comparison isn’t favorable, or it could be because it’s very hard (as your parent post states) to generate a valid comparison that controls for driver age and skill, driving conditions, and vehicle safety.

What I do know is that Tesla have previously published a comparison of the safety of their cars before and after the autopilot feature was made available, and there is a statistically significant improvement of nearly 60%. This study should factor out most of those caveats, since it’s the same cars, same drivers and same roads before and after the feature release. http://bgr.com/2017/01/19/tesla-autopilot-crash-safety-stati...

They have done so. As mentioned in their previous blog post, Tesla's crash rate was reduced by 40% after introduction of Autopilot based on data reviewed by NHTSA.

You're saying autopilot refuses to run on rural roads?

It is only dangerous, if it is worse than the average driver.

No, it is still killing people, but average people are cuppable. Big difference.

You realize killing people less than average is not dangerous. It is actually safer. Without understanding this basic logic, than I am not sure what else to say.

> "Controlled Flight Into Terrain"

Thanks, I never knew this term before. Seems like a dark humour gem - equally funny and terrifying.

"The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken."

This seems to be trying to suggest that the driver had 6 seconds of clear driving-toward-death time to correct for the car's actions without explicitly making such a ridiculous statement (while also throwing in the crushed crash attenuator). If the car makes a quick change in direction due to an autopilot error, a driver at speed would have very little time to make an effective correction.

Depending on the system behavior, it could be akin to having a passenger reach over and yank the wheel. I'd honestly rather Tesla just said "ongoing investigation" instead of being so transparently evasive.

They also said in their first post they would respect the family by not updating until the investigation was over. The investigation isn't over but here they are trying to manage their image with confusing and misleading partial information.

Yup this also mean that Tesla's radar and camera systems had 5 seconds to realize it was driving straight into an unmoving concrete barrier and did nothing...

What's really odd is, in my Subaru which has eyesight, if I drove straight towards that barrier the breaks would engage. Eyesight uses two visual cameras, about two feet apart, and uses the parallax between them to judge depth/distance (in the same way human eyesight judges depth).

So even if Tesla's autopilot steered the vehicle towards that divider, shouldn't the auto-braking system have engaged to avert the accident? Tesla vehicles also have visual image cameras as well as radar based ones, so the information density is even higher than Subaru's system.

I guess what I am getting at is: Is auto-braking disabled while autopilot is on? Wouldn't leaving auto-braking enabled (particularly using the visual cameras) offer a second layer of safety is autopilot made an error?

I raised the same issue when a Tesla with autopilot on drove straight into the side of a truck and the driver was decapitated. The discussion was all about "well radar couldn't distinguish it from road signs!" while ignoring that a Tesla has visual (optical) cameras front and center.

I've wondered about those Subaru's that seem to have two build in dash cams right at the top of the windshield.

In the case of the truck, it was established that neither the driver nor the cameras would have identified the side of the trailer against the sky, and it's still ultimately the driver's responsibility to be alert and avoid such crashes. See the report here: https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF

From the report “NHTSA’s crash reconstruction indicates that the tractor trailer should have been visible to the Tesla driver for at least seven seconds prior to impact.”

Unless someone is operating trucks with adaptive optical camouflage, I think that the driver would be expected to note the presence of a big rig. Binocular cameras and radar may be expected to do the same, too.

(Note: Tesla models, to the best of my knowledge, do not have binocular cameras)

I have a Subaru with eyesight too.

The auto-braking in Subarus with eyesight can engage in two circumstances that I'm aware of.

1: The car is travelling at 50 km/hr or less, and perceives an oncoming obstacle.

2: The car is using adaptive cruise control, and perceives that it is getting too close to the vehicle ahead.

I think there are some circumstances where the designers decided that auto-braking would actually be worse. For instance, at highway speed, it's usually not possible to come to a stop quickly enough to avoid a stationary object that has somehow suddenly come into your path. Say, a deer. It's best to take your feet off both pedals and swerve around the obstacle. This reduces speed while also allowing maneuvering, something that braking works against. It also gives drivers behind you more time to react.

If the Tesla system works similarly I would expect similar design decisions. However, I would expect autopilot on a Tesla to function similarly to adaptive cruise control, which will jam on the brakes when it thinks it's going to hit something. This has actually engaged erroneously on two occasions for me, on windy roads, so I only use adaptive cruise control in straight-arrow highway situations now.

I've noticed that in certain weather conditions the cameras will stop being able to get good enough data to make decisions. The car disables the system and notifies me with a warning indicator.

tl;dr - auto-breaking based on parallax cameras can't be deployed in every situation, and sometimes the cameras can't get a clear shot.

> In the moments before the collision, which occurred at 9:27 a.m. on Friday, March 23rd, Autopilot was engaged with the adaptive cruise control follow-distance set to minimum.

Would've loved to know the exact moments we are talking about here. Is it 5 seconds or 5 minutes?

> The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision.

What does this mean? Did Tesla want to give over control to driver? Or just normal no hands?

> The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken.

Again its hard to know if Tesla wanted to give up control or not. I hope they aren't just saying that because the driver's hands were off the steering wheel, the crash occurred and is his fault for not obeying the agreement.

> I hope they aren't just saying that because the driver's hands were off the steering wheel, the crash occurred and is his fault for not obeying the agreement.

That's exactly Tesla's official stance - if you crash you're SOL and at fault, Autopilot or Auto Emergency Braking be damned.

https://www.cnbc.com/2018/01/31/apples-steve-wozniak-doesnt-... "Man you have got to be ready — it makes mistakes, it loses track of the lane lines. You have to be on your toes all the time," says Wozniak. "All Tesla did is say, 'It is beta so we are not responsible. It doesn't necessarily work, so you have to be in control.'

"Well you that is kinda a cheap way out of it."

If you ignore it long enough, the autopilot will disengage entirely. The car will drive in a straight line and coast to a stop.

The car will drive in a straight line and coast to a stop.

In this case, it would be more like "drive in a straight line and... collide to a stop."

Perhaps braking to a stop at a reasonable rate would be the right thing to do, given that it surely should've detected it was going to hit something?

But according to Tesla, the autopilot was engaged. The car's behavior when disengaged is not the question here.

As a Tesla owner, this is not accurate wording. The car will very quickly decelerate until about 5 mph if you have the crawl feature on.

I need to make a post, as a long time Tesla owner/very frequent autopilot user, because I think I know exactly what happened. Unfortunately I need to go to bed, already up later that I should be for tomorrow. A lot of information and comments I’ve seen about how it works and how autopilot is dangerous etc, are quite wrong. I think it’d be important for people to understand how it works from people who use it and get their opinions first before making statements about AP in general.

For the record, I love using AP and will continue to do so. But it is no silver bullet, and 99.5% of Tesla owners know this and how to correctly use it.

Hey Tesla, traffic and vehicle safety advocates do not call crashes “accidents”.

http://www.roadpeace.org/take-action/crash-not-accident/

https://www.crashnotaccident.com

Yeah, when driving an emergency vehicle, in my case an ambulance and fire engine, you take a course that used to be called Emergency Vehicle Accident Prevention.

It's now called Emergency Vehicle -Incident- Prevention.

> In the US, there is one automotive fatality every 86 million miles across all vehicles from all manufacturers. For Tesla, there is one fatality, including known pedestrian fatalities, every 320 million miles in vehicles equipped with Autopilot hardware. If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident.

"Equipped with Autopilot" is different than running with Autopilot. As far as I know, all model S fatal accidents except one was with Autopilot engaged. Making Autopilot far more dangerous than manual driving from a statistical standpoint.

They all had windshield wipers as well. Maybe that is the cause?

very deceptively crafted sentence. if i drove on AP for 15 minutes and i was alerted to put my hands on the wheel 2 minutes in.

It would fit tesla's specially crafted statement.

It makes it look like user was warned and didn't respond by putting their hands on the wheel but its completely false. The AP drove him straight into the barrier with no warning.

https://www.tesla.com/blog/update-last-week%E2%80%99s-accide...

>very deceptively crafted sentence. >The AP drove him straight into the barrier with no warning.

Exactly - anyone who owns autopilot knows this.

It seems gross that they spend so much of this saying "yeah he died and it may be our fault, but statistically you're still better off as our customer."

The atlantic had a relevant piece on the morality there this morning https://www.theatlantic.com/technology/archive/2018/03/got-9...

And that's assuming they can identify that autopilot is statistically safer. If autopilot isn't used in the same conditions as all human driving, the rates aren't apples to apples comparisons, and the "you are 3.7 times less likely to be involved in a fatal accident" figure they imply is misleading.

If that's literally the best case scenario, then that's pretty damn pitiful and a gross misleading of facts using statistics. It's a shame. Let's use "AP disengage" as the equating metric to human-caused crashes and see how it stacks up.

They didn't even control for wealth. People that can afford a Tesla and the auto pilot add on are much less likely to be driving on drugs, alcohol, etc. It doesn't say anything about the statistics when auto-pilot is engaged, it just says cars with auto pilot. And it doesn't give the statistics for Teslas without auto-pilot (Teslas have other safety factors that might mean auto-pilot isn't responsible for the mortality reduction, etc.)

People that can afford a Tesla and the auto pilot add on are much less likely to be driving on drugs, alcohol, etc.

Is that true? I couldn't find statistics that grouped by income or wealth.

Also look at occupation. Tesla owners = disproportionately software engineers. Software engineers are one of the lowest car insurance risk pools (and we're even before any of them started using auto-pilot).

Here is the confusing sentence: (dots added)

"The driver had received several visual and one audible hands-on warning earlier in the drive ... and the driver’s hands were not detected on the wheel for six seconds prior to the collision."

https://www.tesla.com/blog/update-last-week’s-accident

It's not clear to me whether the "several visual and one audible hands-on" happened right before the crash (6-10 seconds), or the minutes before.

If it was close to the crash they would have straight up just said that. They're just trying to muddy the facts.

It could have been earlier in the drive as in 30 minutes before, or something ridiculous like that. That was complete weasel wording.

Also, Tesla can mistakenly think you aren't holding the wheel when you are. Happens to me all the time.

Regardless of the criticism of Tesla here, surely the driver wasn't watching the road, or otherwise this wouldn't have happened.

"The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision."

Shouldn't you have more audible warnings than visual if you are aware that not complying will lead to an accident?

This feels particularly evasive. If you want to claim that your technology is 10x safer, you should be able to back that up with self-insurance, and not try to put the blame on the driver.

If the autopilot does not think it is capable of driving safely, it should pull over to the shoulder of the road, not beep and flash.

It doesn't seem like pulling to the right would be safe in this situation. I think that automatically slowing down would be, though. Even slowing at a rate similar to a human letting their foot off the gas (to avoid unexpected behavior) would reduce the chance of fatality in a crash.

People pull over to the shoulder safely everyday. If an autopilot car cannot do this, it should not be on the road.

People do not pull over to the shoulder when they are between lanes in (presumably) busy traffic, which is where you have to be to hit a median.

I am not sure if Tesla's current system, which is still just cruise control with steering, should ever be changing lanes unprompted. I don't have a Tesla but I would not want my car's cruise control to do that right now. I do think that pulling over in safe conditions is something that a self-driving system needs to be able to do at some point on the path to full autonomy.

In this situation I think the lane to the right is not the shoulder. So in a situation where there are only seconds to respond to an obstacle, there might be cars blocking the Tesla on the right.

No one shipping a Level 2 system provides insurance for it. The definition of Level 2 is that the driver has responsibility.

That's very troubling. If you saw the Uber crash cam on the driver it's obvious that the "responsible drivers" are completely non-responsive, on their phones, half asleep. A system that lulls the driver into a false sense of security does NOT pass responsibility to said driver.

Uber is testing a Level 4-5 system, those systems will need to be insured since they will not have a driver in the car.

Speaking of insurance, but not self-insurance:

I'm guessing that if I buy a Tesla with autopilot, I'm not going to find any insurers who will charge me discounted rates that reflect autopilot being 10x safer.

This is a blatant excuse - you can gently hold the wheels ready to take over without applying pressure and their system will not detect it you don't firmly hold the wheels, which defeats the comfort of having bought "Autopilot" in the first place.

Actually the warning had nothing to do with the crash. if i drove on AP for 15 minutes and i was alerted to put my hands on the wheel 2 minutes in. It would fit tesla's specially crafted statement.

making it look like user was warned and didn't respond by putting their hands on the wheel but like i previously said, its completely false.

Letting auto pilot get to a point where it does the audible warning means the driver wasn’t paying attention.

There have been too many times when driving that I've needed to visually make eye contact with either another driver or pedestrian in order to decide how to proceed. Can an autopilot differentiate between a 6 year old chasing a ball into the street unaware and an 8 year old chasing the same ball but also sees the car coming? Or someone patiently waiting to cross the street while another impatiently sees they are missing their bus and dart across the road anyway? I see it in their face and body language and respond differently. Can autopilot do that? Until it can I don't trust it. At best it should be restricted to roads without pedestrians like freeways.

In several places, this statement is missing some important information, which leads to the impression that the story is being spun to imply the driver had more responsibility in this crash than may have been the case.

The statement says "The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision. The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken." (my emphasis.)

What is not stated here is how long before the impact was it clear that the car was off course and was not going to make a correction - i.e. how much time would an attentive driver have to realize that something was going wrong? Tesla may not yet know the answer to that question, but if it does not, I do not think it should suggest that the driver had five seconds to respond (the only information I have read on this matter is Tesla's own statements, so I do not know if other reporting on the crash shows that this was the case.)

Hands off the wheel does not prove that the driver is inattentive. More importantly, hands on the wheel does not show that the driver is attentive. IMHO, something at least as effective as eye tracking is necessary to establish that. Furthermore, Tesla's own statement indicates that it is allowing too long a period of inattentiveness, even for highway driving.

It is reasonable for Tesla to use statistics to justify the utility of their systems, so long as they compare like with like. Unfortunately, this statement is cagey about that: "If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident." Compared to what? Teslas without such hardware? Regardless of whether the hardware is being used? depending on what they are comparing it to, these statistics might be on account of simpler, more reliable safety features of the vehicle, or even owner demographics.

Finally, the absence of a working crash attenuator, while a tragic fact, has no bearing on the issue of Autopilot's reliability.

I find it terrifying how auto pilot cars give the driver a false sense of security. When a car has statistically proven claim that it is safe, drivers are more likely to have less focus on the car. It’s like comparing reaction times of a manually driving individual to that of a passenger. What seems to be 6 seconds may in reality feel like a split second due to the lack of context. I want to say that there are overhead costs to getting back into driving mode when you haven’t been fully aware of your surroundings. This happens with a much lesser degree when I’m using cruise control, but I still avoid it when there is a fair number of cars on the road.

If I have to be well aware of the warnings that the system puts out, I might as well stick to no auto pilot. The chances are low, but any chance is enough to keep me from using it. the roads are designed for humans after all.

“For Tesla, there is one fatality, including known pedestrian fatalities, every 320 million miles in vehicles equipped with Autopilot hardware. If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident.”

Their stat is for miles in vehicles equipped with autopilot, not miles driven by autopilot. So if their claim was whether a Tesla equipped with autopilot is safer than a standard car, this would be valid.

The comparison should be with miles driven with autopilot engaged. They must know this number because Tesla tracks everything.

It’s unethical of them to publish such misleading stats.

Cynically, I also think that the fatality rate of autopilot miles is either really bad or just not comparable due to too few miles. Either way, they should reveal this.

True, they know this number, it is public and has been ratified by transit authorities; see pages 10-11: https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF

Why do they use weasel words and misleading statistics when they have this? A crash rate of 1.3 per million miles before installation and 0.8 per million miles after sounds like a slam dunk for the autopilot's statistical safety.

The biggest failure here is Tesla's marketing department. Calling it autopilot implies full level 5 autonomous driving in the average person's mind. Very few will read the fine print.

In reality, the current state of it is level 2 or so -- like an enhanced cruise control. You absolutely can't take your eyes off the road still which is what apparently happened here.

I can’t help but feel that Tesla, Uber and others are being crazy reckless with their self driving technology, maybe even criminally so. Google have put massively more effort into their self-driving tech, AFAIK massively more engineering and QA hours (by probably better engineers and ML people), and they still don’t consider their tech ready for sale to the general public. Tesla seem to be shipping an MVP, which is insane for safety critical software.

Tesla claim “Tesla’s equipped with autopilot have a 3.7x lower fatality rate compared to the average car,” but that’s a very weak claim. It doesn’t say anything about the safety of the autopilot tech itself, as it’s lumping in “normal driver” stats. Also, do new, expensive cars simply have lower than average fatality rates in general?

I think self-driving cars will be great for society, but we need to be super cautious about shipping them. Google seem to be very cautious, Tesla seem to be very reckless. If it’s found they shipped autopilot without strong evidence that it’s much safer than human drivers, and it’s out there killing people, IMO Tesla execs should go to jail.

I have a lot of respect for Tesla, but this incident convinced me not to buy one any time soon. I have children with car seats. The fires that break out with battery right under the middle makes it impossible to save someone in the driver seat, let alone in the back.

More importantly, self driving cars are not the same - Waymo has lidar, Tesla and Uber don’t (correct me if wrong). If a system is clearly struggling with the detection of still objects in the middle of a highway, on a sunny day, it cannot be trusted. If I ever bought a Tesla, I wouldn’t use autopilot unless their hardware proves reliable.

I thought fire started because rescue team cut someone out of the car.

If you look at the pictures, I don't think much cutting was needed, and certainly any that might be would be nowhere near the batteries located beneath the car; the front end basically disintegrated completely from the impact.

The fire started way after the crash.

Not quite. The battery pack actually has an aerogel like insulating layer above it in between the cabin and the pack to allow people time to exit, and ports the flames to the front and rear of the vehicle, rather than the sides for this exact reason. It will obviously eventually consume the whole vehicle, because lithium fires burn so hot, but they've done a lot to ameliorate this concern. I thought the driver in this case was removed _before_ the fire started.

I think you are mis-analyzing. Tesla fires are extremely rare, more so than gasoline fires. And the fires take much longer to occur.

The Uber car had lidar, although only one, whereas Waymo's have six: https://www.reuters.com/article/us-uber-selfdriving-sensors-...

See also this interview. The driver said he had problems with auto pilot in same location. http://abc7news.com/automotive/exclusive-autopilot-part-of-t...

Wow, that's a disgusting amount of spin and statistical misuse.

I had intended to buy TSLA after their next awful earnings call and hold it long, now I'm not sure I want anything to do with this business.

I have two quite different takes on this and the recent Uber crash

1. Simply stop using automated driving in real world until we have clear golden standards (something like billions of hours of simulated driving against real world data collected by other systems makers sensors. The accidents so far have been characterised by "that should not have been possible, not that was really hard edge case"

2. Accident reports make horrific reading (and upsetting for families especially). But like reports in day the FAA, there are benefits - openly sharing crash data makes (ok should make - see 1.) the next round safer. secondly just reading these things makes the humans involved more ... aware.

Some time ago i visited Ireland and was amazed to hear the local news announcing recent car deaths - it was a mandated thing, to try and raise awareness and reduce future accidents - it apperntly worked.

The other factor is:

At 9:30 am, the left lane (lane #2, that stays on 101) is usually packed. He may not have been able to merge in from the #1 lane (HOV-to-Hwy85-flyover lane) into that left lane. Especially if there was a big speed differential.

Although 150 meters is plenty to stop in the median.

(Do we know which lane he was in before hitting the divider?)

This is incomplete and therefore, very misleading. One has to compare what's comparable: what is the sample size for each? That is, how many miles are driven by Tesla vehicles per year, versus how many miles total for all US vehicles on US roads?

It seems, Tesla puts up around 3.2 billion miles/year, vs 3.2 trillion miles for all US vehicles; while the numbers are both big, that's a 1000x times difference, or 3 orders of magnitude.

Tesla numbers seem to imply there are around 10 people killed per year using a Tesla or being hit by a Tesla (because 3.2 billion / 320 million = 10). So, if a Tesla with 5 people on board crashes into a wall this year, and kills everyone inside, that will represent a 50% increase in the fatality rate.

And the phrase "3.7 times less likely" is also very misleading. People who drive Teslas usually wouldn't be driving dangerous vehicles (such as small, 4-door cars which are the deadliest), if they didn't have a Tesla. They would probably drive a German car (BMW, Audi, Mercedes) which are the safest cars in the US today.

> There are about 1.25 million automotive deaths worldwide. If the current safety level of a Tesla vehicle were to be applied, it would mean about 900,000 lives saved per year.

This is another level of nonsense entirely. "Worldwide"? A lot of things happen worldwide; in some countries there are more scooters than cars; in others, there are many very unsafe and poorly maintained country/dirt roads. The Tesla numbers are shady enough when limited to the US, but expanding them for a worldwide comparison is ludicrous.

I notice they don't mention that the driver complained that the autopilot had trouble with this specific spot on the highway a few times before. https://news.ycombinator.com/item?id=16719736

Late edit: According to https://arstechnica.com/cars/2018/03/tesla-says-autopilot-wa... Tesla has no record of his complaints.

I don't care about Tesla at all, not a fan boy or anything, but I don't understand all the blaming on Tesla for this accident. So, the guy had complained 'between 7 and 10 times' about Autopilot malfunctioning there, yet he kept on using it Autopilot disengaged before the crash, giving visual and audible alerts before turning off, yet he didn't take the wheel, so he probably was doing something else instead of keeping attention in a zone where he said the thing failed... How many times do they have to say that even if you're using AP you need to stay focused on the road? I'm sorry for the guy, but if you don't trust a system that's as critical as this you either don't use it or you're very cautious when using it.

The article doesn't say when the visual and audible alerts happened, they could have been minutes previous on another stretch of road.

I agree he should have disabled Autopilot there if he knew it was an issue, however why didn't Tesla recommend that to him when he made the report? And the removal of the concrete barrier may have been a new factor that he didn't expect and caused Autopilot to fail much more severely.

Ha, you just fell for Tesla PR. Maybe you should read their statement again. It is gullible people like you that they create these misleading statements.

No need for ad hominem attacks. You make no rebuttals to the points made by the comment above and just dismiss it as PR. Just because something is (or isn't) PR doesn't mean that it's false.

I love people on this site. It is literally the only site where you respond to a comment and people start listing fallacies. My answer stands. Numerous people in other comments have explained how Tesla carefully crafted the statement so that gullible people would think it said something different and this guy fell for it. Better start listing some more fallacies in this response! Here is a list if you need help https://en.wikipedia.org/wiki/List_of_fallacies

Not at all, and I assume they will be hiding as much as they can to avoid as many problems as they can, but that's not my point. If you have a problem with something, you don't keep playing with it when you know it's going to fail. My Philips TV turns on by itself at night. Instead of waking up at 3am to shut it down I power it down hard before going to bed instead of putting it on standby. The same way, if you know you have a problem with something that can kill you in some road you just don't keep using it "in case it does it right this time"... ...and in any case if you dont trust the system, you dont keep your hands off the steering wheel... (unless the guy had a heart attack or something just before the accident and that was why he didn't do anything, we're all assuming he was capable of driving but he might have lost conciussness or something , then autopilot disengaged, then the crash happened, but I don't know, and everything we can do is speculate...

Personal attacks are not ok here and we ban accounts that do that. Please read https://news.ycombinator.com/newsguidelines.html and post civilly and substantively, or not at all. Would be good to read https://news.ycombinator.com/newswelcome.html too.

The thing I don't understand is the driver has complained about AP not working on this same location multiple times in the past yet still kept the hands off the wheels for more than 5 seconds?

We don't know that is what happened. Tesla is using weasel wording to confuse.

If the driver had "five seconds and 150 meters of unobstructed view of the concrete divider", then didn't the car, too?

I'm not disputing their statistics, but reading past the first two paragraphs of this article left a sour taste in my mouth. Feel like a better message ought to have been "we're going to learn from this, and won't stop improving until the crash rate is 0".

You should dispute the use of statitics here. They make no attempt to control for various factors and compare apples to apples.

I just wouldn’t buy a Tesla at this point because: 1) the autopilot is not reliable. The accident shows clearly that it can dive straight into wrong path without giving warning. This is unacceptable, however small the probability. 2) the battery explosion is making the accident ever so dangerous. 3) Tesla’s response is disgusting. It may be true that statistically or on the average it is safer, but I am not a number. If I drive responsibly I can beat the statistic. Plus, relying too much on the comfort of statistics may be attributing to the lack of human intervention. 4) price is also a factor but the least important one compared to the above three.

My ONLY reason I don't already own one (before recently) is their draconian anti-repair strategy, remote bricking, and other heavyhanded tactics against 'owners'. Apparently you are just paying for the privlige of borrowing one from tesla.

I hate how Tesla framed this as a statistics game.

It's like an amazing heart doctor one day suddenly stabbing his patient in the heart with a scalpel, killing him, and then saying... well, sometimes I malfunction, but by and large I cure patients 10x more often than my colleagues.

That isn't okay and it's important to talk about that strange behaviour to fix it.

Sure it’s strange behavior, but I’d still choose him over his colleagues.

Doesn't look like very aberrant behavior by AP in this instance, this exit ramp is a very misleading one which looks exactly like a normal lane up until it dead ends in a barrier.

Their statement blames:

-The driver

-The crushed barrier

-Statistics

But they do not take responsibility themselves. It's apparent and in my opinion shameful.

If Tesla's goal of this is to just share the facts, why didn't they state what AutoPilot did? In addition to statements like "...the driver’s hands were not detected on the wheel for six seconds prior to the collision.", wouldn't it make sense to have a statement like "AutoPilot seems to have made a grave error here, and steered the car into the barrier."

They never admit that their car didn't perform properly. This reads like a carefully crafted, legal approved statement - not the more human statements that we see many times after tragedies. This absolutely lowers my opinion of Tesla by a decent amount.

"We feel very bad about what happened... and want to take responsibility and do what's right" - A quote from the Walmart CEO after a Walmart truck accident injured Tracy Morgan and killed another man. I wish Tesla could stand up and say the same in this case (if the evidence points that way, which it appears to).

https://www.cbsnews.com/video/walmart-ceo-on-tracy-morgan-ca...

It was a median in the middle of the highway. The car probably continued in a straight line instead of "steering into" a barrier. https://imgur.com/a/iMY1x

Why the car was not able to detect and avoid the collision in such a simple situation (from a human POV)? We have seen Tesla's autopilot performing very well in a much more complex scenario than this. Ref:- https://www.youtube.com/watch?v=FadR7ETT_1k

Stationary obstacles are a harder problem, because there are so many stationary objects and features around a road that have to be ignored.

This autopilot incident is similar to the one with the tractor-trailer[1] in that regard.

[1] https://www.theguardian.com/technology/2016/jun/30/tesla-aut...

Or the one in China with the street sweeper: https://www.youtube.com/watch?v=fc0yYJ8-Dyo

Or the one with the fire truck: https://www.theverge.com/2018/1/23/16923800/tesla-firetruck-...

No it's autopilot steered into the median lane in the first place where it diverges from the HOV lane.

Note: Both left lanes are HOV lanes. #1 leads onto Hwy 85, #2 onto Hwy 101. Do they say which lane he was in before entering the median?

Just curious why autopilot would be in lane #1 if he intended to stay on Hwy 101. Or vice versa. Maybe it couldn't make a safe lane change sooner? (Pure speculation)

Fair. Could be rewritten as "AutoPilot seems to have made a grave error here, and drove the car into the barrier. AutoPilot should have either avoided the barrier or disengaged."

Not excusing tesla but how on earth can there be a piece of flat concrete instead of a v shaped bendable deflector ? Was there one but had to be replaced ?

There is a crushable barrier there. It was broken that day, because someone had recently crashed into it. Drove past there yesterday, it has been replaced.

I see, sad timing u_u

Indeed.

But maybe this is just a variant on https://www.wired.com/story/tesla-autopilot-why-crash-radar/ That is, maybe Mr. Huang was following another vehicle, which merged out of the way, and his Tesla ignored the stationary object.

If that's so, Tesla ought to emphasize the failure mode.

It is pre-mature to say this. Why would they say "AutoPilot seems to have made a grave error here, and steered the car into the barrier." if it is possibly not their fault ?

If indeed it was auto-pilot's fault, they will likely put the statement out later.

To be fair, a famous person wasn't involved in this crash.

What you are asking them to do is legal suicide. If they were to claim this was their fault, they would be immediately slapped with multi-million dollar lawsuits from everybody who ever had a problem with auto pilot. It would end the company.

Really? I don't think so. It's not like a "confession".

Can't people sue them even without their admission of any responsibility whatsoever? I think so.

And that they simply say: "We are sorry, we take full responsibility." does not open them up to lawsuits -- the courts still have to hear the arguments to back up "actual legal liability" and then adjudicate / elaborate a verdict / judgement, right?

Comments from lawyers, please. Very interesting.

I think there the possibility to find a standard for response that genuinely serves the maximum future good.

In my mind — the standard response to such accidents should be to setup an endowment dedicated to the study of techniques promoting the enhancement of autonomous safety, named after the victim, in the geographical region of the accident. Build a damn research facility near the accident site — spread the valuable high tech knowledge across generations and demographics — along with the understanding that the ultimate mission being pursued is the safest autonomous systems possible.

This kind of response would go over really well I think and have a lot of benefits beyond just PR. There are very real possibilities that accident rates will not be uniform across driving regions due to the inability of models to optimize for the uneven distribution of risk sources across all driving scenarios. This gets _really evil_ if profit motive is allowed to be be the only mechanism driving the development of vehicle safety models. I cringe for a future where car accidents are biased to occur more often in poor or ethnic neighborhoods ...

These kinds of unequivocal statements are just begging for a lawsuit, imo. They simply don't know that this is true.

Which part(s) are you referring to? There seem to be a few claims in there that could be provably false.

Exactly. They are taking both sides of the an argument to get credit but not take blame. They want credit for allegedly reducing crashes, but they don't want to be held responsible for driving into a wall.

Even if it was us... it probabbly wasn't!

Yes! That's what I said, when I forgot to say that.

Are you saying that the two paragraphs before this quote are inaccurate?

Consider this graf:

Let's put aside the consideration of how the average Tesla driver differs compared to the average American driver, or how the average mile driven by Autopilot differs compared to the average mile driven by all human drivers; I'm a little bit bothered by the language here:

> If you are driving a Tesla equipped with Autopilot hardware...

> every 320 million miles in vehicles equipped with Autopilot hardware...

Why is the language "equipped with Autopilot hardware" used? Tesla isn't able to give an estimation of when Autopilot was equipped and engaged?

That's an interesting question. Similarly, one could ask why the focus on fatalities rather than crashes as Tesla's superior traditional safety rating means fewer crashes should be fatal, all else equal.

However, these questions don't lead anywhere interesting. Model S and X cars were driven before and after Autopilot was activated (no hardware change), and the crash rate was reduced after activation. This is the "40% reduction" mentioned by Tesla in the article you pulled the quotes from. You can read a summary here: https://www.theverge.com/2017/1/19/14326258/teslas-crash-rat...

This 40% has nothing to do with Autopilot and applies to any cars with auto braking.

The Insurance Institute for Highway Safety says automatic braking — which can bring the car to a stop if it senses an imminent collision — can reduce rear-end crashes by 40 percent.

My bad -- I thought that was one of the features of Autopilot. It still looks like the safety improved after the Autopilot rollout, but hopefully some public numbers with more granularity in regards to these software features can be released.

Tesla likes to market their version of automatic braking as part of Autopilot because it makes the safety of Autopilot look better, but it's a fairly common feature that is available on many models of cars, that the entire industry plans to make standard on all new cars in a few years, and that operates even when Autopilot mode - the controversial feature that drives Teslas into concrete barriers and trucks - is not engaged.

Also, they've had 3 fatalities so far. If one fully occupied model x (5 people) crashes and kills everyone... it's back to the average. If the people in the other car die too, it's less than average.

That safety advantage can be wiped out in a single accident. Tesla has been lucky that all 3 have been occupied by a single person.

Tesla’s aren’t exactly minivans. I doubt there are a lot of families driving around with them.

I really think Tesla is jumping the gun with autopilot.

According to the IIHS, which is sponsored by insurance companies and rates cars for safety, for models years 2009-2012, the following cars have had zero fatalities:

* Kia Sorento

* Lexus RX 350

* Subaru Legacy

* Mercedes Benz GL

* Audi A4 4WD

* Honda Odyssey

* Toyota Highlander

* Toyota Sequoia

> “In the past, when we have brought up statistical safety points, we have been criticized for doing so, implying that we lack empathy for the tragedy that just occurred.”

I have to say that is exactly how I felt reading the piece and I’m by no means easily offended by these kind of stories. The whole piece reads like a story Uber would have published. Throwing all sorts of excuses why it isn’t their fault but the big bad world being in their way. No matter if it is true or not, this is not the kind of tone people want from you right now. They could have brought the same information and show some empathy; especially when criticized before.

> After the logs from the computer inside the vehicle were recovered, we have more information about what may have happened.

Why does Tesla have access to the car after the accident at all?

> The driver had received several visual and one audible hands-on warning earlier in the drive

No. The logs likely show that signals to trigger the warning signals have been issued; they most likely do not indicate whether they had any effect (minor issue, since they most likely had an effect, but still presenting the likely "happy case" as a fact).

Apparently, the victim had complained to Tesla repeatedly that the autopilot would veer towards the barrier: http://abc7news.com/automotive/i-team-exclusive-victim-who-d...

On the other hand, if I knew the autopilot was trying to do such things, I would not use it so much.

Yeah, and it looks like the AP got confused by the lack of attenuation barrier, so a double whammy -- the AP probably would've avoided it if the barrier had been there, and it would've saved his life.

Honestly, I agree with Tesla in that maybe blame shouldn't be directed towards them. They should be held responsible for the crash, but not the death -- if you want to save lives, those barriers should be replaced immediately, and that is something CalTrans should definitely do.

As far as the autopilot previously giving several audible and visual warnings previously in the drive, and not detecting his hands on the wheel 6 seconds prior to the crash...

The Tesla model S detection of hands on the steering wheel is, in my experience, not very reliable. It often detects my hands off the wheel if I'm holding it lightly, ready to take over. And it often detects me taking control and disengages if I am holding it firmly. And if I'm paying attention to traffic, I will about a fifth of the time not see the visual warning in the dash.

These additional pieces of data can't really provide information on whether the driver was paying attention.

However, I am really curious about the reports that this driver had several times reported to Tesla that the autopilot was trying to drive him into this barrier. If so, I wonder why Tesla hasn't reported that fact. And why the driver would be using autopilot in that area at all, let alone not paying attention while doing so.

>semiautonomous Autopilot

Seems like an oxymoron that's getting drivers into danger.

>Tesla said its vehicle logs show the driver’s hands weren’t detected on the wheel for six seconds before the collision, and he took no action despite having five seconds and about 500 feet of unobstructed view of a concrete highway divider.

And it's not the first time.

and he took no action despite having five seconds and about 500 feet of unobstructed view of a concrete highway divider.

I suspect he may have been distracted or otherwise not paying direct attention, thinking "the car will drive itself, I know I need to watch it but I'll probably be fine if I don't watch it all the time"... and just happened to pick a very wrong time to not pay attention.

...and even when he did realise he was going to crash, there might've been a very strong https://en.wikipedia.org/wiki/Buridan%27s_ass effect which prevented him from quickly deciding which way to turn the wheel.

I wonder if Tesla can recreate the crash environment given the data captured from their sensors to reproduce the failure in a simulation?

See this article about a company attempting to do this in Unreal Engine 4

https://www.unrealengine.com/en-US/blog/51vr-crafts-photorea...

Imagine a future where every car uploads their sensor data to reconstruct a growing 3D model of the road network. Using this network AI can simulate millions of miles of driving without ever risking a physical real world crash. Any update to the cars autopilot system could first prove itself against 1 billion miles of simulation before ever being deployed to the 'real' world.

I would hope the serious players are already doing this.

Comparing the accident rate of autopilot Teslas to all cars is absurd and disingenuous, since obviously Tesla drivers are wealthier than average (they bought an expensive car), and thus much less likely to have accidents in the first place (car better maintained, more intelligent, more risk averse, less stressed, less dangerous behavior, etc.)

They are also admitting that their system can be used as a self-driving system (you can not have your hands on the wheel, and the warnings can apparently be ignored), and that it failed to act as such for 5 seconds and 150m hitting a static object.

Again, there should be criminal liability for them, and they should not be allowed to offer any system that is autonomous enough to be used without hands on the wheel but that can't correctly self-drive the car.

The one thing I want from a self driving system is to not drive at full speed into an object straight ahead of me.

Knowing that the auto pilot system had problems on this stretch of road, I don’t know if it makes good sense to trust it, but this shows we have a ways to go on the most basic test.

> The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision.

This would be hilarious if it wasn't so tragic. So did Tesla's autopilot!

Reading between the lines:

> The Tesla brand name takes this seriously. Question the headlines, opinions, etc.

> You are safe using Tesla. What happened to this person won't happen to you.

> The driver made some questionable decisions; the blame isn't uniquely Tesla's. Other safety mechanisms not under Tesla's control also failed.

> Statistically, Tesla Autopilot is safer than not using Autopilot.

> We accept that we will not be able to prevent all deaths. You need to too. Telsa has saved lives in addition to taking them. Focus on this please.

> We care about safety.

I am wondering if 100 Tesla would have passed that spot at that time would all 100 hit the barrier? Tesla should reveal why they can't detect solid objects and hit into them.

Now that we're set on introducing more and more autopilot and autonomous vehicles why dont we add sensors and beacons to our infrastructure to make all these things easier. Nothing really has changed since traffic lights.

It would help if barriers and construction obstacles would have beacons in them. This is not to replace cameras and radars but to help augment data whenever possible.

I think there is an opportunity for governments to help solve autopilot and autonomous vehicle challanges.

Why isn't the end of the barrier tapered instead of vertical e.g.:

http://fortressfencing.com.au/pub/media/catalog/product/d/b/...

(I know there was a previously destroyed crush zone, but still, better to be lifted and deflected than brought to a sudden stop.)

Answering my own question: "These crashes often led to vehicles flying at high speed into the very objects which guardrails or barriers were supposed to protect them from in the first place. To address the vaulting and rollover crashes, energy-absorbing terminals were developed."

https://en.wikipedia.org/wiki/Traffic_barrier#Barrier_end_tr...

I can understand this happening if the Tesla was following a human driver with “follow-distance set to minimum,” and that driver dodged the barrier, meaning the Tesla had no time to brake and could not move out of the solid-white lines.

According to another comment here the Tesla manual states that if you’re following another car that moves to reveal a stationary object, the autopilot may not be able to stop in time.

The driver has ultimate responsibility for operating their car safely. Autopilot does not change that, it says so in the manual.

And yet, that is not how Tesla advertises autopilot.

Does anyone have any data of % of accidents of autopilot vs drivers that don't speed? I am wondering, because I recently say some statistics that showed that a majority of drivers normally speed by at least 10mph, and that the majority of crashes occur when people are driving over the speed limit.

Is this going to be inherent and unavoidable for the machine learning based self driving cars? Perhaps self driving is never going to become a reality, just because of the problems like this (unless/until the road infrastructure supports them explicitly).

How much can we trust this data, provided by Tesla? Maybe the sensors that tell if the hands were on the wheel malfunctioned? Maybe he tried to restle the car into control? Would be helpful to have a video recording of the driver instead.

I would trust the data, because lying at this point could be criminal, or civilly significant. I wouldn’t trust all of the fluff and interpretation and hand-waving around the data though.

A level of skeptism is always healthy. Also recognise there are a endless 'maybes'. Anyone that works with data knows there has to be a level of trust and common sense otherwise you hit analysis paralysis.

And on 'trust' Tesla have a history of being open and upfront. Organisations tend to repeat a culture. So for Uber I would closely question their statements given an ongoing history of unethical behaviour across many levels. It seems likely they will lie or show limited truth to reach an agenda. Tesla's history seems open and honest.

Yeah, they've been open and upfront about things like the fact you can't maintain them yourself (they refuse to provide service manuals except where required by law in MA, where even then you pay a three digit fee to -view-, not get, a copy, and have to make an appointment).

Or the fact that they will remotely downgrade your car (to an older software version) and remove features of it (disable ethernet ports) if they find you "snooping" and discover references to new models.

Thats the kind of openness and honesty you laud them for?

Forget about Software / Ai bugs, My understanding is there is a hardware(lidar or radar) in tesla car. Shouldn't the car should have stopped when it's hardware sensed that its definitely going to hit the wall.

I just cancelled my Model 3 reservation based on the way this is being spun.

Imagine if every single fatality caused by a car had this much scrutiny. They should, and while Tesla is copping it , I’m very happy that safety in autos is actually a topic of interest.

The dude didn’t have his hands on the steering wheel for over 5seconds? Madness

Humans do not understand such an arguments in such a situation. Hope you will do better business on Vulcan dear Tesla.

PLEASE stop comparing crash statistics for a car with MSRP $80-150k to the general population. Sample bias (among other basic problems) is such a basic consideration that Tesla ignoring it makes me sick. This company will do/say anything to manipulate public perception. People on HN should at least attempt to see through this rather than buying it because Tesla gives them the warm fuzzies.

https://www.technologyreview.com/s/601849/teslas-dubious-cla...

Personally I would only use a self drive feature on the middle lanes of a highway if I was going to be distracted with something else, there’s too much bullshit going on at the edges for it to be safe.

As someone who drives this exact stretch of road every day, in a Tesla, I agree. This lane is the end of a long stretch of road with either zero or minimal shoulder, where even the slightest glitch could cause a scrape on the side of your car. I'm always paranoid driving this stretch.

This concrete barrier is also quite a distance past where this left exit lane starts to separate from the rest of US-101, so there are really only two ways you'd risk hitting it. First, and probably most common, is people who change their minds last-second about taking that exit. Second, would be a serious AP screw-up (like what might have happened here).

FWIW, I've never had AP engaged when driving through this spot, and almost instinctively disable it when taking a highway off-ramp/change-over.

In the moments before the collision, which occurred at 9:27 a.m. on Friday, March 23rd, Autopilot was engaged...

The rest may be true, may be germane, or may not be. The only part that matters to me until the results of the NTSB investigation is made public, is that the autopilot did this.

It's more about what the autopilot didn't do than what it did do. It did drive straight ahead into a concrete barrier. It didn't see the barrier. If it had seen the barrier (as is reasonable to expect), it wouldn't have driven into it. Thus what it didn't do caused what it did do.

The fact that Tesla even made a press release of this nature disturbs & annoys me.

When the NTSB is investigating an aircraft or train accident we don't see the vehicle manufacturer rushing out a "well we told him to take control" statement and a dump of telemetry. At most they state "we are of course working with the authorities to establish the cause and any corrective actions" until the investigation is completed.

Tesla needs to mature from a blame-the-user software-industry mentality to something more befitting their responsibilities.

This is 100% anecdote: A friend of mine test drove a Tesla, and said that when he was in the far right lane on a highway, the autopilot would get a little sketchy at off ramps. It didn’t ever off or anything, he described it as “jiggling” like it wasn’t sure what do when the right lane marker disappeared. He found that less than inspiring, although he did find the acceleration intoxicating.

I have a question- why keep ap on at all? Are they beta testing it and using metrics? Because I don't think it's the difference between someone buying a car and not.

I repeat.

This is a very deceptively crafted sentence. if i drove on AP for 15 minutes and i was alerted to put my hands on the wheel 2 minutes in.

It would fit tesla's specially crafted statement.

It makes it look like user was warned and didn't respond by putting their hands on the wheel but its completely false. The AP drove him straight into the barrier with no warning.

But this deceptive statement worked as Tesla carefully planned. look at Jalopink title "Tesla Says Autopilot Was On Before Fatal Model X Crash, But That Driver Didn’t Abide Warnings"

Please don't post duplicate comments to HN. It lowers signal-noise ratio and makes it hard to merge threads.

I get downvoted reliably for taking issue with Tesla and Musk, but here I largely agree with them. People need to consider such incidents in the context of the overall danger of driving.

That said, Tesla needs to maintain its PR in order to do their next capital raise shortly or they may be out of luck.

Reply on news.ycombinator.com