Hacker News Reader

Comment by apeace | Hacker News Reader

I'm a huge Musk fan and wish him and Tesla all the success in the world. But releasing any kind of machine-assisted driving without a near infallible "don't hit objects in front of the vehicle" function is simply irresponsible.

This crash, as well as the one where a Tesla crashed into a concrete barrier[1] are evidence that their tech is not ready for release.

How much do you want to bet the Waymo self-driving technology would have avoided both of these? Seems like one of the simplest cases to handle: something up ahead is blocking the way!

Anyone who uses Tesla machine-assisted driving features today is putting themselves in grave danger (except for low-speed features like parallel parking).

[1] https://www.youtube.com/watch?v=YIBTBxZ3NWw

View on HN

I think Musk looks at it differently. There appear to be two fundamental truths about the current autopilot system. First off, the number of fatalities that have occurred with autopilot active compared to the amount of miles driven with autopilot active suggest that autopilot is safer than a human driver. Secondly, autopilot still is not flawless and it is possible for it to cause fatalities.

Some people feel the latter should be enough to stop allowing drivers to turn on autopilot. I am betting Musk feels that the former serves as a moral obligation to get the safety features of autopilot in as many cars as possible in order to save as many lives as possible. He is throwing the switch on the trolley tracks [1].

[1] - https://en.wikipedia.org/wiki/Trolley_problem

"the number of fatalities that have occurred with autopilot active compared to the amount of miles driven with autopilot active suggest that autopilot is safer than a human driver."

Obviously you see how this deduction is subject to dozens of biases stemming from the fact that people turn on autopilot when the road conditions are safe to begin with. One would have to really carefully filter this data to avoid these biases, otherwise such conclusions are absolutely bogus.

The 40% reduction in accident rate was the result of the NTHSA investigation: https://techcrunch.com/2017/01/19/nhtsas-full-final-investig...

"It’s essentially as good as result as Tesla can have hoped for from the U.S. traffic safety agency’s investigation, which took place over the last six months. Reuters reported earlier on Thursday that the investigation would not result in a recall of Tesla vehicles, but the full findings show that in fact, the federal regulatory body found plenty to praise while conducting its inquiry.

The investigation does conclude with a minor admonition that Tesla could perhaps be more specific about its system limitations in its driver-assist features, but acknowledges that all required information is present and available to drivers and vehicle owners."

I know this report. This does not deal with the bias problem however. Obviously it reduced the number of accidents in good road conditions where the autopilot works and is primarily used (much like auto-stop feature did and BLIS). Particularly after they introduced the driver monitoring (this report was conducted after that) - now in fact the driver using autopilot is likely more alert then one not using it.

But that does not indicate anything about conditions where autopilot should not be used in which it is gravely dangerous.

It is all about tail of the distribution, which why it is not so obvious.

Why does it matter if it's dangerous if used incorrectly, if the majority of users use it correctly, and the overall effect is that fatalities go down?

I don't think the argument here is whether technology features can improve safety. The core of the argument is (and always has been) over misleading marketing and naming (suggestive "autopilot"), which exactly encourages the misuse of the feature. When used correctly, with alert driver it can clearly be beneficial. But when misused e.g. by this poor young person who plowed into a truck, this feature could be deadly.

This has a long way to go for true "autonomy" and until then such marketing should be avoided and examples of misuse (which youtube is full of) should be explicitly discouraged and if possible penalized. Tesla should be more explicit and pro-active about it. This is my only point really.

40 years old is "young"? You must be as old as I am :-)

I was thinking about the Chinese boy who hit the truck parked on the service lane, not the guy who decapitated himself in Florida. But either way, both were tragic.

Put "don't crash into things" in the manual and all accidents are incorrect use by definition? That's why it matters.

uh, because if the effectiveness drops dramatically then you are literally going to be killing people.

>But that does not indicate anything about conditions where autopilot should not be used in which it is gravely dangerous.

Yes it does include those conditions because this specific accident occurred in conditions in which autopilot should not be used. That single fatality is the only one that autopilot has seen in close to one billion miles. That is an order of magnitude better than traditional fatality rates.

That leaves two possible scenarios. Either people are likely to use this system incorrectly, in which case many of those billion miles were used in poor conditions for the autopilot. Or few people are likely to use the system incorrectly, in which there is little to worry about since the autopilot system has never been involved in a reported fatality when used properly.

There was another accident in China, in a controlled access divided highway where a Tesla failed to notice a maintenance vehicle moving slowly ahead on the side of the road, ended up rear-ending it and killing the driver.[1]

By the way, can you point me to statistics about “traditional fatality rates” at controlled access divided highway in conditions where the line markings are clearly visible? It's dishonest to compare apples to oranges, especially given that a Tesla can't or won't recognize one from the other.

[1] https://www.nytimes.com/2016/09/15/business/fatal-tesla-cras...

How can autopilot be involved in a fatality when used correctly, when by very definition the correct use implies alert driver who is responsible for everything that happens?

We have both used the term "involved" and not at fault. If autopilot is on when (or immediately before) a fatality occurs, that is being involved in a fatality. The driver may be at fault in those instances if they are not alert. Although it is also possible that a separate driver is at fault and both the autopilot and Tesla driver performed adequately. For example a second driver might rear end the Tesla driver and push the car into oncoming traffic.

That sounds like a comparison between autopilot miles and general driving miles. What would the comparison look like between autopilot miles and interstate highway miles?

Edit: nope, I was wrong. 1.3 to 0.8 airbag deployment crashes per million miles for Tesla cars before and after autopilot release.

This seems like trying to argue that brakes are bad because, while people normally use them at times when they should stop, it's also possible to use your brakes at times when you shouldn't stop.

The fact that something can be used used in a dumb manner is good to know so you can avoid doing so, but that doesn't negate the fact that cars are safer overall with it than without it.

Yes but if I asked you to prove brakes were safer you'd be able to do it pretty easily. The fact that no one has actually set out and done a study proving that autopilot is safer than human driving is disconcerting considering these vehicles are already being put on the road.

Seeing people abandoning the precautionary principle so they can latch onto an over-marketed, over-promised Muskian view of the future is sad. And in five or ten years we're all going to be looking back wondering what made us so dumb to think that computers could actually drive cars.

Toyota came out saying you'd have to gather 8.5 billion miles of driving data to make any claims with statistical certainty about the ability of any autonomous driving system to prevent fatalities (I've heard other outrageous numbers thrown out to this effect, though I don't recall them off the top of my head).

Though by the time you get to the 8 billionth mile the technology will have evolved, rendering the earlier data moot. So it will be pretty much impossible to do any sort of authoritative study on how safe this technology is until it is widespread, and mature.

If a study could reliably demonstrate that autonomous vehicles have fewer collisions it would strongly hint that fatalities are also fewer.

Number of collisions and number of fatalities will not necessarily have the same proportions as for humans.

You could have a car that never gets into little collisions that often happen to human drivers because of carelessness, but are rarely deadly, but the same AI might have a faulty reasoning every once in a while leading it to accelerate to 200kph before running straight into a wall. This hypothetical AI would have very few collisions, but maybe still be much more deadly than a human.

Humans are the only data we currently have significant numbers of miles logged for. Any extrapolation to AI is meaningless.

After the first fatality while on Autopilot, the NHTSA did a study on whether Autopilot was inherently safe and whether it had any defects that needed to be addressed. It found that Teslas with Autopilot were substantially safer than Teslas without. I'd like to see a better study (since that one was mainly aimed at answering the question of whether Teslas with Autopilot should be recalled), but someone has looked into it at least.

That's not quite accurate, the NHTSA study was not designed, in any way, to compare human-driven miles and autopilot-driven miles for safety. The only thing it showed was that vehicles with TACC + Autosteer had less accidents than vehicles with just TACC. Whether TACC + Autosteer is safer than human drivers is up for debate.

True. However, it shows a very substantial decrease in accident rate in the population with Autopilot features, vs. the population without. Correlation, not causality. It could be that Model X drivers are inherently safer than Model S drivers (they would be in the latter set), or it could be that just having the features in the car makes drivers do better even without using them.

However, the inference that Autopilot reduces incidents per mile seems very likely to be true.

The key point is that you can say things like, "per million miles, Tesla auto pilot use has resulted in X fewer fatalities", however, unless the data leading to that statement has been normalized against data representing the exact use case of Tesla auto pilot then it is false. The only way to say that Tesla autopilot is safer per million miles than a human is to base the figure on when the Tesla auto-pilot could be used. I'm not sure I've ever seen that data from Tesla. Who knows, maybe they have it and I've just not seen it. If they do, very cool. I would assume they could get the data by taking sensor readings from Tesla cars under human control in situations where auto-pilot could be used.

The NHTSA found a 40% safety improvement when comparing Teslas with Autopilot to Teslas without. I wouldn't put a huge amount of stock in the exact number, but because one group never uses Autopilot and the other offers a realistic view of when people actually do use Autopilot, it seems like enough to say "Yes, Autopilot makes cars safer in aggregate."

The data is suspect at best and should require a third party evaluation. I'm speculating that the reason for that 40% safety improvement is more a result of the difference between where autopilot can and cannot be used. People with autopilot equipped Tesla vehicles get the option because they likely drive more highway miles where it can be used mile after mile. Those who don't, likely don't and may be in statistically more dangerous driving situations like short commutes over high speed roads. When I commuted to the office, I'd pull out of my 25 MPH subdivision road and immediately accelerate to 65-70 MPH onto the highway and proceed to drive two miles. There's no reason for autopilot in those conditions so I wouldn't purchase the package, but yet making a left-hand turn onto a five lane highway where cars are regularly traveling at high speed is one of the more dangerous things you can do while driving for which if I got hit in a side impact could easily be killed and because of the location on a highway, would be counted as a highway fatality.

> The data is suspect at best and should require a third party evaluation.

And the NHTSA doesn't count as a third party?

> I'm speculating that the reason for that 40% safety improvement is more a result of the difference between where autopilot can and cannot be used.

To say that tesla owners without autopilot tend to drive in situations with 40% more likelihood of accidents than those with autopilot, sounds like a massive stretch without data of your own.

You are making an incorrect assumption. When Tesla released their original Autopilot, it was a no-charge SW update for all cars equipped with the necessary hardware (which was all cars built within the previous xx months). Thus, the drivers with Autopilot capability were driving the same commute in non-Autopilot capable cars the day before. There was no customer selection based on perceived utility of Autopilot.

I think you misunderstand the data. This isn't "people who opted into Autopilot vs. people who opted out" — the no-Autopilot group is people whose Teslas were made before Autopilot was introduced.

And therefore obviously this is not an unbiased control group. Perhaps the more recent teslas have slightly better/less used brakes? Have they controlled for that? There are countless ways in which such conclusions can be flawed.

Statistics is really tricky, even with very large samples at recent presidential election had shown that beyond any doubt.

How many fatal collisions did they use in this dataset? This can't be a serious assessment.

With the same logic if people use it when road conditions are safer then they are simply using it as intended.

The design can help prevent people from using it when it's not appropriate or have mechanism to keep them alert and focused.

That is all NTSB seems to be saying here... they aren't calling into question the idea of highway driving assist systems, which are (already) showing potential to significantly make highway driving safer - the time when people are driving at the highest speeds.

No system will be perfect from preemptive testing either, it really does take real world mass adoption to detect some uncommon flaws unfortunately... Tesla is going to bear the brunt of this fact. You simply can't simulate every human (irrational) behaviour nor can you simulate every environment it will be used in.

Note, the original NHTSA report compares accidents per mile traveled in Tesla vehicles with and without autopilot, and notes that vehicles WITH autopilot have 40% fewer accidents than those without.

This seems like very clean data, as it doesn't actually consider how often autopilot is turned on, and it's over a fairly large sample of drivers, locales and miles driven.

This is just correlation, not causality, but the effect is too big to be by chance.

The comment you are replying to is suggesting that the miles driven with autopilot may have been inherently safer miles, e.g. autopilot was more likely to be turned during the safer parts of a drive.

If true, this would be an unfortunate example of selection bias.

You are missing the point. The report doesn't compare miles driven with and without autopilot. It compares ALL miles driven in cars with Autopilot function, whether it is turned on or not, vs. cars without Autopilot function.

To boot, these are, for the most part, the same cars before and after, since Tesla shipped cars for almost a year with the Autopilot hardware built in but not activated, then activated it for all cars at virtually the same time via an OTA. Before that date, any miles clocked show up in the "no autopilot" column, the next day they show up in the "with Autopilot" column.

To further flog an expired equine, the study showed a 40% reduction in airbag actuations per million miles driven across all miles driven in cars with vs. without Autopilot function, not comparing only miles where Autopilot was turned on.

There are certainly possibilities for this data to be skewed or biased, but it's definitely not a comparison of safe Autopilot miles vs. unsafe manual miles.

> suggest that autopilot is safer than a human driver.

This is what is misleading in my opinion. The study says that Tesla's assistive technology works, that is they assist humans to make driving safer. All driving assistance technologies satisfy this.

In other words

Autopilot + human is better than a human.

Then why is the furore about Tesla? Because they named it autopilot and call it safer than a human. To say autopilot is safer than a human seems to imply that autopilot alone reply be better than a human driver.

Maybe you're smart enough to see this, but as is evident, many believe that autopilot alone is better than a human driver.

There's nothing controversial about front crash avoidance being beneficial, especially with auto braking. As a feature it is more and more widely deployed, probably becoming standard in the next few years.

It would be interesting to see how often autobraking is key and how often the rest of the autopilot improves safety.

but most safety devices are explicitly for when conditions are not their best and sadly Autopilot doesn't function in those bad conditions. Its a fair weather driver assistant.

His moral obligation is to admit its limitations instead of grand standing on what it might do one day and dancing around with selected statistics.

the simple fact is, it failed and will continue to do so. they have made some technology choices they may not be able to surmount not matter how much noise they generate or their sycophants generate.

the promise of level 4 and 5 autonomous driving is so great that we cannot afford to have it damaged in the public eye

Releasing any kind of machine-assisted driving without a near infallible "don't hit objects in front of the vehicle" function is simply irresponsible.

Yes. There are four videos of Tesla cars crashing into objects which partly protruded into a lane. Three vehicles, one construction barrier [1], all on the left side. One fatal.[2] Relying on the Mobileye vision system, which is "recognize car/pedestrian, compute range to target" for anti-collision was grossly negligent of Tesla. Tesla has a radar, but it can't be very good if this is happening. Even the old Eaton VORAD radar, which we used on our 2005 DARPA Grand Challenge vehicle, would have avoided that.

This is the simplest and most important part of automatic driving, and Tesla botched it.

[1] https://www.youtube.com/watch?v=YIBTBxZ3NWw [2] https://www.youtube.com/watch?v=fc0yYJ8-Dyo

And yet, according to the NHTSA, cars are still much safer with it than without it. You seem to be smack-talking rather than offering a reasoned analysis of whether Autopilot is better than no-Autopilot.

>Autopilot is better than no-Autopilot.

If only there was something to put in the middle of this reasoning!

Elon Musk fucked up as soon as he named it "autopilot". He set expectations at autonomous, and released it as a "beta" because that's what you do with some web app, and then when people started using it as an autonomous driving product (because that's what "autopilot" means), they unsurprisingly got killed. But shruggie! Move fast and break things! It's beta! What ya gonna do?

On the other hand, if it was called "driver assist", or "super cruise", or some other thing, then drivers' expectations wouldn't be set at "autonomous driving", and then the'd be paying more attention, and probably wouldn't end up dead.

I'm sorry, but this is all about setting expectations, and Elon fucked this up.

I'll agree that the name "Autopilot" seems to mislead people, but that is a complete side track. Neither the person I replied to nor the person he replied to were talking about the name; they were talking about the technology and whether it was responsible to release it at all, with any name. (For one clear example, the GGP criticized "releasing any kind of machine-assisted driving" with Autopilot's limitations.)

It doesn't seem completely misleading, airplanes do a lot of flying on autopilot, but it doesn't mean there's no need for the pilot/co-pilot to be at the controls...

No it doesn’t but maybe most people think that’s exactly what it means...

What you're starting to edge towards, then, is an educational issue. Personally I feel the only time I've seen an 'autopilot' shown as a fully autonomous system was in the 1980's film Airplane!

I couldn't narrow down exactly where in life I learned about autopilot systems in aircraft, but I always considered it to be common sense that autopilot doesn't mean "can completely fly itself"

The media is constantly bringing together the terms autopilot, and driverless or self-driving cars. There's definitely the suggestion that the driver is completely unnecessary. You can't expect the average Joe to look this up. It should clearly be marked as beta and dangerous without supervision.

Also Tesla says so themselves: >All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than...

AFAIK Autopilot is clearly marked as a beta and displays a dialog reminding you of that when you activate it in your car. And "full self-driving capability" is not a reference to Autopilot — it's saying that Tesla eventually plans to implement full self-driving (as a separate feature) and the car includes hardware that will allow you to enable it with just a software upgrade.

Marking something as beta is a bullshit CYA move. If it was truly a beta test, it wouldn't be in wide public release.

A car is not a web app. It just isn't. This behavior literally wouldn't fly at say Boeing, or anywhere else that makes machines people's trust their lives to.

I think the big problem is that "autopilot" in a small plane just means "fly in the same direction at the same height and speed" like cruise control, but the big commercials jets absolutely can and do take off and land themselves, technically requiring zero input from a pilot at any point in the flight.

I think more people are aware of the second point than the first, and that's why the common perception of "autopilot" is more about level-5 driving than cruise control

I'm gonna need something better than myth and superstition to buy into this meme that autopilot users don't understand what autopilot is/isn't capable of because of how it is named.

The car makes it clear to drivers what "autopilot" means. The only people who might be confused by the term are non-Tesla drivers.

The NHTSA should compare ACC+LKA+AEB vs Autopilot (which is all those plus Lane Centering).

I'd bet those 3 combined are much safer than Autopilot, which complicates their implementation to offer a marginal convenience of not having to steer (but to constantly have to nanny lest it put you into a ditch)

I think the problem is deeper than 'is Autopilot better than no-Autopilot'. Is Autopilot better than no Autopilot under what conditions? Is Autopilot being marketed to customers in a way that encourages them to use it in the wrong conditions? How are changes to the Autopilot code being vetted before being deployed OTA to customers?

I think if used properly Autopilot can be safer than most drivers without it, but the crux of the issue is that many people will trust the system more than they should, which could make them more dangerous. We're in a dangerous inbetween state with autonomous driving where it's just good enough to fool some people into thinking it's foolproof when it's not.

They have a Bosch supplied radar but insisted on doing their own signal/object processing.

What seems to have gone wrong in radar processing is that they'd get false alarms when approaching an overhead object such as an overpass when not on flat ground. Older model automotive radars scan horizontally, but not vertically, so there's no height information. The "solution" was to ignore some of the radar returns, not fit a better radar. Even with that, there's no way that the China incident of rear-ending a street sweeping machine on a freeway should have happened. From the crash videos, it looks like any obstacle not directly ahead of the vehicle centerline is ignored.

2D scan radars exist.[1][2] Tesla didn't use them.

[1] http://www.fujitsu-ten.com/business/technicaljournal/pdf/38-... [2] http://www.oculii.com/sensors.html

On the other hand, if you know that the technology is not completely infallible, but better than humans in a general sense and will save lives by releasing it, wouldn't you be morally obligated to do so? Wouldn't holding it back even though it would save lives be worse?

I think people will prefer risking being killed by their own mistakes than by the mistakes of an AI, even if that risk is somewhat higher, maybe even considerably higher.

Nope... I am actually happy to admit that I am a below average driver (as driving is sooo boring), and from what I read, the AI is already driving better than me - if used in the conditions it is designed for.

Don't they already have that choice, though? No one is forcing anyone to use Autopilot.

That's my main issue with the parent posts and the self-driving sentiments they're expressing - the person using it was grossly negligent and ignored all the warnings presented. You accept that the technology is in beta and that you accept the risks of that. The person ignoring those warnings is, more than likely, much less safer of driver than Autopilot ever could be.

I just really don't see how this could be pinned as Tesla's fault. We're treading very closely to territory where we absolve people of personal responsibility simply because we have technology that we can blame, even if it's irrational or misdirected blame.

Surely they can not turn on autopilot, if that's the case?

For me, I prefer life over a false sense of control.

I do. I have no interest in a self driving car. I hate being a passenger in a human driven car too. Give me control of my own destiny!

Does it matter what you prefer? You're also risking everybody else's lives while driving. Should they have a decision in whether you're allowed to drive?

Disagree. I will have my family use whichever is statistically safer.

This might be harder to gauge than you think. Maybe a Tesla is much safer on the highway, but less safe in the city, or the other way around. Or less safe on ice, etc. The averaged risk (and we probably won't have anything else for a while) over the whole population might be quite different from that for your actual use.

It can't be marginally better than humans, it has to be an order of magnitude or more better. Humans are irrational, and incidents like this erode public trust.

I agree, but there is also news about how Tesla's automatic braking features saved people from getting into collisions. I would think that might counteract some of the negative news. See this one from six months ago: https://www.youtube.com/watch?v=Ndeb1pMAsh4

It shouldn't counteract the negative news because every car manufacturer offers automatic emergency breaking as a feature so it's not a distinguishing benefit of Tesla, it works fine without Tesla's controversial and dangerous Autopilot mode enabled, the main example Elon Musk himself posted of it preventing a collision was in manual mode in conditions totally unsuitable for Autopilot, and the entire industry already agrees that it's a worthwhile safety improvement. I believe it's even going to become mandatory on new cars in the US at some point for safety reasons.

Most manufacturers have automatic braking. But they do not pretend it's "autopilot" nor let user to rely on it. It's just there to save their ass if shit hits the fan.

Apart from this being a very basic feature in almost every (premium) car manufacturer offers, this is a very grave misunderstanding of human psychology. You can never cancel fear with good news. Otherwise we'd be happy about how safe we are today, not worrying about terrorism (in the US that is, there are countries with legitimate elevated terrorism risks that are scary and actually worrisome).

I think your point is void here, because in both of the cases I mentioned, the car was clearly worse than a human. Who drives into a concrete barrier or the middle of a tractor trailer truck?

If the technology were better than humans I'd agree.

>>Who drives into a concrete barrier or the middle of a tractor trailer truck?

I have seen a drunk driver do exactly that.

Someone who falls asleep might also do that.

[edit] Come to think of it, I have almost caused an accident a decade ago. I was momentarily distracted as I turned to talk to my passenger. I was able to brake in time, as a car in front of me had suddenly stopped. Nevertheless, it was only a matter of maybe one second to avoid running into the other car. Me-a-decade ago might been in a similar accident.

I know one person that drove into a light post in the middle of an empty parking lot. Daylight, sober, not texting, not tired, not stunting - they were looking at the exit and simply didn't see the one light post between their car and the exit. "That light came out of nowhere!"

Human car accidents are not news because roughly 100 Americans die in car accidents every day, out of ~15300 car accidents per day.

Also somebody who's been blinded by a bright light, or is suffering from a medical event, or is looking at their cell phone. There are lots of things that cause people to hit objects that should be easily avoidable.

Neither of those people should be using autopilot. It wouldn't save them.

I can also avoid being in either of those situations.

If autopilot is comparable ot an exhausted or drunk driver why do I want to use it?

> I think your point is void here, because in both of the cases I mentioned, the car was clearly worse than a human.

But separate cases are not really interesting when comparing accident rates. Think of a more extreme made up scenario:

You can either drive yourself and be part of the "1.25 deaths per 100 million vehicle miles" statistic (US), or use an automated car which causes 0.25 deaths - but it spontaneously explodes in that case. No other car would just explode randomly while you're driving it. (It's clearly worse in that way)

Do you want to drive yourself, or take the automated one?

Should we judge performance in every scenario distinctly, or in aggregate?

That is, if the risks of one edge case goes up while the car performs far better in the vast majority of situations, is the car worse than a human?

> Who drives into a concrete barrier or the middle of a tractor trailer truck?

It happens all the time, dude. Most famously with Jayne Mansfield, but commonly when people are paying attention to their damned phones, or trying to sort out a fight among the kids, or just distracted by their marriage, or work problems, or...

I used to know a woman who drove right into a city bus that was stopped at a bus stop, lights flashing and everything. Fortunately no one was injured in that accident.

Who drove into the rear end of my car two weeks ago on the interstate? A human.

The bar is not infallible in general, just infallible at driving directly into obstacles.

Is it realistic to worry about a car that can't get that right, but is simultaneously safer than a human driver overall?

What if Tesla released autopilot as a completely passive accident prevention driving assist?

Tesla seems to be allowing only two options - one with no driving assistance, and one where they can call it autonomous.

Emergency braking is still active even if you don't have autopilot turned on. This is also true for a lot of other cars with Level 2 autonomy.

The statistics being touted for reduced accident include emergency breaking.

Yes. I'd like to see AEB broken out of the accident number. Looks like this NTSB investigation did not release such a number.

It's a tough thing to weigh however when it's perhaps "better than humans" in 98% of situations, but "on par with" or perhaps even "worse than humans" for the remaining 2%. That's where it gets sketchy to me, at least.

That said, I'm casually in the market for a Tesla (ideally pre-owned) and one of the features I'd require is the latest auto-pilot hardware. I'd definitely make use of it right away too -- but I'd do so with the understanding that it is far from infallible, and that I still need to keep substantially all of my attention trained to the road just as I would in any other non-autopilot vehicle.

> Anyone who uses Tesla machine-assisted driving features today is putting themselves in grave danger

That's their right to decide or not. What bugs me is that they also put me (and everybody else on the road) in danger.

"In 2013, 32,719 people died in motor vehicle collisions." (fro: https://www.millerandzois.com/car-accident-statistics.html) And you think it's a big deal that 2 crashes have happened? Also that truck driver broke the laws of the road and caused that mans death. That man broke the laws of the road and caused his own death. Pretending that the computers are to blame is silly. Put the blame on the people responsible not a computer.

BTW same with the barrier incident. That was a driver failing to be vigilant. It was his fault for leaving autopilot on in a non-standard environment.

Obviously this truck may have broke down in that lane and could not be moved out, which removes the fault from the truck driver. In general cars have drivers, exactly because such things can happen and will happen without anyone else to blame. Neither does the blame matter if you are dead as a result.

The 2 (in fact there were more) crashes are important because they show the state of the technology. Also keep in mind 99.9% drivers don't have an autopilot, 0.1% does. 2 crashes among the 0.1% are just as important as 32k crashes among the 99.9% of the rest.

> [...] where a Tesla crashed into a concrete barrier[1] [...]

Of course it should never have happened. Still, I'm impressed by how gracefully it handled the accident; emergency lights, keeping the lane and slowly braking the vehicle to a halt (again, such frontal collision was still unacceptable).

This crash, as well as the one where a Tesla crashed into a concrete barrier are evidence that their tech is not ready for release.

But then you're forced to compare the autopilot accident rates against the regular old human accident rates. When you do that, the autopilot wins every time.

Put another way, I'd be comfortable getting into one of these today. I trust the computer's bugs a lot more than I trust my own. I also trust its reaction time above my own, which is more likely to keep me safe if someone else does something stupid.

That's a major flaw in people's thinking about autopilot programs.

An autopilot doesn't have to be 100% safe, it just needs to be significantly safer than a human pilot and in the long run it will save lives.

I think the fallacy is thinking that it's okay for humans to be flawed but machines should be infallible.

Why are your options autopilot can unaided human?

Wouldn't you compare autopilot va autopilot assisted human? That is, where the autopilot is a passive system that only comes into play to prevent accidents.

I feared him rushing the feature long ago. Thought they raised the bar adequately since. He should stop spawning new projects and give more resources to this one instead.

I mostly agree and I wonder how long until incidents like these start eroding public trust. I fully believe that fully autonomous driving can and will bring a whole new paradigm to transportation in the US, but it won't happen if we try to rush something so critical.

Was it rushed though?

Can you compare the number of hours driven with the autopilot and compare it to the statistics of human drivers, and determine which was safer?

This was a fatal crash, but faulty airbags caused the death of 10 people.

What selfdriving cars needs, rather than more time to bake, is a PR or lobby group, that can help spin (put in perspective) these types of events.

I can only speculate, but the fact that engineers have left Tesla over this very issue is telling.

No, these are inexcusable deaths. This is literally a fatal flaw in the system.

I'm sorry... I just don't buy that. Airbags have deployed and killed people by snapping their necks yet we include airbags in every single vehicle on the market without requiring that the drivers or passengers read and accept a warning about airbags.

Autopilot requires manual consent from the user to even enable it and has so many overrides for the user that I feel like you're really stretching it by calling it a "fatal flaw". All it would have taken to prevent this accident was the person driving to hit the brakes in time. Even if they were just using cruise control in their vehicle, they would have died. This was completely about inattention and irresponsibility on the part of the driver. I can't see how anyone is making the leap to this being a "fatal flaw" in the system.

The public will trust it as long as it is properly regulated.

Have you examined the history of air travel? Entire airliners have crashed and killed everyone on board and people still continued to take flights. Today airplanes are safer than even trains. The reason is simple: the industry is very heavily regulated and everyone has to adhere to strict processes with tight controls.

> The reason is simple: the industry is very heavily regulated and everyone has to adhere to strict processes with tight controls.

The reason is even simpler: private companies using private money to improve flight technology

You really honestly think that those same companies would achieve the same record without FAA?

Banks are highly regulated, yet that hasn't stopped them from enriching themselves and the wealthy elite at the expense of the average citizen. It's all sanctioned by the government, built into the system.

Governments after all are run by people, just like companies, and people are prone to greed, corruption, and abuse of power, regardless of their position.

So, basically, you want to outlaw most current implementations of Level 2 autonomy, which are being shipped by a large number of carmakers.

I was unaware that Waymo was planning on shipping a Level 2 system.

BMW's auto cruise control and distance keeping + emergency braking etc isn't nearly good enough either as I know from experience. But it still helps.

I use cruise control on my 2012 Honda all the time.

Am I putting myself in danger?

their tech is not ready for release.

Sounds like a typical software development effort. we always used to say that if we built bridges the way we write software, there wouldn't be many bridges. Well, now we build cars the way we build software, and guess what is happening...

I firmly believe that this is all safety critical code, and we should look to NASA to ask how its done:

https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev...

and

https://www.fastcompany.com/28121/they-write-right-stuff

But hey, all that boring "safety" stuff is expensive, and takes too much time to complete....

EDIT: As has now become usual for HN, I am being downvoted instead of debated. This used to be a site for good debate. Too bad that's no longer the case.

I think you're being downvoted because you're making the assumption that Tesla and every other autonomous car manufacturer isn't putting any resources into "all that boring 'safety' stuff". That singular point really devalues the entirety of the rest of your post.

Interesting point, and not what I was assuming at all. I simply believe they don't put enough thought into it (and clearly the NTSB agrees).

Downvotes should be about bad form, not about "I don't agree with you". HN used to be about the former, but now is much more the latter, and it is a poorer community because of it. HN isn't the first online tech-news community, and every single one of these communities that uses an up/downvote mechanism to self-regulate comment quality have gone the same way: as an increasing number of users gain voting privileges, voting replaces debate. When that happens, the community becomes a simple echo-chamber for the "established" truth of that community. This increasingly keeps away quality commenters, strengthening the effect.

When voting, especially downvoting, is a frictionless, costless action, the quality of varied discourse will sooner or later die.

Again, I disagree. I think that downvoting is for comments that are bad form or don't contribute significantly to the discussion. As I mentioned before, your singular statement devalued the rest of the comment which, to me, makes it lack any contribution. You've make a very large generalization that is easy to dismiss. Why should anyone put any weight into the content that came before that?

Move fast and break things turns out to be a bad way to build cars.

Reply on news.ycombinator.com