Hacker News Reader

Comment by sfifs | Hacker News Reader

Is the base size and confidence levels reported anywhere? I get professionally suspicious when reports publish percentage figure comparisons of rare events.

View on HN

What events do you consider rare? Car crashes?

Crashes of cars that were hard enough to trigger the airbag, but still not so hard that the car would not get an autopilot update later in it's life. The "before autosteer" column does not include miles or crashes of Teslas that did not survive until OTA availability.

Presumably they meant airbag deployment crashes in Teslas.

We can estimate the absolute number from Tesla's press release in December stating that 1.3 billion miles have been driven on Autopilot, while the NHTSA report states the accidents they count occur 0.8 times per million miles.

So in that case it's data from about 1000 crashes in the Autopilot case, and more in the non-Autopilot case. Which tells us that the statistics should be good enough (anything else would be a major scandal).

However, that is not what they are comparing! As the NHTSA report states in footnote 21, they're not actually comparing Autopilot miles to normal driving. They are comparing crash rates in vehicles before and after Autopilot was installed in those vehicles. That muddies the waters quite a bit in my opinion, especially when they don't tell us how significant this decrease is. I'm not saying it's wrong, but I'm also not very confident that it is a causal effect, as opposed to just a correlation.

Here is a PDF link to the NHTSA report, since TFA only links to a Scribd (thus unusable) version.

https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF

So they're comparing miles driven manually before a certain date to miles driven manually and automatically after a certain date? It certainly implies that Autopilot is safer, but why not separate it properly (Autopilot vs manual) and give us far more accurate information?

It just seems...odd.

Comparing all miles both before and after neatly sidesteps the problem that the autopilot only works on relatively good roads.

Even so, Tesla should be able to tell whether or not you're driving on a highway with all of the data they collect. Why not compare that data to the Autopilot data? This report wasn't made by Tesla, it was made by the NHTSA. So why not do a more meaningful comparison?

It's weird.

Actually you can argue that car crashes are rare. 1.2 deaths per 100 million miles is rare. (I'm not able to find statistics for accidents without deaths, but we can extrapolate)

A lot of people drive a lot of miles which is why there are a lot of crashes. So you can argue either side of this depending on how you want to twist the numbers.

Auto crashes are the leading cause of death in some age ranges. You have to decide what you want to normalize to... other kinds of death is a good choice.

Reply on news.ycombinator.com