Hacker News Reader

Comment by prepend | Hacker News Reader

Scaling those numbers is not useful and in fact reduces usefulness.

Comically, that’s why OP said not to do that.

Comparing dissimilar things is actually worse than not comparing at all since it will increase the likelihood of some decision resulting from the false comparison.

View on HN

The goal is to use the best set of information available to us. I merely cited the normalized numbers because it's been asked various times in this thread - questions along the lines of "how does this rate compare with human drivers?"

The purpose of the extrapolation was to get a (flawed) approximation to that answer. By itself, it doesn't say much, but all we can do is parse the data points available to us:

- Uber's death rate after approximately 3 million self-driven miles is significantly higher than the national average, and probably comparable to drunk drivers.

- Public reporting around the Uber's self-driving program suggests a myriad of egregious issues - such as running red lights.

- The company has not obeyed self-driving regulations in the past, in part because they were unwilling to report "disengagements" to the public record.

- The company has a history of an outlier level of negligence and recklessness in other areas - for example, sexual harassment.

But this is precisely why you should simply extrapolate. Of course people ask, and of course the answer will be useful. But extrapolating one figure of 3M miles to a typical measure (per 100M) is not useful because it provides no actionable information.

Providing this likely wrong number anchors a value in people’s minds.

It’s actually worse than saying “we don’t know the rate compared to human drivers because there’s not enough miles driven.”

Your other points are valid but don’t excuse poor data methods hygiene.

Even now you are making baseless data on its face because you don’t know the human fatality rate per 3M enough to say is “significantly higher.” Although I think it’s easier to find enough data from the human driver data to match similar samples to Uber. But dividing by 33 is not sufficient to support your statement.

I haven’t seen data on the public reporting. That seems interesting and would appreciate it if you can link to it.

> the self-driving car was, in fact, driving itself when it barreled through the red light, according to two Uber employees, who spoke on the condition of anonymity because they signed nondisclosure agreements with the company, and internal Uber documents viewed by The New York Times. All told, the mapping programs used by Uber’s cars failed to recognize six traffic lights in the San Francisco area. “In this case, the car went through a red light,” the documents said.

https://www.nytimes.com/2017/02/24/technology/anthony-levand...

It depends on what question you're trying to answer with the data (however incomplete one might view it).

Is the data sufficient to say if Uber might eventually arrive at a usable self driving vehicle. Plainly no. It's not sufficient to answer this question one way or another.

Is the data sufficient to indicate if Uber is responsible enough to operate an automated test vehicle program on public roads. Maybe.

There still needs to be an investigation of cause, but if the cause is in a autopilot failure, or the testing protocols preventing a failing autopilot from harming the public, then the question is what the remedy should be.

I agree there should be a investigation.

I agree that you have to use data available to make the best decision possible.

There may be methods to account with all of the problems of comparing two different measures, but it requires a lot of explanation.

But extrapolating one measure into another is wrong without those caveats. That’s the comment I replied to. So in no situation would the method I replied to be useful for what reasonable question is asked.

I think it's very relevant. If the testing protocols are insufficient to prevent an avoidable accident within an outer bound of accident rates. If it is a clear data point outside those bounds (even with uncertainty) one could make a case to severely limit or ban Uber's testing on public roads, and require that they demonstrate sufficient maturity of testing procedures and data to be allowed back onto the roads. This as opposed for waiting for another 'data point' (death).

Reply on news.ycombinator.com