Hacker News Reader

Comment by JohnJamesRambo | Hacker News Reader

"Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States."

Wow there goes that "safer than human drivers" argument.

View on HN

With one data point, you can't extrapolate much. This is misuse of statistics.

Consider if there was a new lottery and you weren't sure what the odds of winning were. You play it three weeks in a row and the third time you win a million dollars. Conveniently, no one else tries the new lottery yet.

Does it follow then that the odds of winning a million dollars are 1 in 3? Or should you play it a few more times before you declare to all that one in three plays will make one a millionaire?

They did not sample three times and win once. They sampled three million times and won once. Hardly "one data point."

I believe “one data point” referred to one winning, not three plays.

It still falls under "one swallow does not make a summer" - even though a lot of days of winter preceeded that bird sighting.

That is one datapoint - one death.

The point remains: it isn't fair to decide what the probability is with only one data point on one side- especially for rare events.

Would it have been fair if Uber last week were to declare that they have a 0% probability of pedestrian deaths, since they'd never had one yet?

The goal of these statistics is to predict future outcomes. But with such a small data sample, you cannot fairly predict the future- just as in my lottery example.

It would be fair to approximate it was near zero per however number of miles they drove, just as it is fair to approximate it is one per three million miles today. Think of it the other way, we know with a high degree of certainty the fatalities are not one per mile, for example.

What if Uber’s first self-driving car killed a cyclist in its very first mile of operation? Would you find it equally hard to draw conclusions?

If you do want to extrapolate from their data, it would be worth looking at the total number of accidents--not just fatalities. Insurance companies say people file an accident claim every 18 years on average. If the average miles someone drives each year is 12,000 miles, this means they get in an accident every 216,000 miles on average. If Uber drove 3 million miles, we should expect them to have been involved in about 14 accidents over those 3 million miles if the cars are on par with humans.

(I'm guestimating on some of those numbers, but the should be somewhere in the ballpark.)

You'd also need to account for disengagements. Some portion of all disengagements were likely to avoid accidents.

Good point. Although I think many of the disengagements are basically the car saying "I don't know what to do safely" and without a driver to take over it would simply pull over and stop.

One data point? So if Uber had driven a billion miles with zero deaths, they'd have zero data?

If you're looking for the rate at which something happens, for the purpose of predicting events in the future, you need to see the event occur many times before you can fairly estimate it's rate. That's what my example means.

I'm not defending Uber here- I'm defending statistics!

With more data, we may discover that Uber cars are 100X worse, not just 25X. Or we may discover they're better. But we don't have the statistical power to make that estimate when we've only seen the event happen once.

Your knowledge of statistics could use some enhancement. The response referring to poisson processes is a better view of things.

> Your knowledge of statistics could use some enhancement.

In this we agree.

With 0 accidents you cannot accurately measure the rate, but you can estimate an upper bound on it for whatever probability of being wrong you're willing to accept.

One data point might not be enough, but two may be too many for the industry to bear.

It only took two incidents to shut down the Concorde program.

It should be possible to create a Bayesian model of the posterior distribution of fatalities at this point. That distribution will be pretty broad, and not Gaussian, so talking about the mean is somewhat meaningless. Nonetheless, you could certainly compare that distribution with the posterior for human-driven cars and draw a conclusion like: “it is Xx% likely that the Uber fatality rate is at least twice that of a human driver.”

One accident is clearly not one data point. If Uber had driven a billion miles with 0 accidents, we would safely conclude they were safer than human drivers with "0 data points".

Assuming that accidents are independent, we can model this as a Poisson point process. If the accident rate is 1 per 100M miles and Uber has driven 3M miles, the probability of there being zero accidents in that time is P{n=0} = ((λt)^n / n!) * e^(-λt), where λ=1/100M and t=3M. Doing the math, it seems that's 97.04%.

So, yes. It is possible that Uber's accident rate is 1 in 100 million. If so, this incident would fall in that remaining 3%. It's unlikely, but possible.

Gil Pratt, who heads up Toyota's autonomous development initiative mentioned that we would need to drive 8.5 billion autonomous miles to be able to declare with 99% statistical certainty that autonomous vehicles are safer that human driven ones. Of course, as we are witnessing, great pains will be taken with every preventable injurious or fatal collision to ensure that sort of failure never happens again, so by the time we get to the 8 billionth mile the software and hardware will have improved considerably, rendering the early data moot.

One thing we can say about the woman killed the other day by an autonomous Uber, is that unlike the other ~40,000 killed on America's roads over the past year, her's was not in vain.

Every day that we delay the widespread deployment of this technology, it's another 100 or so people dead. Of course, the public is unlikely to see it that way. They see one death as a tragedy, but 40,000 is just a statistic, business as usual, nothing to get excited about.

Maybe her death wasn't in vain, but it was definitely avoidable. If Uber rushes out half-baked driverless cars, fallout from the incidents they're responsible for will cause serious delays to widespread deployment.

Trading lives to save on R&D time would violate professional codes of ethics in literally any other industry.

It wouldn't violate professional codes of ethics in a war, and we're taking wartime casualty numbers on our roads everyday.

This is the real trolly problem when it comes to the ethics of developing self driving cars.

Her death was in vain. These kind of fundamental scenarios can be practised with dolls or stunt men on closed test tracks not on public roads ...

If Uber needs data they could have driven manually. Obvously their obstacle tracking is bad.

I don't really understand a whole of similar comments. It's as if Uber and all other autonomous driving pursuer is doing so for the betterment of life for everyone and so their laziness in making sure their tech is good enough can be excused. The car in question failed spectacularly and the response is her death was not in vain?

Uber is doing this for money, so is all the other companies even if there are some potential huge collateral benefit for the human race from that. It's definitely not the goals of the companies regardless of any PR talk. So when you gamble with people's life for money and fame you should go to jail for a long time executives or engineers alike.

The statistics are just being used to sustain corporate greed in my mind and we should not let them. Self driving cars has lots of potential to save life, so are other techs. Does't mean that all sense of responsibility and ethics goes away just because of the potential.

The pharmaceutical industry tries to save lives, for money. They're driven by greed. They also have a long track record of fucking up big time and people dying because of it. Does this mean we should stop giving people medicine? Would letting people get sick and die be preferable than allowing imperfect industries with a profit motive try and save them?

That's a strawman argument... These driverless car tests are more like early phase clinical trials. Clinical trials are conducted under very tight controls, using participants who have given informed consent. Drugs don't get licensed until they can prove efficacy and safety and are approved by the FDA.

The current procedures for clinical trials are the result of decades of experience, where mistakes (and yes, occasionally shortcuts driven by greed) did result in avoidable deaths of trial participants.

For an example of how the pharma industry deals with this 'safety first' versus 'stifling innovation' dilemma, read this article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1526936/

The inevitable result of this crash is driverless vehicle testing will get regulated more like drug trials...

That’s not really an apples-to-apples comparison. In the case of the pharmaceutical company, people don’t really have any other choice other than to take experimental medicine for terminal stage diseases.

On the other hand, there are widely used, cheap, and efficient alternative to self-driving cars on the roads today: human drivers, public transport, carpools, etc.

If you want to make analogies, I think the self-driving car accident is more like if an elevator company accidentally crashed an experimental high-speed elevator in a shopping mall. I think it would be grossly negligent on the part of the company to test such an unproven device on the general public, especially if people’s lives are being put in a position of risk which they did not to worry about before.

What is currenly being done with self-driving car testing on public roads is basically like a pharmaceutical company mixing a new experimental drug into the dishes of random people at a restaurant, which neither are compensated in any way, nor did they have a chance to decline their involvement in the test.

Such a thing would be entirely unthinkable in the pharmaceutical industry of today. So if this comparison suggests anything, it is to much more strictly regulate self-driving car development and testing!

Other big players have taken great pains to ensure that it wouldn’t happen in the first place

Uber should be banned from doing AI research on public roads

It is not a misuse of statistics for "one data point" to significantly shift our beliefs. Let's do the math.

Bayesian approach: To make the math really simple, let's assume a discrete prior on Uber's death rate. Say 33% that Uber's cars are much safer than humans (0.1 deaths per 100M miles), 33% that they are equally safe (1 death per 100M miles), and 33% that they are much more dangerous (10 deaths per 100M miles). After observing one death at 3 million miles, your posterior is should update to {safer: 1%, equal: 11%, more dangerous: 88%). This is a substantial shift in confidence.

Math: http://www.wolframalpha.com/input/?i=(1-10%2F100)%5E2*(10%2F...)

Frequentist approach: Let the null hypothesis be that Uber's self-driving cars have the same death rate as humans - 1 death per 100 million miles. The odds of Uber killing someone within 3 million miles is about 3%. Therefore, we can reject the null hypothesis with a p value of 0.03. One positive "data point" is statistically significant.

Statistically, one death after 3 million miles is not proof that Uber's death rate is higher than 1 in 100 million miles. But it is statistically significant, in both a frequentist and Bayesian framework. You have to get really, really unlucky to have a death at 3 million miles if your death rate is 1 per 100 million miles.

Bottom line: This collision isn't proof, but it's strong evidence. (To go along with all the evidence from crash rates, disengagement rates, engineers working at these companies, and the video of the crash itself.)

There’s more than one data point showing that Uber should not be allowed near anything as safety critical as self driving software

They let a car on the road that couldn’t even stop at a red light ffs

https://www.theverge.com/2017/2/25/14737374/uber-self-drivin...

The fact that they were allowed to deploy in Arizona after this is really regrettable

And it’s totally unsurprising that Uber “got the first kill”

Driverless cars are also being tested in relatively nice driving conditions. People, on the other hand, drive in all sorts of conditions. X deaths per Y easy driving miles is going to translate to many more than X deaths per Y representative driving miles.

>Driverless cars are also being tested in relatively nice driving conditions. People, on the other hand, drive in all sorts of conditions.

I would add that people represent (as a whole) various kind of drivers, including the non-expert/fresh licensed, the elderly, the too sure of themselves, those speeding or not respecting road signals (or driving in connection with a crime), the emotionally fragile, those under the effect of alcohol or drugs, those tired by not having had enough sleep, etc., etc.

The "model" for an autonomous vehicle should be instead a particular "perfect" subset, ideally it should replicate the behaviour of an hypothetical extremely prudent, healthy 30-something with some experience with safe driving, very familiar with the way his/her car handles, having had a nice night sleep, no use of alcohol or drugs, without any external pressure (to arrive in time, to do other chores apart driving) without a cellphone or tablet distracting him/her.

If we could find this latter kind of drivers (should they actually exist) and isolate from the statistics the amount of incidents they caused, that would be the reference benchmark.

I think focusing on the safety statistics is somewhat of a red herring. Uber wins when you get hung up on how well its self-driving cars drive, because that's something that they can improve. Instead, I think we should focus on the fact that these dangerous machines are being operated by chronically irresponsible companies and because cars in general have issues, not because we expect them to be less safe than human drivers.

Any comparison between self-driven miles and typical human-driven miles has to take into account all the times a safety driver took over driving to prevent an accident. Those self-driven miles have a huge asterisk.

This is a good point.

Taking it a step farther, I’d expect that road conditions in the Uber tests so far have been more benign than average city driving. Less bad weather, straighter roads, etc.

It's much, much worse than that, since that 37,461 deaths number includes all deaths, including motorcycle/Truck/SUV deaths, which have higher death rates than passenger cars, perhaps 5x-10x higher.

A proper comparison in this case is comparing passenger car death rates.

And then you need to factor in other conditions, such as the fact that weather was clean, and that you should be looking to compare pedestrian/bicyclist deaths, and you see that this incident already throws out wack the death rate for autonomous vehicles.

Given exponentially-distributed distance between fatalities, this would have a 3% chance of happening if Uber cars were as safe as humans. So it's unlikely.

Well maybe not currently safer than human drivers. I don't think any sound-minded person would claim that they will never be.

Currently we're losing a million lives a year (about a 9/11 per day) to auto accidents. I think there's a serious argument that it's ethical to push autonomous driving onto the road before they're safer than humans if that would help them debug the software faster and advance to superhuman safety levels even a day sooner.

Why would you want to increase the death rate by pushing more autonomous vehicles onto the road?

Because doing so will cause issues (like this one) to be found and fixed faster. When enough bugs are fixed that the autonomous drivers are safer than the meatbag drivers, then the death rate will start to decrease. The argument is that you can trade more deaths today for a sooner decrease later. Under certain assumptions, you'll save net lives.

Why is the FDA so strict about human trials? We could save net lives if we just did drug tests faster to figure out what works. /s

It's slightly different though. Even if FDA was lax, the participants would be volunteers. This victim though didn't have any say in joining Uber's trial.

You jest, but the comparison is that we do drug tests at all, which we of course do. It's the same calculus. The math works out more in favor of being more aggressive in our pursuit of driverless vehicles as they kill more prime-age, otherwise healthy people.

This argument assumes that autonomous drivers will be safer than human drivers. Which is an unproven proposition. It is possible that autonomous driving may never be safer, or that the rate of improvement may be much slower than expected. How many "more deaths today" do you think are reasonable to test this question? One? ten? one thousand? ten thousand?

Yea, it does assume that. I happen to think that's true, but it's a separate argument, and one worth having. Humans are really quite bad at driving, so I think a conservative estimate is that autonomous vehicles could cut road deaths by at least 90%.

Cars are killing by the millions now, so do the math, and I'm willing to tolerate a lot of deaths in the process, contingent on the deaths actually helping researchers and engineers reduce future faults. I would happily take a million deaths right now if it meant driverless tech instantly became available to everyone, but I think there's an upper limit to how many accidents can be realistically examined at once.

What's the alternative position? That you're for more road deaths in total? Are you also the kind of person who wouldn't pull the lever in the trolley problem?

> What's the alternative position?

Developing and testing the technology in actual test scenarios (build entire testing cities and fill them up with paid stuntmen and -women, if necessary) instead of in public, until you're able to prove that the tech is statistically at least as safe as human drivers. After that, you can continue testing in public, provided that you deal with any changes to your system responsibly and ensure that the public is not exposed to additional danger because of your testing.

This is actually very common in the software development world. For production-critical systems, companies go to great length of creating a staging environment as realistically as possible, but fully separated from the production systems. You may get around that effort if the only damage you can potentially do is people not being able to see stupid cat pictures on a social network. But sorry, for life-critical tech, the move-fast-and-break-things approach is irresponsible bullshit.

> I would happily take a million deaths right now if it meant driverless tech instantly became available to everyone

What if your own death is guaranteed to be among them? Still "happily taking" it?

Guaranteed, no. But I'd take a 1 million divided by 7 billion chance. It's roughly my odds of dying this year in an auto accident anyways.

> I'm willing to tolerate a lot of deaths in the process, contingent on the deaths actually helping researchers and engineers reduce future faults. I would happily take a million deaths right now if it meant driverless tech instantly became available to everyone

For the sake of the discussion, let us assume that this is hyperbole and that "happily" was an infelicitous word choice. That said it raises a few questions:

- Uber and the other companies presumably are developing this technology for their own private benefit and do not plan to make it "available to everyone". In which case, how many deaths are acceptable?

- Should they be successful and succeed in developing a safe autonomous driving system should they be compelled to make it "available to everyone"?

- Are these million doomed citizens volunteers with informed consent or are they to be struck down unawares at random?

- What if the technology proves more difficult to develop than you anticipate? Suppose that after one million are sacrificed for the greater good it is improved but not yet good enough. Should we then continue with another million or should we abandon the project after only one million fruitless deaths?

- Assuming success, should we then ban human driven vehicles completely?

[edited for formating]

You need to have SERIOUS evidence for the claim that autonomous driving will actually be safer than human driving one day to even attempt to make this argument of yours. "Faith in technology" is not "evidence"!

If your faith is strong enough, you'll surely be welcome if you sign away all of your rights and serve as irrational traffic participant in a city where self-driving cars are tested. You could run through the city all day and produce difficult safety-critical situations by suddenly walking in front of them.

My faith isn't strong enough for doing that, so I expect lawmakers to protect me from companies like Uber who apparently think it's okay to basically make me their guinea pig in their public experiments. But I'll applaud you if you decide to take the risk on yourself for the (potential, unproven) advancement of the human race!

I'm already doing it all day for human drivers to no benefit at all.

I am pretty convinced that computers have the potential to outperform humans in this, but I'm not really interested in hashing out the debate here; the arguments have all been made more elegantly elsewhere. Though I don't know why you keep ascribing 'faith' to me. I never brought that dirty word up. I reasoned my way into my positions.

> Though I don't know why you keep ascribing 'faith' to me. I never brought that dirty word up. I reasoned my way into my positions.

In that case, what do you base your assumption on that self-driving cars will be able to deal with the complexities of todays' traffic in a significantly safer way than humans? Because even though you might not bring that word up, if you just assume that to be the case because you expect technology will one day be able to make that wish come true, that IS "faith" in technology. Whether you like that word or not doesn't matter.

So you're volunteering to be an unpredictable pedestrian or cyclist around large numbers of early-stage driverless cars? Put your ethics into practice!

The rationale response for individual cities is to say: "Do your risky testing elsewhere, thank you very much. You may come back here once you are as safe as our average driver."

I'm already a pedestrian surrounded by unsafe human drivers, and every day I hate what I see. What I'm talking about is making a tradeoff between risk now and risk later such that my total risk is lower. I'd take that tradeoff, and I think it's the ethical thing for society to do. What you're talking about is NIMBY, which certainly makes sense locally -- better for some other locale to perfect the technology -- but on the societal level I think it's wrongheaded. But don't worry; there's no danger anybody will take my stance on it seriously. Autonomous vehicles will not be widely deployed until they're provably drastically better than humans.

The rational response is to try passing legislation that prevents companies like Uber from (ever?) testing on public roads while allowing responsible companies like Cruise and Waymo a way to prove their competence and be allowed to test/operate.

What do you base this claim on? It sounds more like a statement of faith.

> I believe with perfect faith in the coming of the Messiah; and even though he may tarry, nonetheless, I wait every day for his coming.

You can't just look at Uber to make a sweeping conclusion of all autonomous cars on the road. How many miles has Tesla, VW, Volvo, Waymo, Google, Ford, and Apple have?

I don't know how to say it kindly but there is a difference between "what" type of people the car killed. If the fatality was another rule abiding driver on the road or a pedestrian crossing at a crosswalk that would be really bad. However if it was someone not following the safety rules by j walking, then that person accepted upon themselves a higher probability of being in an accident. When making laws, for the most part, they are for the benefit of law abiding people.

This is a ridiculous statement.

/s ?

It's a pretty unfair comparison, with 1 death on one side and over 30k on the other...

No because the rate is per million miles driven. It allows us to compare two different things. The sample size is one but it isn't looking great so far.

Reply on news.ycombinator.com