Tesla Autopilot Crash: Why We Should Worry About a Single Death
spectrum.ieee.org
spectrum.ieee.org
Article's logic is dumber than a sack of hair.
Who would say no to this?
Expect people to be highly resistant to that change at the beginning.
My experience with people who are afraid of planes is that they know planes are safer than cars, but none the less feel a visceral, non-rational fear of the airplane that makes flying deeply uncomfortable for them. Kind of like how you might feel if you were covered in insects, even if you knew that the insects were non-venomous, non-disease carrying, and clean.
I am all for self driving cars, but the common statistical argument (lower number of deaths than humans) for them is just silly for a number of reasons.
If you're going to go the utilitarian route, you need a more complex utility function. Otherwise you end up with really weird badness.
Unfortunately people prefer the current way of implementing things: pretend to make everyone happy because people can't think 5 seconds ahead of time or even imagine that their actions affect others.
It's so easy to sell everything for the small, immediate benefits while ignoring the bigger picture.
Other reasons here:
Which is a controversial position, but not stupid exactly. I would call it misguided, and certainly not a very utilitarian position. And given the options I personally would vote for the utilitarian answer: Save five people even if the one who dies wasn't otherwise threatened.
I think it would fail utterly in practice. The per-person incentives are all wrong. You're less likely to get picked if you ruin one of your kidneys, for example. "Donors" are going to be unwilling in general, and that will cause a lot of trouble and turmoil. Lots of protesting. Not to mention any abuse of the system would be kind of horrifying.
But the general idea of substituting "bad luck you got a disease" with "bad luck you lose the lottery", when there's only half as many lottery losers as there were disease losers... maybe that's not so bad. When it comes to trolley problems, you also have to imagine yourself on the track. Do nothing and there's a 5/6 chance you die. Throw the switch, and there's a 1/6 chance you die.
No, obviously I don't advocate that we go around arbitrarily murdering people to potentially improve the lives of others. But there's a huge difference between searching out and selecting specific people and killing them and simply changing the number and identity of people who will be killed by random events.
It's the difference between premeditated murder and a no-fault accident. Policy/law decisions must necessarily make choices like this, flipping the switch from tracks that typically have five or five hundred people standing on them to tracks that typically have one, for the good of the five or five hundred.
Like air bags that kill some people but save thousands more. Or vaccines that kill (a very small number) of kids (typically with allergies) but save millions.
The fundamental conceit that you can add and subtract lives, or happiness, or the like as though they were values in a formula is core to consequentialism, and I don't think that people are actually willing to follow through with the implications of that when it stops being at a safe remove.
I don't understand the bright line you're drawing between "murder someone by pointing a runaway trolley that would otherwise absolutely miss them in their direction" and "murder someone by harvesting their organs."
You can (and, I'd argue, should) believe that air bags are a good idea without believing that the reason that air bags are a good idea is that you can add and subtract lives. For example, you could say, "If YOU get into an accident, it is far more likely that an airbag will save you than it will kill you," which is a different thing than saying, "An air bag will kill YOU, but it will save two other people." And there are many other moral theories which would suggest that air bags are good for reasons other than that "I killed Bob but saved Alice and Charlie" is a generalizable rule for moral good.
The difference is key to understanding policy decisions like this.
You absolutely use math with lives when you're making policy. Right down to calculating lives with dollars.
If there is a fixed budget of $100M to apply to saving people's lives, you ideally will divvy the money around in such a way that the most people live. You don't extend one person's life by 1 month using extreme measures that cost $50M and let a hundred die who would could have had their lives extended by years for $50k each.
To some degree the original question is a real-world policy example of the trolley problem: Do we allow companies like Tesla to save thousands of lives, possibly your own, in exchange for indemnifying them against the (ideally) few lives lost (also potentially your own) when the technology fails? That sounds identical to the air bag policy: In both cases the life saved could be yours. You might not even get into an accident if there were enough self-driving cars, so not only would you live where you might have died, you might even avoid an inconvenience.
I'm not talking about seeking out people and killing them. That is committing murder for an alleged "greater good", and the ends don't justify the means. I'm talking about redirecting a trolley that is going to kill five random people and instead making it kill only one random person. And we can talk about focusing on further advances to make sure that one person doesn't die (statistically speaking) either, but I think that throwing the switch, in a policy sense, is always the right first step.
I think the moral argument that flipping the trolley switch is wrong is bankrupt, and that it does map onto the real world and real world policy decisions like requiring mandatory air bags. Flipping the switch isn't murder. Not flipping the switch when you have the opportunity is murder.
Each day when we drive we assume a certain degree of risk and implicitly acknowledge that we may be hurt. We make the decision to put ourselves in danger, whereas in the trolley problem we are led to assume that all parties are not responsible for their present situation. This distinction may be very important: if all parties in the trolley problem had entered into a covenant acknowledging that they may be killed, would more people opt to kill the one and save the five?
The logic is: "People are not able to drive. Thanks to X, they now are, but have probability Y of dying. Given that Y > 0, X should not be granted."
With X = driving license, and Y = human car fatalities, that logic concludes that driving licenses should not be granted.
The Trolley problem is an interesting ethical question because it is not a daily occurrence. This scenario, on the other hand, is.
To me, the critical factor is that the risk of death is equal or reduced statistically, which so far seems to be the case.
Another scenario, imagine you are driving on a freeway but a couple people decide to hop onto the freeway and cross it on foot. Would you rather just run them over or ram your own car and kill yourself just so that 2 people aren't harmed?
Do I want for him to face a 1/10000 chance of being killed each year by a human driver?
Or do I want for him to face a 1/20000 chance of being killed each year by an autonomous vehicle?
I'll take option two, for him and for me. I don't give a damn about philosophical questions. And I don't give a damn if it's a different dice I'm rolling.
The only reason I care much about this one death is because it risks slowing down progress. Lives are at stake here. Yours. Mine. Your family's. My family's.
The statistical/utilitarian argument for self-driving vehicles is less compelling than the personal story argument against them. So I want to make the for argument as personal and compelling as we can.
But autonomous vehicles will eventually be much, much safer than human drivers if they aren't already.
But there's a huge difference between a self-driving car and a novel drug. These cars are using statistical learning mechanisms -- it is impossible to prove that the car will always function correctly.
>Yes, it could be that autonomous cars will save many more lives than they take. But “the ends justify the means” is a dangerous approach in ethics, capable of justifying any evil as long as the math worked out.
I mean, yeah, "the ends justify the means" is a dangerous approach. This is a strawman: allowing less than perfect algorithms to transport us a little less than safely (though safer than currently) is not an instance of "the ends justify the means." It's an instance of pragmatic rationality in response to a very real and very common danger.
>If torturing innocent people or infringing on other rights can deliver some greater good, we would (or should) be deeply troubled by the choice.
The author is using an analogy between torture or general rights-violations as an argument against driverless technology! How does this seem reasonable?
There's one question I have for the author: what and who's rights are being violated by driverless technology that are not already being violated by human driving?
You would think that sending something to a machine on the Internet would imply that it would actually reach the recipient and that no one else is listening. Not only is that not true, it was essentially never intended to be true. We work very hard to try and make it so, but will probably never reach a point were it will be equal to what it could have been if that was the intention from the beginning.
https://medium.com/message/everything-is-broken-81e5f33a24e1 http://swiftonsecurity.tumblr.com/post/98675308034/a-story-a...
Sorry for the rant, for those that care for a reasoned argument along the lines of the above, Scott says it better than I ever could -- http://raikoth.net/consequentialism.html
Reason 1: Different people will die. Example, blind people who are currently not at risk (because they don't drive) will die in greater numbers.
And this is an argument against saving 500,000 lives every year? Really?
Reason 2: Humans are irrationally afraid of situations where they lose control. Example: flying in airplanes vs driving a car
And we're in favor of 500,000 additional people dying every year, because of people's irrational fears?
Reason 3: The ends don't justify the means. Example: Torture
Even if you accept the underlying premise that the ends don't justify the means, this argument only carries weight in situations where the "means" are considered evil. Like torture. Or robbery. What exactly is "evil" about allowing consumers to voluntarily purchase and use self-driving cars, and saving thousands of lives every year in the process?
I was surprised to see Musk's candid and blunt defense of self-driving cars. It's not PR savvy, but honestly, it's exactly what we need. The reasons being marshaled by people like this author, against self-driving cars, are horrendously bad. And while we sit here twiddling our thumbs, discussing human irrationality while sipping our lattes, hundreds of thousands of people are dying. Maybe it's time we stopped splitting hairs and started doing something.
Keep in mind total US road fatalities in the US is ~32k. I'm not sure how Elon got to 500k, but even if we handed out a free Tesla to every single American, and not a single person was ever killed in a Tesla, it's off by an order of magnitude. Of course Elon is quoting global fatalities, but road fatalities in 2nd & 3rd world countries could be improved with lots of basic infrastructure & existing car technology. It's simply not comparing apples to oranges, before you even start considering the unrealistic cost of a Tesla for those markets.
tl;dr; the 500k number is bullshit and any argument based on it is asinine.
Regarding the numbers behind the 500k:
- there were 1.25 million road deaths globally in 2013
- yes, this can be improved through seatbelts and safety standards, but it can also be improved by self-driving technology. This isn't an either-or problem. Having one doesn't preclude the other.
- self-driving cars aren't limited to Teslas. Any car manufacturer can integrate self-driving into their cars.
- there's no reason why self-driving technology won't penetrate the market and become mainstream, at some point in this century. This is obviously not going to happen overnight. The more we delay the initial roll out, the longer mainstream penetration will take.
In developing cancer drugs that could save millions of lives—just like robot cars are promised to do—we understand we can’t ignore problems in clinical and human trials. We can’t cut corners just because we want to rush a life-saving product to market.
“The perfect is the enemy of the good,” as famously declared by Voltaire, is also a common reaction to ethical critiques of autonomous cars. But this is a straw-man argument: no one is demanding perfection, just due diligence, especially if death is on the line. Just as with cancer drugs or anything else on the market, a product doesn’t have to be perfect, though that’s not an excuse to not be more careful.
Look at seatbelts as an iconic safety device: even they aren’t absolved of all sins, just because they save a lot of lives overall. Unlatch buttons that are too large (and can be accidentally bumped open) or too easily opened have sparked lawsuits and massive recalls. These aren’t really malfunctions but only bad designs, and bad designs can kill.
The extra care needed to avoid these problems doesn’t have to take a Herculean effort or stall research and development. It just means investing some time to think it through and properly set expectations. This could save lives, and every one counts. (Just ask their families.)
Just because this technology MAY[1] save lives in the future doesn't mean you can sacrifice others for greater good.
This is also the reason why we don't allow for experimenting on humans in the medical field.
[1] This is something that while I am fan of Tesla I'm still convinced Google is doing this properly i.e. build autonomous system from the scratch. What Tesla offers is an autonomous system that is good enough to make driver confident about it and pay less attention (e.g. watching Harry Potter movie while driving), but not fool proof that it would prevent accidents that regular driver could easily avoid.
Not the above poster, but no, I don't. One thing I'm NOT seeing in the articles (for- and against-) on vehicle automation is a cavalier attitude. (What I _have_ seen is a lot of "who do we sue" arguments that haven't worried about total human misery/safety, but we'll ignore that for the purposes of this argument and only consider those who HAVE considered human safety)
Can you clarify?
Nobody forced them to use the driverless features. The driver took a risk and died.
EDIT: newline after quote for formatting
The existing model where car manufacturers are basically free to add whatever safety features they want and eventually the government drafts standards when thing have settled down has worked very well so far in improving automobile safety.
While I'm a huge fan of vehicle automation, I have plenty of concern that it not be commonly used until it is in fact safer overall (I allow that come circumstances become less safe while others become more). I'm not sure what "cavalier" attitude you refer to? Where, exactly, are people claiming there is no concern for safety? What legislation has been passed that is seeing through rose-colored glasses?
Heck, the death referenced in the headline got lots of attention, both in criticism and in praise of automation, so I can see a lot of conclusions from that but "cavalier" isn't one of them. Looks to me like all sorts of caution is being considered, proclaimed, and hammered.
So what cavalier attitude is there that this article, with it's admittedly weak arguments, is good to be fighting?
I get the sense that a lot of people feel like since the intent is good and driver's are opting in that basically anything goes---which seems pretty cavalier to me.
Naming it something like "cruise-assist" or similar would remove so much confusion and expectation.
It's a wonderful car, with great features and one terrible UX flaw in labeling a core feature.
Have they even released this data for examination? I am thinking that many people have approached this feature with caution, in that its a novelty but they aren't "trusting" it or relying on it like you would a human driver to safely get you somewhere. It would be also interesting to see the types of roads and conditions where the feature is enabled. In the rain? Snow? At night? Poor visibility? Traffic conditions.
Just a single number paints an incomplete picture of the data. Why has no one mentioned this? People are sheep that are quick to declare victory after seeing a single number without questioning where or how that number came to be.
My thoughts are this: give this feature to your average Joe driving a Honda without much guidance and we'd have a lot more mechanized death.
Youtube is full of Tesla autopilot fail videos where the driver had to take quick action to avert disaster. What if we let this into the hands of a lot more irresponsible people who think they can tool down the highway watching a Harry Potter movie, blind faith and trust in a "feature". How many Tesla owners here would honestly trust the feature that much with their life at this point?
2014 stats[0] show that 38% of fatalities in car accidents were not drivers. It's reasonable to assume that even more people who were driving were not at fault. Those deaths are arguably near-random in most cases. The argument that hundreds of thousands of random deaths are somehow more "fair" is an entirely poor argument in my opinion.
[0] http://www.iihs.org/iihs/topics/t/general-statistics/fatalit...
The short answer is that yes, it's generally agreed to make the active choice to save more people, even if it means different people die.
Air bags kill people sometimes in accidents where they likely wouldn't have died, but we put up with the risk because overall we're more likely to live through a serious accident. Air bags are fired off because a computer reading a sensor decides that they're needed. It's a very close analog situation, and yet the people making the moral argument against them have been downvoted to oblivion, and air bags are now mandatory.
Even if this is a trolley problem, if we look at the problem from the point of the regulators, then the situation could be considered the trolley problem turned on its head. Regulators can do nothing, and let Tesla develop their autopilot and save lots of lives. Or regulators can actively intervene, stop Tesla, and thousands more will die. From this point of view, the trolley is headed towards one man and you have a switch in hand to kill 5 people instead. It's hard to imagine that anyone would choose to flip the switch.
Fair enough. Like your analogy.
It actually is possible. It is a common mistake to confuse statistical average and median. For instance, 75% of the following numbers (1, 1, 1, 0) are above average. More than 90% of humans have an above average number of legs.
Sure, it is difficult for the general public to understand academic results. Still, level zero of academic reporting is to link to the original paper, instead of playing a game of Chinese whispers by linking to an article that links to an article that links to the paper.
What the paper really points to is that people assume by default that strangers are worse at driving than they are, which ironically is what is happening with autopilot instead of strangers.
The metric to compare both alternatives then becomes, how many miles of texting while driving vs. being driven by an autopilot while texting does it take to kill one person. I don't have the numbers, but I'm pretty sure the latter is lower. Maybe autopilots encourage "bad" behavior, but convenience also spurred construction of coal power plants at the cost of shortened lives. Much like fines for neglected pollution abatement, I would favor mandatory life insurance for each mile on autopilot to internalize risks.
I actually would welcome the day we have autonomous "self-driving cars" so I could tell it to go to the shop for service/inspection and drive itself back, but if I'm in the car I will always be the driver.
At some point computers will be safer drivers than human driven cars in every way. Are you going to insist on irrationally putting your fellow man in harms way at that point?
The Tesla crash makes it easy to imagine a 100 car freeway pile-ups caused by AI drivers discovering novel ways to crash. Driving headlong into the side of a trailer because the color looked like the sky?
The possibilities are endless, and the subsequent media attention and reactionary legislation could set back the whole project back a generation.
I won't claim that that never happens, but I doubt it's very common. If you were to take 1000 random car crash deaths I do not think a very large proportion of them are caused by people running head-on into a sideways tractor trailer that they had a long distance to notice and react to.
Tesla says this is the only fatality in 130 million miles of Autopilot driving, and that in the US there is a fatality every 94 million miles of driving overall. But it's known that fatalities are more common in risky driving scenarios where Autopilot is not used, so this is not an apples-to-apples comparison (https://www.progressive.com/newsroom/article/2002/may/fivemi..., http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1449863/). If the appropriate adjustment is 40% or more, then the Tesla is riskier than human driving in the "non-risky" scenarios. The fact that the accident here was an uncommon failure mode for humans, and the Tesla was at fault, should give a hint as to which direction the numbers need to be adjusted.
The distraction is especially bad since most of the time the feature works well so with time it makes you less careful.
If we assume a death is binomially distributed, 1 death in 100 million kilometers is consistent with a real rate of anywhere between 0.03 and 3.7 deaths per 100 million kilometers. That's a big bracket.
Tesla is not safe car, it is as safe as Honda Civic, with ~40 deaths per million cars. Do the math if you don't believe me.
As far as doing the math, are we counting deaths like the guy who stole one and crashed it going 120MPH without a seat belt?