Moral Machine

If the only choices are "plow through a bunch of pedestrians, killing them" and "swerve into a fixed obstacle, killing your passengers", your self-driving car has already made an immoral choice to drive too fast for the conditions.

The correct choice was "if there are hazards in the roadway limiting your maneuvering options, pedestrians (or other objects) that might suddenly enter the roadway or visual blockages that would prevent the car from determining whether there are people/object that might suddenly enter the roadway - slow down before the car's choices are so limited that they all involve unnecessary death".

A self driving car that encounters any of these scenarios has already failed at being a safe driver.

The breakdown says a lot about the law, my gender preferences, age preferences, fitness preferences, literally things I absolutely did not take into account. It is a bad methodology because it assumes the reasoning behind my decision is as complicated as the game itself.

I know drivers who will blast around blind turns, potentially putting them in these situations where they'd say "there was no time to react!", when they should be slowing down just in case there's someone they'll need to avoid. It's why we talk about defensive driving.

If the brakes are broken often enough for these kinds of dilemmas to even matter, you've failed to avoid the situation in the first place, and need to recall the product to add more redundant systems. Instead of agonizing over what to do in a situation that comes up 1% of the time, turn that 1% into 0.000001%.

In the real world I have a hard time imagining a scenario where a case like this would actually come up.

Just look at real accidents and you know what people do. They don't do moral decisions, they just act without thinking. If they even notice in time, they brake the car sharply and avoid the immediate obstacle.

Then think about improving the results.

These questions and their answers are highly dependent on cultural contexts. They are behavioral research not ethics.

The only choice where I wouldn't apply morality would be preferring the lives of the poor compared to the wealthy, since risks on the wealthy make the whole system safer. To be fully pedantic, I'd include the passengers based on wealth in this equation.

I guess that's what it feels like to be a bolshevik.

It is not making judgement about pedestrians with canes, who are pregnant or veterans, or who are an underrepresented class. To do so is to design the wrong machine.

The correct design is to either safely swerve to avoid hitting anything, or to hit the brakes. There's nothing else. As soon as you start doing "something else" you open a horrible, horrible can of worms. The even more correct design is to drive safely so that you never encounter this failure mode in the first place.

So, this website is a critique of a system that does not and should not exist.

I also chose to hit the dogs instead of people but the results said I was neutral on that? It appears I also favored fit people... On my phone I couldn't even tell that some were supposed to appear fit/fat etc. I just saw kids, adults, elderly

1) Save humans at the cost of animals

2) If both options involve killing humans, always prefer to do nothing (continue straight ahead). That way it's the failed brakes that killed them, rather than the car's "decision". I know not making a decision is also making a decision, but a passive decision is not on the same level as an active one.

No difference between who the people are, whether they're passengers or not, or even how many.

Software and legal codes are going to collide in interesting ways here.

Two, it's a completely useless survey because the cases are heavily biased. (I.e. if you always select 'swerve', you seem to be much more biased to kill women. That's not the most solid survey design)

https://robots.law.miami.edu/2019/wp-content/uploads/2019/03...

Seriously, why are we so enamoured of vox pop that we think $random_internet_user is a good way to make difficult moral choices?

There's an underlying assumption in these examples that we'll be in a car that can tell if someone is homeless, or that knows the net worth of the person driving it. And those scenarios are kind of absurd, why would a car be able to do that? A car that can in milliseconds identify whether or not someone is an executive or whether they're breaking the law should never be getting an accident in the first place, it should be able to identify way ahead of time when someone is about to cross the street. But we come up with these really ridiculous premises that are fueled by simultaneously an overestimation of what technology is capable, a lack of thought about the implications of the technology we're describing, and an under-awareness of the real challenges we're currently facing with those technologies.

An analogy I think I used once is that we could have the same exact conversations about whether or not Alexa should report a home burglary if the burglar is stealing to feed their family and Alexa knows the family is out that night. It's the same kind of absurd question, except we understand that Alexa is not really capable of even getting the information necessary to make those decisions, and that by the time Alexa could make these decisions we'd have much larger problems with the device to worry about. But there was real debate at one point about whether we could put self-driving cars on the roads before we solved these questions. Happily, in contrast, nobody argues that we should stop selling Alexa devices until as a society we decide whether or not justified theft ever exists. And it turns out the actual threats from devices like Alexa are not whether or not machines believe in justified theft, it turns out the actual threats are the surveillance, bugs, market forces, and direct manipulation possible from even just having an always-on listening device at all.

The danger of an Alexa device is not that it might have a different moral code than you about how to respond to philosophical situations, the danger is that Amazon might listen to a bunch of your conversations and then accidentally leak them to someone else.

So with self driving cars it's mostly the same: the correct answer to all of these questions is that it would be wildly immoral to build a car that can make these determinations in the first place, not because of the philosophical implications but because why the heck would you ever, ever build a car that by design can visually determine what race someone is or how much money someone makes? Why would that be functionality that a car has?

We have actual concerns about classism and racism in AI; they're not philosophical questions about morality, they're implicit bias in algorithms used in sentencing and credit ratings, fueled by the very attitude these sites propagate that any even near-future technology is capable of determining this kind of nuance about anything. The threat of algorithms are that people today believe that they are objective enough and smart enough to make these determinations -- that judges/lenders look at the results of sentencing/credit algorithms and assume they're somehow objective or smart just because they came from a computer. But I remember so clearly a time when this was one of the most common debates I saw about self-driving technology, and the whole conversation feels so naive today.

I'll take another tack: I believe it is a category error to ask humans to determine the moral or "most moral" action in these scenarios. There are two sufficient conditions for this:

1. There is no present, efficient moral agent. A self-driving car, no matter how "smart" (i.e., proficient at self-driving) it is, is not a moral agent: it is not capable of obeying a self-derived legislation, nor does it have an autonomous morality. Asking the self-driving car to do the right thing is like asking the puppy-kicking machine to do the right thing.

2. The scenario is not one where an efficient moral action occurs. The efficient moral action is really a complex of actions, tracing from the decision to design a "self-driving" car in the first place to the decision to put it in the street, knowing full well that it isn't an agent in its own right. That is an immoral action, and it's the only relevant one.

As such, all we humans can do in these scenarios is grasp towards the decision we think is most "preferable," where "preferable" comes down to a bunch of confabulating sentiments (age, weight, "social value", how much gore we have to witness, etc.). But that's not a moral decision on our part: the moral decision was made long before the scenario was entered.