Self-Driving Mercedes Will Sacrifice Pedestrians to Save the Driver
fastcompany.com
fastcompany.com
Consider driving along an undivided highway. An approaching car veers into your lane. You definitely want to swerve, rather than brake and risk being hit head-on.
Of course, we can contrive a situation on top of that, like a group ride of cyclists to your right, so swerving will take them out...
But that is besides the point. I am just saying that there are cases where braking is not the safest thing for the occupants of the car.
So the decision to swerve is the trolley problem either way. You know the cyclists are there, and swerve because the occupant is valued more. Or the car doesn't know if cyclists are there, swerve and these possible/virtual cyclists are sacrificed for the occupant of the vehicle.
The person I was replying to said this is nonsense, as in a car you can just brake and nobody gets killed. I explained that there is a real-world situation where braking is not an escape, it is—as you say—equivalent to directing the trolley to the line where there is one victim.
We agree that the trolley problem in the abstract is about making a choice as to who is harmed. I was just pointing out that car brakes do not invalidate the concern.
I've predicted for years now that the first day of automated driving will have to solve the trolley car problem. I can only see one way out: we make a rule e.g. automated cars cannot leave the roadway.
Agreed it shouldn't be left to the companies - that's why the 'make a rule' idea. Else the companies are in an escalating war of ad-hoc behaviors to preserve their customer vs all others.
Braking is not instantaneous at any speed. You will still continue to travel at least some distance while you decelerate.
Your comment makes sense if you are trying to prevent a collision with another moving vehicle in front of you, you can make sure you have enough braking distance to come to a complete stop if the other car does as well.
What if you're doing 50km/h with a pedestrian on the sidewalk 10 metres ahead of you to your right? If the pedestrian steps or falls out suddenly into the road you won't be able to avoid hitting them without swerving, most likely into the adjacent lane. That lane might have another car or a gaggle of nuns tied up by an ethics professor in it already that you will have to contact in avoiding the pedestrian.
If you maintain a safe braking distance between your vehicle and all objects, even ones you don't expect to enter your path, you would be unable to travel above 0km/h in most situations.
Artificial Intelligence is a tool. Just like anyone who uses a hammer, backhoe or dump truck is responsible for what happens when they are using it, the same is true for AI.
As a tool, all AI systems should be designed to always do the most harm to the user of the AI first. It should be embedded into every autopilot-like system, and users should be aware of that choice. It's the only moral and ethically correct solution.
Let's say your AI-controlled car is driving at speed and turning the corner, there suddenly appears a little girl in the road in front of you. It can either swerve into a wall or run down the girl. A human might not be able to make the decision in time, but an AI system would. It needs to be programmed to always hit the wall.
Though this might seem extreme, the opposite is completely immoral. If it was a human driving, one could assume bad luck to be put in that situation: aka innocent until proven guilty. But for an AI system, we have to assume the opposite: The AI should always be presumed to be faulty. Therefore the user of that AI is culpable of putting it in that situation.
And just like any other tool, the manufacturer of that tool is legally responsible for its quality and reliability.
The opposite of the above means we'll all be riding around in autonomous tanks, aggressively maneuvering around each other at higher and higher speeds, pedestrians and others be damned.
https://www.bussgeldkatalog-mpu.de/bussgeld-wAssets/img/verk...
These example strive me as fairly contrived however. In traffic there simply is no time to make complex moral decisions. In the fraction of a second between realizing that an accident is imminent and impact, people just react (more or less skillfully and fortuitous, depending on many circumstances, including training and sheer luck).
Perhaps that's actually the way out for developers of such autonomous vehicles: just focus on what's ahead and if there's danger and braking won't be effective, swerve. If then someone is hit, it is an accident (due to incomplete information), rather than a (planned) sacrifice. Ignorance is a bliss.
But moving forward into a world where every vehicle is autonomous, we definitely want to do better than the status quo. Debates like this shape our definition of "better."
- brand A saves who it decides.
- brand B saves the occupant.
which would you buy?
Also, I can't help but think of the brand A airplane that knew better and crashed [1] and the recent brand B airplane that in one case it thought it did and crashed [2]
How is this quote relevant? THe point of the trolley problem is that you face a clear choice between who dies. Is Mercedes willfully programming its cars to understand the mortality of the car's occupants and nobody else's?
That said, I don't expect business to do anything that's not in its own interest. This sort of problem, if in fact it's a problem[1], would seem to require a legislative solution.