EDIT: Apparently MIT has turned this whole line of thinking into an app where they collect a consensus on what a machine should decide: http://moralmachine.mit.edu/
This is because they will have made an agreement with the driver in advance, and have a known quantity to pay in case of collision death, while amount they have to pay out for other random people will be less predictable [1] and err on the high side since they're so deep-pocketed.
This matches my intuition on the trolley problem, which is that you have to factor in the "non-involvement" of the guy not on the tracks -- he should count as more than one person because he did not deliberately take on the risks of crossing an active track. Ditto for diverting the trolley off the track into a random person's house. But such people aren't worth infinitely more either: you should probably divert a terrorist nuke from an urban area to a sparsely-populated desert, even though the hermits might have deliberately chosen the desert because it was an unlikely terrorist target.
[1] Assuming governments don't legislate specific damage caps
If you're ever in a situation where it's possible for a child to jump in front of your car and you are not able to stop in less than a car length or at most two, you were driving too fast. And if you're driving slow enough to stop that quickly, it doesn't matter what the car does, it's not going to kill anybody.
The problem is recognizing the obstacle with a better accuracy than people do. Tesla's a good example for this, and being unable to recognize certain obstacles in certain conditions.
It feels weird saying this, but there is a sort of physical threat that being behind the wheel gives you.
Same goes for when the lights at a junction are out of service. How will an AI handle that?
In general, proper autonomous cars are going to need a shit-ton of data collection at a very fine level everywhere. Together with the required sensor suite, the cost increase over a normal car in the same class is going to be huge. I'm not talking 20%, I'm talking at least twice as expensive.
This is why I think Ford are making exactly the right bet: taxi companies will adopt driverless cars many years, even decades, before private drivers. Because a taxi company saves a lot of money on not having any drivers (I'm guessing not far from 100-150k per year per car) on an autonomous car, they could easily afford to pay $150k for a Ford Fusion (normally $30k).
This purported solution seems to be a special case of "assume I will never encounter a moral dilemma that forces me to make a hard choice". It's not solving the dilemma, but refusing to recognize that one exists. [1] Even if you think it can't happen this way, there are a hundred other ways it can manifest while driving.
[1] LessWrongers advocate the practice of choosing the "least convenient possible world" to avoid such side-stepping: https://wiki.lesswrong.com/wiki/Least_convenient_possible_wo...
For every "but that'll never happen" there's a contrived scenario where it could. Yes, it could be a vanishingly small chance, but it's not going to go away.
The point I was trying to make is that the trolley problem is always going to exist, saying "no, just slow down" doesn't make the problem of making decisions go away.
There are always going to be scenarios where an automated system is going to have to decide what to do in a life or death situation. And someone's going to have to decide how such systems will respond.
So either the situation has obvious probabilities and subsequently obvious solutions or there are such huge error bars that there's no right answer.
- stuff that is relatively common, like children playing on streets. So if you get into that situation you were driving too fast, or there's something you should have done in the first place to avoid it.
- stuff that isn't common, like children playing on freeways. In which case it's rare enough that the answer is a nice philosophical exercise but isn't practical. The real world is never as clean as such philosophical exercises. Defaulting to 'slam on the brakes' may perhaps be improved upon, but the difference will be very marginal.
But none of these are really important because they only apply in events that already have a relatively low base rate: accidents.
Any ethics system that may or may not have to sacrifice your life for a school bus full of nuns will only be forced to make decision with an even lower baserate: accidents + a bunch of additional conditions where the only way to save your live would be risking others, with a reasonable confidence to boot.
Elevating this small aspect over everything else seems silly.
But I guess I'll qualify my original statement: "within reason".
Well, your thinking might be irrelevant.
Let us assume the following points:
- we live in a democracy with self-interest driven voters.
- most commonly the victims in accidents are pedestrians, not occupants of the vehicle. measured by frequency and severity
- not every citizen is a car-user (even in an autonomous-vehicle world). e.g. some may prefer bicycles, especially children
Additionally not every citizen will have any interest at all in being protected while inside a car because they're always outside the car. Thus the overall interest of the citizens will be to prioritize pedestrians over car occupants. And this being the majority view means means cars with selfish behavior will banned.
Also note that in this scenario your own self-interest aligns with that of everyone else when you're outside a car. It only mismatches while you're inside a car.
But in the end this is all a storm in a teacup. When a system gets to a point where it has to make such tradeoffs then you already failed. And ideally this happens very very rarely. So you just end up shifting 0.0001% risk factors around. There are more important things.
Who gets to write the "within reason" code?
Anyways, the main thing here is that I don't want my car making some calculated moral decision.. 3 nuns vs. my current occupant. It should obviously make a reasonable effort to avoid hitting meatbags, but if doing so puts the occupant in overwhelming harms way, it shouldn't.
Anyways, I think this is going to be a 0.00001% thing, and in the case that the car may need to swerve (dangerously) to avoid hitting an obstacle, it would be able to do it with precision and care that I'm simply not capable of.. Think swerving on a highway. Pretty dangerous for me to do since I can't check my blind spots that quickly, but it always can.
Say your car detects a high speed head-on collision with another non-autonomous car is imminent, your car detects one adult and two children in the other car, and it determines it can only avoid collision by ditching your car, risking your life. What should it do?
There is plenty evidence that many adults will sacrifice their life to save children.
Not driving around minimizes the chance of accidents after all.
If one takes the fact that the problem is partially observable into account, many of these moral issues disappear. We can control the environment inside the car making it easier to be certain that a human is inside. Because the car is certain that it contains a human, and less certain that it is running into humans, it should take actions to preserve the life of the occupant.