Hacker News Reader

Comment by kxrm | Hacker News Reader

The trolley problem is a tough one.

I've never understood why. I can't imagine any programmer or self-driving car company having any interest whatsoever in going down that particular rabbit hole. The answer is simple -- we don't expect humans to do realtime analysis of who is worth killing when an accident is happening, so we will not expect that of computers either. It's an unsolvable problem. So we just tell the computer that in the event that an accident appears unavoidable it should do everything within it's power to stop the car immediately. End of story.

And this kind of evaluation of one life against another is completely illegal in Germany. I.e. the german automotive industry legally can not make such systems in their automated cars and no cars operating in Germany can ave these systems either. I fully exect other countries to enact similar laws, if it ever becomes necessary.

It's a thought experiment; do not expect a literal trolley (or even literal corpses!) any more than Schrödinger's cat deals with actual felines.

Here's a restatement without all those corpses: In the event that an accident appears unavoidable, the computer should do everything within it's power to stop the car immediately. (This is already a given: safety first.)

"If there are multiple ways of doing the above, what variables should the computer optimize for - stopping distance alone, or stopping so that it doesn't immediately get rammed from behind?" There's your trolley problem again, just restated so it doesn't appear so offensive: in both cases, the occupants of the vehicle are in danger, as are the occupants of nearby vehicles. Now is the interest in the rabbit-hole clearer? The problem doesn't go away just because it's inconvenient to solve...

Well, that's where crumple zones come into play. Trying to have a computer calculate whether or not it is feasible to stop slower to avoid or reduce an imminent rear collision gets very complex in a hurry. What if the guy behind you is much smaller, or much bigger, what if his car has better crumple zones, or none, what kind of energy is he going to impart to you when he hits and where will that send you, who is at that location, etc. These are the kind of questions that can't be answered with certainty even when you give a supercomputer a few days to work it all out. So I think the only reasonable solution is to not expect self-driving cars to make any kind of moral decision. As a bonus it is much easier to explain in court and saves us endless pontificating on whether or not the computer made the right choice and who is actually responsible if it did not.

We tell humans not to swerve for squirrels, because it's a great way to end up dead, so the computer should get the same instruction. Slow down as you can, the prime directive is to maintain control.

In that case, I present a SDV that never makes a wrong choice - by remaining stationary at any cost ;o)

In other words, there is always a balance between safety and usefulness, and the question of "is this the right choice" is always upon us, whether we want it or not. You can never have certainty anyway, what the software is doing is maximizing on some reward function. The TP also asks "is there even a moral component to this?" It seems there is, from the range of emotions this conjures up.

(btw "do not swerve to avoid unknown objects" has directly caused at least 1 dead person - Elaine Herzberg - so that's a really unfortunate maxim for illustrating your point: the car has also an obligation not to be a danger to others. "We just maintain control and everything else be damned" is easy to explain in court, true: IANAL, but sometimes you need more than a simple explanation to avoid being convicted.)

To be fair, I said don't swerve for a squirrel. History is filled with examples of people giving up their own life for a squirrel/cat/dog because they swerved and lost control. You should have planned in advance what your threshold is for swerving. E.g. just brakes for anything <= white-tailed deer, and active avoidance w/braking for anything bigger, or human.

Elaine is not a great example because not only did the car choose not to swerve, it also chose not to even try to brake. And on top of that, a human would have seen her from a lot farther away.

Because it was trying to decide if it's a shopping bag or a bike or a human or a squirrel...for six seconds, long enough to stop multiple times. The problem, from what we know so far, wasn't "car didn't see her," it was "car wasn't sure what it was, therefore squirrel."

And the larger point stands: "protect occupants, ignore outsiders" is a choice in the TP, always choosing the same strategy is not "TP is irrelevant."

In my entire life, I've never encountered the Trolley Problem, and I've never met anyone who has.

Most safety problems in driving can be solved by slowing down.

The problem with AI is all those weird, little edge cases that humans can reason through -- for example: if there's a deer next to the road, then I'll slow down, even if it's not on the road yet. I've known many people who have hit a deer when it spontaneously jumps into traffic. Or something like: someone's not quite staying in their own lane, so I have to be careful when I pass them.

Perhaps most safety problems. But if you need to outrun an erupting volcano, a tsunami, or the police, slowing down won't really help you.

"... outrun ... the police"

I doubt that this is a use-case that legal, for-profit companies will be pursuing.

For the other use-cases, you can just say: Manual driving only.

I think that this is a more realistic/relevant example of "slowing down or stopping is not always the safest action":

https://www.youtube.com/watch?v=eWuK-fi-D_w

It has nothing to do with the trolley problem. The trolley problem is a headline-grabbing non-problem in the actual design of driverless cars.

The real trolley problem:

[ ] Let self driving cars kill people during development but try to make it up by saving thousands of lives after they are perfected. (Uber)

[ ] Make self driving cars extremely safe from the start but more people end up dying from manual driving because development takes longer. (Waymo)

The underlying assumption "general autonomous driving can be made much safer in a short timeframe [years]" is not granted. It may be that the first option could have a death toll higher than what it would prevent - but the thing is, we don't know, it's an unknown, an assumption "it will be much better once everybody drinks the koolaid."

Reply on news.ycombinator.com