Should your driverless car kill you to save a child? | Hacker News Reader

Should your driverless car kill you to save a child?

qz.com

7 pointschton19 comments

This scenario should never be possible.

An autonomous vehicle should never be traveling fast enough to not be able to stop for any objects that could potentially jump out in front of it.

If there is a blind spot, slow the vehicle down. If there is a crosswalk, slow down.

What about animals though? I often see squirrels, geese, and deer in the middle of the road. Does the vehicle honk the horn and inch forward? Geese are dumb and will stand in the middle of the road until you scare them or they decide they want to cross, squirrels will actually challenge the vehicle many times.

Someone will die from an autonomous vehicle eventually, there's no way to prevent that just because of the odds of some fluke happening. If someone trips or purposely jumps in front of an autonomous vehicle then there is nothing that could have been done, don't kill the driver for the pedestrians mistake.

Autonomous cars should obviously be programmed to obey all traffic laws. However, since human drivers don't, this in and of itself may create the impression for passengers that autonomous cars are inexorably slow. If one assumes that even a human driver obeying the traffic laws and speed limits is still too unsafe, having them move even slower than that may be intolerable.

Because they're considered more safe and more efficient, autonomous traffic may be designed to move even faster than is legal for humans (they would be subject to their own distinct traffic laws.) This may be especially likely for automated trucks, which will be expected to deliver their cargo as fast as possible, as often as necessary.

I agree with everything you said, as long as the vehicle has a clear line of sight of the road ahead and there is enough room to drive off of the road if something unexpected happens.

In the situation mentioned there is a blindspot where a child could be (for whatever reason) and a wall on the other side of the road. The vehicle was traveling fast enough so that it could not stop or avoid the child and if it crashed into the wall it would kill the passengers, in my opinion this vehicle in this situation was traveling too fast.

How about this scenario, say you purchase an autonomous supercar that can travel at 150+ MPH down the highway and a deer approaches the road, the vehicle should slow down to a point where it can safely avoid the deer no matter what random action it takes, correct? If the deer hits the car, everybody loses so it is in everyone's best interest to avoid that situation even if it means that person gets to work a few seconds later every day because there are always deer at a certain place on the highway.

This is especially true in cities, yes, autonomous cars could probably "safely" travel at 100+ MPH but if something unexpected happens it would not be acceptable to hit several pedestrians due to the unknown happening, like someone or an animal running out into the street.

What exactly is the argument you are trying to make?

Your initial position of;

"This scenario should never be possible."

"An autonomous vehicle should never be traveling fast enough to not be able to stop for any objects that could potentially jump out in front of it."

Seems to flatly contradict your following statement that;

"Someone will die from an autonomous vehicle eventually, there's no way to prevent that just because of the odds of some fluke happening. If someone trips or purposely jumps in front of an autonomous vehicle then there is nothing that could have been done, don't kill the driver for the pedestrians mistake."

Do you really think you have thought through your argument here?

If the vehicle has enough time to detect someone has fallen in the road to be able to swerve and hit a wall (killing the driver), then the vehicle should also have been able to detect that someone was close to the road (or that there was a blindspot near a crosswalk) and it should have been traveling a speed that would allow it to either stop or hit the wall without fatally injuring the passengers.

About the vehicle killing someone, every object ever made can and probably has killed someone. Cellphones, pencils, car keys, snow-skis, etc. If someone tries to get hit by an autonomous vehicle I'm sure it would be possible somehow, but the car should not drive off of a cliff if someone jumps in front of the vehicle or throws a mannequin in front of it.

Basically, if there is plenty of room on the side of the road for avoiding unexpected objects and there is a clear line of sight in front and around the road, go fast. If there is a very narrow road with a wall on one side and pedestrians on the other or blindspots where pedestrians could potentially be, go slow enough to avoid having to choose who to kill.

"If the vehicle has enough time to detect someone has fallen in the road to be able to swerve and hit a wall (killing the driver), then the vehicle should also have been able to detect that someone was close to the road (or that there was a blindspot near a crosswalk)"

Those are two different problems and an ability to do the former does not necessarily indicate an ability to do the latter.

The car needs to be able to see well and detect pedestrians, they can already do that:

https://www.youtube.com/watch?v=eXeUu_Y6WOw

Note that this was a few years ago, it's undoubtedly better.

Ok, so in the situation that this code is not perfect, which we know it will not be, what code do we run?

You have already agreed that these situations will arise and the question is not about what should be done to stop the situation arising in the first place, which is what your answer concerns itself with.

Rather, it is what to actually do in the situation that if it does nothing the car is going to hit a pedestrian at speed and all avoidance strategies it has calculated seem to massively increase the danger to the occupants.

Saying that the vehicle should not be in this situation does not answer the question of what to do in this situation, unless you can provide a strategy that makes this actually impossible, which you admitted at the start is not achievable.

"Ok, so in the situation that this code is not perfect, which we know it will not be, what code do we run?"

You're going to have to ask someone who works at Google that kind of question.

Not all of the rules are hard-coded. Most of what makes Google software smart is their advanced machine learning, prediction, and AI algorithms.

"Rather, it is what to actually do in the situation that if it does nothing the car is going to hit a pedestrian at speed and all avoidance strategies it has calculated seem to massively increase the danger to the occupants."

Apply brakes, activate horn, swerve, deploy airbags, call 911.

If the pedestrian gets hit, then it was obvious that the pedestrian did something very wrong. If it is determined that the vehicle was traveling too fast for that particular stretch of road (because of blindspots or because it's beside of a playground or busy sidewalk) then adjust the algorithm to look for clues that will make it operate at a lower speed in this type of area.

You're going to have to ask someone who works at Google that kind of question.

Apply breaks, activate horn, swerve, deploy airbags, call 911.

If the pedestrian gets hit, then it was obvious that the pedestrian did something very wrong.

If it is determined that the vehicle was traveling too fast for that particular stretch of road

You are just relentlessly contradicting yourself.

I'm not saying that my answer is the correct answer, it's just my current opinion.

Do you have anything to contribute to the conversion or do you just like telling people that their opinions don't make sense to you and are therefore wrong?

The point of a forum is to allow discussion. Please read over your comments and tell me if you think that they contribute meaningfully to the conversion.

What do you think autonomous vehicles should do in this situation?

Yes, you should ask someone at Google for a better answer, but I provided my thoughts above. Yes, a pedestrian can walk and trip into the street and the car could be going too fast for that stretch of road, both did something wrong.

Being able to stop for anything that jumps in front of your cars is impossible at any speed. You could be going 5mph, if the thing jumps close enough your reaction speed will never be good enough to brake in time. it's the same with the car's reaction time. It doesn't matter what speed you pick, accidents can happen. If anything, self-driving cars would be allowed to drive a lot faster, since they have far faster reaction speeds and far more information about their surroundings than human drivers.

The slower speed would allow for the vehicle to crash into the wall (if that's the only option) and everybody would live.

Unless the person falls from above, from a blindspot, or falls a few feet in front of the vehicle, it should be able to stop or maneuver around the object without a problem. Cars are VERY maneuverable, especially when you add computer controlled reflexes.

Yes, autonomous vehicles would have a much higher speed limit (having a supercar would actually allow you to get to work much faster than people with normal cars) but they should also be able to recognize higher risk areas and know to slow down in order to minimize the likelihood of killing someone.

Every crash has the potential to be fatal, and forcing all self-driving cars (or indeed all cars, since the same argument applies to human drivers) to keep to extremely low speeds isn't a viable solution. Cars are only manoeuvrable at low speeds. At high speeds, the laws of inertia dictate that they take a lot of energy and time to stop or turn. This is true for all cars, computer controlled or not.

But all this is beside the point. There will be a moment where a me-or-them situation will arise with self-driving cars, regardless of precautions. The discussion this post is trying to start is about what happens at that point, not how we can minimize the occurrence.

I'd like to see a simulation of this scenario, I find it hard to believe that the only options would be kill the child or the passengers. If that was the only options, then the vehicle shouldn't have put them in that danger by traveling that fast. The road would have to be extremely narrow, and if there was a wall on the other side, where was the child trying to cross to?

If there are pedestrians right at the edge of a road and there is the possibility that they might fall into the road, then the vehicle should be traveling at a speed which would not be fatal or at a speed where it would be possible to avoid them by driving off the road.

I absolutely did not say "force all vehicles to travel at an extremely low speed", please don't twist my words.

They're pretty maneuverable: https://www.youtube.com/watch?v=4zOFNTkxMvQ

Also, what if a deer walks or falls into the road? Should the vehicle kill the passengers? What if you just throw a mannequin of a small child into the road, should it kill the passengers? No, it should do what it can to avoid the accident but always protect the passengers if possible.

To answer to your edited version:

I agree that it should avoid an accident as much as possible, but I don't assume that that will be an option in all possible situations. The case given was a thought experiment to indicate the ethical conundrum, not a real life situation. It's not supposed to be realistic, it's a jumping off point for a discussions.

The problem with your statements so far is that they assume perfect knowledge of the surroundings. How fast is the car allowed to go if it's 99% confident that there is nothing that can jump in the way? how about 99.999%? When is enough? If you deal with absolutes, the only thing you can end up with is "an extremely low speed".

Lastly: I'm not sure what you're trying to prove with that video. At any speed high enough, a car needs distances measured in yards to stop and turn, regardless of how good it is. In the case of trucks, that can even be measured in miles. There are rigid limits on how manoeuvrable a car can be, and they need to be calculated into anything the car does.

Would a driverless truck with $5 million in cargo be programmed to drive off of a cliff or run over the child?

Either way, less people will die with autonomous vehicles, let's start rolling them out ASAP.

I agree with your conclusion, but for different reasons. There will be loads of situations that are far less clear-cut than this. A lot of times, they will involve other cars. By always taking the options that prefer the passengers, the car will be reacting in far more predictable and simple manner. This will make it safer for other cars that do the same to react to. Communication with the other cars might help reduce danger overall, but all of them will need to be able to manage traffic even in situations where this doesn't work. If the choices of other cars are too unpredictable for a given situation, that's a recipe for disaster. Unfortunately, the selfish choice will always be the simpler.

"When robot cars must kill, there are good reasons why designers should not be the ones picking victims."

There are also good reasons for them to be picking victims, in the case presented, switching control to the driver may increase the chance of both the child and the driver dying. The designers of roads made the decision anyway, roads for motor vehicles could all have big sidewalls and be up on stilts and never be a threat to pedestrians or wildlife, if we wanted them to. It would be very expensive to do this, so we accept the deaths.

Reply on news.ycombinator.com