Hacker News Reader

Comment by NPC82 | Hacker News Reader

I think this problem is nonexistent to an autonomous car. A situation where the car has a choice to hit something would never happen. The only times humans have that choice is if we don't look far ahead enough or our field of view is impaired.

Sure, autonomous cars may eventually hit something. But if they do it will be totally out of their control, by design. We can make them see further, stay focused, and be cautious. Humans can't always do that, and that's why we have to constantly choose our lesser evils.

View on HN

There will always be unexpected situations, especially while we still have human drivers on the road. Ultimately, if you want to actually get anywhere while there are other cars on the road, you have to make some assumptions about how they're going to behave. And even when you're completely automated, there may still be unexpected failures that cause these situations.

However, I'm not sure the ethics of this are such a big deal, because the decision is such a tiny part of the causation of the eventual accident. I think it's far more likely that people will write code that just keeps trying to dodge until it can't any more, and I don't think that's a bad thing. Ultimately you have to make some compromises on safety vs resources, and deciding not to spend time writing special behaviour for pathological cases is no more ethically suspect than deciding not to spend any more time improving the brakes.

It is a real problem. It should be fairly obvious that it is. More realistically, It won't be a real time decision of hit this or that, it will be that it is programmed to do behavior X instead of Y. X is known to be 30% more likely to hit a pedestrian, but 10% less likely overall to be in any accident. Then you dramatize it by saying the car decided to hit a pedestrian instead of a car. Just substitute the behavior described in the story or some variation.

I think it will probably be much more complicated than that. Suppose the car is controlled by an artificial neural network that has been trained with a combination of real world experience and human-generated driving data. There is no giant switch statement that someone wrote for the outcome. Who is to blame then? Would using that data to train the model further be sufficient accountability?

I disagree. Seeing further and predicting more moves in advance will not help you avoid a child that suddenly runs out in front of you from behind a parked car. Autonomous cars might not have blind spots in the traditional sense, but I doubt that they can see through solid obstacles.

There just won't be enough time to deal with such a situation without hitting anything, even though there's probably more than enough time to analyze a thousand possible maneuvers. The best "seeing further" that you can do is to slow down when entering roads that are statistically more likely to have children behind obstacles, but even that is not bulletproof. The only place where "seeing further" is enough is when you're in the middle of the prairie with miles of clear visiblility in all directions.

Well, yea, actually it would. Because the vehicle would detect the movement long before I do. And even in that situation, hit the brakes is the right answer.

@NPC82 claimed that the car would never need to decide to hit anything. And I disagree.

Just because braking is the obvious "right answer" doesn't mean that there's nothing to decide.

Modern cars have electronic stability control, which allows steering even while braking hard. In fact, it would be irresponsible for an autonomous car not to consider exercising that option in addition to braking. But as soon as it considers steering, it needs to decide what (or whom) to aim for.

No, it really doesn't. Humans don't make that decision very well, hence why motor vehicle accidents are such a high rate of death.

You're framing this as a false dichotomy. The car has to hit A or B. If all the sensors are working, it won't have to hit anything, it will break.

Never is a hard thing to prove. Just as human drivers have limits, so does any autonomous system. It's quite possible that an perfect driver can become aware of a problem with insufficient time to avoid collision, due simply to inertia and circumstance.

An example: The autonomous car is driving happily down a residential street, tailgated by another car. A car pulls out of a blind driveway immediately in front of the autonomous car. The car must then make the choice between T-boning the car in front or slamming on the brakes and hitting the car behind. To remove the ability to swerve, let's add pedestrians in the sidewalk and a steady line of cars in the oncoming lane.

Good example, but what the autonomous car would likely do then is brake gently as soon as it starts to be tailgated so the speed is safe for the distance to the car behind. This would no doubt seriously discourage tailgating, and mostly prevent the scenario you describe from developing. But I agree, you can't completely avoid all crashes, especially if someone is determined to crash into you.

I believe you're making the mistake of assuming that, in practice, autonomous vehicles will be perfect implementations of perfect models, operating under perfectly predictable conditions. In fact, in mass production, costs will be cut, compromises made, specs ignored, exploits will go unseen or ignored, and unforseen consequences will occur.

This is not an indictment of autonomous vehicles as a concept (though I am critical of them, mostly on ethical and not technological grounds) but a statement of fact. You can't mass-market a perfect machine.

Reply on news.ycombinator.com