Hacker News Reader

Comment by mikeash | Hacker News Reader

"Per hour" is rather ridiculous. I'm talking about per crash. There are a couple million vehicle crashes in the US each year, so if it really is 99.9999%, that's two or three ethical crises per year here. Among a billion vehicles it would be maybe ten per year. And honestly I'd be surprised if it's that many. When have you ever heard of a crash where there was any kind of ethical dilemma in the response? People keep imagining scenarios, but I've never heard of one happening in real life.

As for speed limits and such, I'm talking about the car's ethical choices in crashes, not the ethics of the programmers.

View on HN

I'm actually familiar with a situation like this, the person was on a highway at night and came around a corner to see a deer lying in the road and a crowd right behind the deer. There was a couple of people off to either side. They couldn't brake in time to stop and not harm the people on the other side of the deer.

What's the ethical dilemma there? From your description it sounds like they were just screwed regardless.

Also, an autonomous system would probably get information about that sort of problem and slow down prior to going around the turn.

At least, they will if we don't screw it up.

Human drivers can already get information about that sort of problem, e.g. by using Waze. This requires that a human who uses the app has reported a hazard (the act of which could sometimes result in distracted driving), however, I would expect autonomous cars to tell each other about such things in the future.

Self-driving cars can also limit speed such that they'll always be able to stop for stationary obstacles. If it's a blind corner, they'll slow down.

True, but seeing as the article was about the ability of people to modify their car to for instance not do that I see where there could be a dilemma here.

thank your for saying this!!!

Because the article was completely about people REPAIRING technology they purchased, and not at all about mods, modding or modifications.

They're pretty much one and the same, how can you allow someone to repair something without allowing them to modify it as well? What if they repair it badly?

how can you allow people to own guns without allowing them to shoot people?

how can you allow mechanics to fix brakes without allowing them to break brakes?

I dunno... laws maybe? Like, the same laws that can require manufacturers to allow aftermarket competition.

> how can you allow people to own guns without allowing them to shoot people?

even more, correct me if I'm wrong here, but you can repair a gun, but it's illegal to make certain modifications to it, right?

>but it's illegal to make certain modifications to it, right?

That's correct in the US. For instance, you can repair an AR-15 back to factory specs, you can modify it in certain ways (adding different sights, etc.), but if you modify it for full-auto operation (which is quite feasible actually), you can be thrown in jail IIRC and be subject to a gigantic fine.

This system works well: people can repair and modify their property to a certain point, but making specific changes are highly illegal, and the penalties are extremely severe, so almost no one does it. The same can be done for other things if the case for public safety warrants it. So repairing your robo-car should be completely legal, but modifying it to blatantly violate emissions laws or to run cyclists off the road can be punished harshly with vehicle confiscation, $1M fines, lengthy prison sentences, etc.

Are you saying the correct response is to swerve in to the group with the fewest people?

I don't think human drivers have the reaction time to make these decisions, not sure why so many expect computers to be able to.

Because eventually, computers will be able to.

Those computers probably won't get into situation like this in the first place, being able to stop or react preventive. Anyway, it will be so rare an occasion with no guaranteed outcome (it's all about probabilities), so it's not really matters as much as it's discussed. With same kinda of attention you could ask if making and selling ladders is ethical, because people fall from them and hurt themselves.

> Those computers probably won't get into situation like this in the first place, being able to stop or react preventive.

It will always be the case that circumstances can change faster than something with the momentum of a fast moving car could adapt. Somewhere between "you're boned" and "your car saves the day" lies room for a scenario such as this, with potential room for a large amount of thinking.

> Anyway, it will be so rare an occasion with no guaranteed outcome (it's all about probabilities), so it's not really matters as much as it's discussed.

I agree that it will be rare, and that the concern is overblown. That said, "it's all about probabilities" is no reason something can't be vitally important.

> With same kinda of attention you could ask if making and selling ladders is ethical, because people fall from them and hurt themselves.

I think there is a substantive difference between the two. We're not asking whether selling a car that has a chance of injuring the user is ethical - we're fine with that. We're not even asking whether selling a car that has a chance of hurting others is ethical (already meaningfully different than the ladder case). We're asking about the ethical ramifications of making particular tradeoffs in "chance to hurt the user" versus "chance to hurt others". It's an interesting question, so it gets a lot of attention. I don't think most of those involved see it as a reason to prevent self-driving cars - long before the dilemma is really relevant, self-driving cars are already safer than human driven.

> It will always be the case that circumstances can change faster than something with the momentum of a fast moving car could adapt. Somewhere between "you're boned" and "your car saves the day" lies room for a scenario such as this, with potential room for a large amount of thinking.

I think it's possible that with improving AI of self driving cars, making overall safety better, there will be ≈0 occasions when car simultaneously 1) doesn't have time to react to some sudden problem and 2) has time to make informed decision (and physically perform it) on how many people to save. Especially if pedestrian airbags become popular.

But by then the car will be able to engage its flight module, take off, and fly you directly home.

Maybe. Flight generally uses quite a bit more energy.

Yes, but perhaps less than you think. I fly small aircraft, and the one we fly gets ~15 mpg in cruise carrying up to 6 people at just over 200 mph. There are cars/trucks on the road that get less than that. If I slow it down to 150 mph, I can get over 20 mpg.

Even if, the car will have to calculate whether it has enough clearance to deploy the wings so that it doesn't cut down a pedestrian, or whether there's anyone that could be caught in the engine wash.

Interesting as it may be, it's only shifting the problem ;).

They also could not think. If you've ever been in an accident, you know that there's no time for thinking. Reflexes take over.

Ascribing ethical decisions to the driver in such a situation is not rational nor ethical.

no, but it's a situation in which a computer could make a decision

I imagine you mean a choice like "should I run over this child who suddenly ran in front of the car or should I swerve into opposing traffic?"

The people crowded behind the deer ought to have sent a person off in either direction to warn approaching traffic of the obstacle.

I recently attended to an echidna sitting in the middle of the other lane in on a blind corner. I reversed until I could see straight road for 60 meters, stopped with my car at an odd angle on the side of the road with the hazard lights on to give an indication to approaching vehicle something unusual was occurring ahead, namely my partner and I were on the road on a blind corner. Then we listened carefully for approaching vehicles while we got the echidna off the road.

The ethical dilemma lies not with the driver, who is otherwise driving to the conditions, but with the people who have put themselves at risk by performing a task in a dangerous environment without appropriate hazard protection.

A properly designed self driving car would never out drive its sensors in the first place. This is just basic defensive driving. If there's a blind corner coming up then it would slow down sufficiently that it would be able to come to a controlled stop if there's a stalled car or other static obstruction around the corner.

I'm pretty sure the correct behaviour is to not come around a corner so fast that you can't stop for a non-yet-visible obstruction. Human drivers routinely flout this due to poor risk assessment, but an AI driver would easily be able to to do it.

I'm pretty sure the correct behaviour is to not come around a corner so fast that you can't stop for a non-yet-visible obstruction.

Do you actually practice this? Setting aside legality concerns (many freeways have minimum as well as maximum speeds), if you truly did this at every curve and corner you'd be more likely to cause than to prevent an accident. The number of times there will be something you avoid by slowing down will be outweighed by the number of times your speed differential relative to surrounding traffic causes an accident (and it is speed differential, not speed itself, which is responsible for virtually all "speed-related" accidents).

For truly blind corners, yes. However, I don't have great faith in my ability to correctly judge all partially blind corners, and occasionally find myself surprised.

I don't advocate slamming on the brakes seconds before the turn, there's such a thing as slowing down safely as well.

All of this is something AIs will be nearly infinitely better at doing safely than even a minimally impaired (tired, distracted) human driver.

> I'm talking about the car's ethical choices in crashes, not the ethics of the programmers.

Cars are not entities to which ethics can be ascribed; the humans building, designing, and programming them are.

Let me clarify: the topic here is specifically the ethics of whoever in a crash where the car may be able to get different people killed depending on what actions it takes. Speed limits and DRM are an entirely different topic.

I think another way to phrase what you are saying is that "Cars are not moral agents." .

Apparently there has been some disagreement as to whether cars can be moral agents?

I like how you're going to fight about other people's numbers, but act like "99.9999%" is in any way meaningful.

I happen to think that a rate of one in a million crashes is realistic (even if far from accurate, it's roughly in the ballpark) while a rate of one in a million hours is absurd on its face.

> I'm talking about the car's ethical choices in crashes, not the ethics of the programmers.

There is no practical difference. Computers do what we tell them to do, whether what I tell it to do has a conditional in code (the computer chooses), or instead always chooses one option because I left that condition out is a completely irrelevant distinction.

You realize that the common `debugging caveat' applies: computer do indeed do what we tell them to do. That doesn't mean than anyone understands what the computer are going to do.

Debugging sucks.

Formally verified code is not unheard of.

Formally verified code is awesome. It moves the `debugging problem' up one level, "Does our formal spec capture our informal meaning?"

Ideally in practice, that problem is simpler than "Does our code what we want?

In theory in the abstract, of course, the problems are the same.

Reply on news.ycombinator.com