Will your driverless car kill you so others may live?
latimes.com
latimes.com
Why did an autonomous car with a map that tells it there's a crosswalk around a blind corner round the blind corner so fast it couldn't stop if there was unexpected debris on the road?
What a stupid scenario, autonomous cars have infinite patience, and when the people inside them can watch netflix all the way to work they won't care much about a few extra seconds either.
The obvious solution is to program the car to never take any blind corner, mapped crosswalk or not, so fast it can't stop on a dime. These things are going to have to work in winter in Canada. Low traction is a regular problem they'll have to solve. That'll mean every auto that takes this corner will do it very slowly, but that's fine because it can report back to the citywide network that this corner is unsafe, meaning all other autos will avoid it if they can (so no congestion), and as a plus it can be registered for modification with the local council. Maybe a networkable sensor can be erected, allowing the cars to see around the corner.
Taking corners faster than you can stop if there's a problem is a very human thing to do. One of the key benefits of autos is they won't do that.
We were talking about the Trolley Problem at a recent dinner with friends, and one of the guests remarked about how it was like Star Trek; "The Needs Of The Many" versus "The Prime Directive".
So it's a general argument about where a car has the capability of making an uncomfortable utilitarian choice (The Needs Of The Many), or of simply shutting off any ability to affect matters and let nature take its course (The Prime Directive). The uncomfortable bit there is that with the latter, there's still that actual choice (by the designers or whoever) to follow the Prime Directive - it's arguably not "natural" anymore.
In general, that's right. There is no reason for the vehicle to out-drive its unobstructed sensor range.
One effect, which the Google cars already demonstrate, is that their behavior at corners with obstructed visibility is markedly more conservative than when the LIDAR can see down the cross street. That's a good thing.
Having once coded this for a DARPA Grand Challenge vehicle, that's how you look at the problem. You're always plotting stopping distance against the map constructed from sensor data. You figure out how far it's safe to advance given sensor data, and compute a top speed from that. We did this every 100ms. If the map wasn't filled in with new safe space into which it could advance, the vehicle would slow down and stop at the end of the safe area. So it never out-drove its sensor range. Remember, this was for off-road operations, with obstacles and cliffs; we had to be very conservative.
Humans tend to get into a mindset of not wanting to slow down. Software doesn't have that problem. Constantly having a plan available for an emergency stop is not a problem for software.
Even if a self driving car runs over a pack of school children daily, it will still be less fatalities than humans cause.
Gerry Sussman explained this recently way better than I can.
"Sure, when you're running the program, that whole thing is a black box. So is your brain. But you can explain to me the reasoning of why you did something. At that point, being able to inspect the symbolic reasoning of the system is all you have."
made me immediately notice that there's a reason we have negative connotations with the world "rationalization" - because most of the time, post-fact rationalizations are bullshit and do not reveal one's actual thoughts at the time of the event, but what one wishes one thought at that time. Just bolting a "reasoning explainer" on top of a Bayesian inference system won't fare any better than human rationalization does.
If he's coming out of a blind sidestreet, why are you approaching so fast and with so little lateral room that you can't dodge a surprise motorcyclist?
These are all human patience related problems. An auto should be perpetually correcting to make sure that it's current minimum collision avoidance distance is smaller than it's current visual clear zone.
As long as there are other actors in the world who may behave irrationally I don't think you have to work too hard to come up with either/or scenarios for the AI to deal with.
And even if the irrationality is removed there's still a chance of mechanical failure (e.g. tire blowout causes loss of control) which other AIs may need to make decisions on how to avoid.
It applies human limitations and distractions to a computer.
The scenario will never come.because the autonomous car sees everything with way greater clarity than any human. And it will never miss a person crossing the street because its rubbernecking at am accident or gawking at a hot girl or answering a test or trying to turn up the radio because its favorite song is on.
There won't be a blind corner as the thousands of other autonomous Google cars passing by would already be napping and sending live data.
Probably even some of the children crossing the street will have Google devices aw ding the data to Google and through to the car.They're programming them to try and avoid accidents in general. There will be times when the computer fails to do that, but that doesn't mean it decided to kill someone by making a moral choice about human life priority. Maybe in the future this will be a relevant argument but the sort of algorithms mentioned that decide which crash scenario is 'best' simply don't exist.
The best way to deal with these moral issues is to just... not program that behaviour.
https://en.wikipedia.org/wiki/Trolley_problem (They even have an "Implications for autonomous vehicles" section now.)
So, who knew? Suddenly it's not so theoretical, it apparently has a real-world application. Except, as you seem to be getting at, probably the only thing engineers will be considering in any depth is how and when to brake, not trying to swerve the car elaborately in one direction or the other. The number of occasions when a serious accident occurs because a driver reflexively swerved to avoid an animal or other obstruction (sometimes literally dozens dead in one accident) emphasises that swerving is not so good a strategy as braking. Or, as my driving instructor once told me, "There is only one cause of accidents on our roads: failing to stop in time."
Why would a computer choose to swerve and not slam the brakes. With reactions that are as fast as a computer could be what kind of a situation even is a swerve vs a course correction.
Then if you run the outlandish scenario, and assume extraordinary leaps in AI comprehension in the next 30 or 40 years, it still remains silly. How many teenagers are equal to one baby? How many puppies does it take to equal one 80 year old grinch? There is no scenario under which it will work. Ultimately the AI will not be able to make accurate judgments about morality / character, it can never choose correctly even if you try to program it with what would be strictly subjective value systems - that one flaw guarantees it can never work properly.
Well, this is something we don't expect from humans. No human, except maybe high-ranking businessmen, leaders and armchair ethicists ever does that kind of calculation, because in the usual case we meet them, there is no time. When you're a second before a car crash, you don't count the passengers of the other vehicle and evaluate their relative worth compared to yourself. You swerve, or brake, or generally do whatever you can to not hit them. And so will the self-driving cars, except better. In fact, I think we can avoid the moral issue altogether by just focusing on making it nigh-impossible to surprise an autonomous vehicle in such a way that it won't be able to reach a safe state[0] in time. After we have that, then we may consider optimizing the algorithms to take into account that e.g. the other vehicle is empty, so its suitable to be used as a braking device.
[0] - Either avoidance trajectory or a speed that would not be likely to kill a pedestrian if they were hit directly.
Well, not everyone at Google agrees with you. Yonatan Zunger seems to think that yes, you do have to work out what the car has to do in certain worst case scenarios.
'A self-driving car, on the other hand, would require that these decisions be specified in advance.'
https://plus.google.com/+YonatanZunger/posts/FMdAD6ckH38
And well, he's right. Once you have programmed the car, it will respond deterministically to inputs. You can close your eyes and ignore the consequences of the decisions you made for the Trolley analogs, but that's willful ignorance of your decisions at that point (and bad QA).
This is true, but that doesn't mean that the response is based on the car making moral decisions about the value of the human lives involved.
As the commenters below mentioned, the technology doesn't even exist at this point to do anything other than avoid the object in front of you and try not to crash while doing it. Sometimes this may result in a dead pedestrian and sometimes this may result in a dead occupant. That isn't the car making a moral judgement, it's just collision avoidance.
But the human designers of the "collision avoidance" algorithm will design it with moral judgements in mind. E.g. what to do when face recognition shows 10 kids in from of you and you have 1 passenger in the car that if you turn you'll avoid hitting them but risk his life.
You'd be surprised. Actually even if they don't, they can (and probably will) be regulated to do so.
For example, to use ultrasound to map the hardness of nearby obstacles and go for the one that provides the quickest safe deceleration, or to orchestrate a decelerating glance with an oncoming vehicle?
Okay, that's probably a very unpopular opinion, at least when expressed in that way. But to be perfectly honest, it's probably what a lot of people riding in said cars want to happen, and perhaps a bit closer to how cars work in real life when not driven by a computer.
Good luck trying to sell selfish AI to the public though...
I don't consider that an unpopular opinion, at all, thank you very much.
In reality, these questions don't take into account the fact that humans don't actually even make decisions in these situations. People don't choose whether to kill that child or swerve into oncoming traffic--they don't even react in time for there to be an option or the reflexively jerk away without thought. Truckers don't get time to consider whether they're going to kill other people on the road with a jackknife or roll when they have to do a sudden avoidance maneuver.
Autonomous cars are going to be so much better at avoiding situations preemptively that you will have to be pathologically stupid to be able to put an autonomous car into such a situation. And, if you are that stupid, YOU deserve to die.
Now imagine the other autonomous cars following it have to make an urgent decision; go straight or swerve out of the way. Imagine the only somewhat clear path has some innocent people on it, and the alternative is basically to let the car end up as a giant fireball after it hits whatever's in front.
You can try and avoid similar situations as much as possible, by having the car figure out what's going on ahead and do things to avoid it (change lane, slow down, etc). But the situation isn't avoidable all the time. If something happens close to you and you're going down a motorway at 70mph, then physics alone says the car cannot stop in time. It will have to decide then to either take the risk or get obliterated.
It's not pathological stupidity, it's often bad luck. And in cases like that, the driver is right to assume the vehicle won't sacrifice them for others.
People buy SUVs because they perceive them as "safer" -- but clearly that safety comes at the expense of passengers in other smaller cars.
Yes, that "behavior" had to be programmed in.
The autonomous vehicles had to learn to be "assertive" (ie. start rolling slowly into the intersection) in order to not confuse other drivers at the stop.
The vehicles also had to learn that a hipster standing on a fixie's pedals rocking back-and-forth was not an intersection hazard in spite of scoring high on the idiot detector.
A funny story I read about once:
> I've got a good story about the mines using this tech. A mate works for them and was telling me after changing over to driverless trucks they started getting huge divots in the ramps in the mines.
> They went to site and watched the trucks to see what was going on, and the trucks were (with centimetre precision) changing gears in the exact same spot causing huge divots in the ground. They ended up having to program in a random number generator in that algorithm to avoid them.
Source: https://www.reddit.com/r/Futurology/comments/3p7qg6/driverle...
> A taxi rounds a blind corner on to an inner city railway crossing. Half a car's length across the rails is a small child who's fallen off their tricycle. A steaming locomotive is barreling down the track. If the driver breaks in time, your vehicle will stop short of the child but be torn to bits as its shuttles down the tracks in front of the train. If they maintain speed, you'll just make it past the train -- but the child is not likely to survive.
When there are no barriers installed we're already talking about a different situation and all cars should stop and confirm there is no train approaching.
The reasons I thought of:
1. As mentioned - building/upgrading infrastructure is probably much more expensive and logistically elaborate.
2. Market penetration - current batch of driverless cars can drive on existing infrastructure - no need to wait.
3. If smart cars won't be usable dumb roads it will also be hard to sell
4. Smart, durable, cost-effective infrastructure hasn't been solved.
It's hard enough to get a government to keep roads pothole-free, how much harder is it going to be to get them to build "smart roads" when they have little or no incentive to do so?
I can't imagine today's pedestrians assume a driver will attempt to preserve their life over the lives of the people in the car. So as a pedestrian you take measures to avoid putting the driver of the car into those situations.
I think it's reasonable to assume a driver's objective is self preservation and preservation of his/her passengers before preservation of people outside of the car.
I can certainly imagine a scenario where a person driving by themselves might make different decisions because they can't imagine living through an event where they were responsible for killing others, and didn't do everything they could to prevent harming others. But then what if there are passengers? I have a dilemma of attempting to preserve others' lives in the car and outside of the car. Are we honestly going to ask our drivers to "rate" each passenger? ie. If Joe is in the car with me, I could care less if he survives a horrible crash because he slept with my wife, so you can feel free to kill us both in extreme circumstances. If Mary is in the car with me by all means do everything you can to preserve her life. She has kids and a husband and is happily married.
The objective of the driver (and thus driverless vehicle) is to transport the driver and passengers safely. They shouldn't attempt to actively reduce likelihood of achieving that objective in order to preserve non-passengers.
It also doesn't really work outside the US, since a lot of places are quite fine with 'jaywalking'. For example, in the UK, you can cross the road just about anywhere. These autonomous vehicles will have to programmed with this in mind.
For example, we all just know not to step in front of a train. We have NO EXPECTATION it can stop. Do accidents happen? Yes, but only at the fault of the pedestrian (intentional or not).
If we all know that every car won't stop for us in between crosswalks, we may still jaywalk but we'll be danmed sure a self-driving car isn't coming. Right now a lot of people jaywalk and are overconfident if a car is coming it'll stop.
A schoolbus full of young children has lost its brakes and is careening down a hill toward an almost certainly lethal high speed impact with a building at the bottom of the hill.
Two Google cars are idling at an intersection halfway down the hill. Both determine that if one of them pulls in front of the bus just so, it will result in a survivable crash (for the kids on the school bus). But which car should go?
Both cars run a PANN (psychometric artificial neural network) analysis of their owner's email. Car A determines that its owner is an abusive sociopath. Car B determines that its owner is a nice person, newly wed and possibly pregnant. The each compare their owner's PANN scores.
Car A signals "CARMA program activated. Taking one for the team!"
The schoolbus plows into Car A, which absorbs most of the impact. The kids are fine, minor injuries only, but the owner of Car A expires before the fire department can extract him with the jaws of life.
The law distinguishes between deliberate actions and accidents in which there isn't time to decide. But a car will not have the luxury of a jury understanding that it panicked.
"I panicked" is not an excuse, you need to control your vehicle at all time. You should never be in a situation where there isn't time to decide, you need to adjust your speed and safety distance for that.
Almost feels like plagiarism.
A computer will rely in an algorithm, and most important than that in multiple sensors, not just their own sensors but also other autonomous cars sensors.
And there is a huge opportunity here to create even more sensors around crossways or sense people cellphones or bikes, or static speed sensors to warn other cars, creating tons of information for the car to know where cars, bikes and people are.
I see this as a remote scenario if we start building more sensors on top of the current highways infrastructure.
I think in general the cars will be programmed to avoid any damage.
But what if a company starts providing rides that promise no injury will happen to the passenger?
That's why laws are needed to prevent this from happening in the future.
Side note: in The Netherlands we have an automated sea barrier [1]. The system is made automatic so nobody has to make the decision to close it. So yeah, some systems are programmed to save peoples lives even if this means the lives of others can't be saved.