Hacker News Reader

"The obvious solution is to program the car to never take any blind corner, mapped crosswalk or not, so fast it can't stop on a dime."

In general, that's right. There is no reason for the vehicle to out-drive its unobstructed sensor range.

One effect, which the Google cars already demonstrate, is that their behavior at corners with obstructed visibility is markedly more conservative than when the LIDAR can see down the cross street. That's a good thing.

Having once coded this for a DARPA Grand Challenge vehicle, that's how you look at the problem. You're always plotting stopping distance against the map constructed from sensor data. You figure out how far it's safe to advance given sensor data, and compute a top speed from that. We did this every 100ms. If the map wasn't filled in with new safe space into which it could advance, the vehicle would slow down and stop at the end of the safe area. So it never out-drove its sensor range. Remember, this was for off-road operations, with obstacles and cliffs; we had to be very conservative.

Humans tend to get into a mindset of not wanting to slow down. Software doesn't have that problem. Constantly having a plan available for an emergency stop is not a problem for software.

That's not really the point - knocking down that example doesn't knock down every possible scenario where uncomfortable questions come up.

We were talking about the Trolley Problem at a recent dinner with friends, and one of the guests remarked about how it was like Star Trek; "The Needs Of The Many" versus "The Prime Directive".

So it's a general argument about where a car has the capability of making an uncomfortable utilitarian choice (The Needs Of The Many), or of simply shutting off any ability to affect matters and let nature take its course (The Prime Directive). The uncomfortable bit there is that with the latter, there's still that actual choice (by the designers or whoever) to follow the Prime Directive - it's arguably not "natural" anymore.

The AI used in these cars is probability based. They can't be held accountable in the sense that they can't explain their decisions rationally. You can't actually predict what you'll get by slight modifications of probabilities when adding new test cases.

Gerry Sussman explained this recently way better than I can.

Your brain is also probability based and you don't have an audit trail for your thoughts written to a black box.

So what is really needed is to add post-hoc rationalization engines to these self-driving cars to facilitate satisfying our sense of justice.

That was actually my first reaction to reading about Sussman's AI ideas in an article[0] that was on HN recently[1]. Propagators seems neat, but the quote:

"Sure, when you're running the program, that whole thing is a black box. So is your brain. But you can explain to me the reasoning of why you did something. At that point, being able to inspect the symbolic reasoning of the system is all you have."

made me immediately notice that there's a reason we have negative connotations with the world "rationalization" - because most of the time, post-fact rationalizations are bullshit and do not reveal one's actual thoughts at the time of the event, but what one wishes one thought at that time. Just bolting a "reasoning explainer" on top of a Bayesian inference system won't fare any better than human rationalization does.

[0] - http://dustycloud.org/blog/sussman-on-ai/

[1] - https://news.ycombinator.com/item?id=10388795

I think the auto/software makers should take responsibility for it, though, as some already have said they will. Just like when you have bad brakes in a car, it's the auto maker that is responsible not the supplier of that component. Also, they can't just say "it happens, what are you going to do?"

I think it was more of an example of an imaginary emergency situation. You can hard program the corners, but you can't successfully predict all kinds of emergency situations that may arrive. Another example in that article is better: A rogue motorcyclist just turning blindly in front of you. Maybe, the only two choices are either you run them over, or steer your car off the road.

Why are you so close to the motorbike that if he stops, you'll collide with him?

If he's coming out of a blind sidestreet, why are you approaching so fast and with so little lateral room that you can't dodge a surprise motorcyclist?

These are all human patience related problems. An auto should be perpetually correcting to make sure that it's current minimum collision avoidance distance is smaller than it's current visual clear zone.

So every time there's oncoming traffic (or a person on a sidewalk nearby) you'd have to stop until they were past, just in case they decide to turn/step in front of you.

As long as there are other actors in the world who may behave irrationally I don't think you have to work too hard to come up with either/or scenarios for the AI to deal with.

And even if the irrationality is removed there's still a chance of mechanical failure (e.g. tire blowout causes loss of control) which other AIs may need to make decisions on how to avoid.

I'm wondering why in that scenario the car doesn't honk the horn at all? Are all those children death and blind? No adults are supervising those kids either? I just don't understand. I hope the safety standardization of autonomous cars is a joint agreement that involves the companies involved, and the public, especially if a real scenario occurs...

They can be deaf and blind if it helps push anti self driving agenda.

Even if a self driving car runs over a pack of school children daily, it will still be less fatalities than humans cause.

I'm much more worried about self-driving cars getting hacked than "whether it will decide to kill me" in a scenario that is only going to get less likely as more and more self-driving vehicles are on the road.

This is my entire problem with this bull shit question.

It applies human limitations and distractions to a computer.

The scenario will never come.because the autonomous car sees everything with way greater clarity than any human. And it will never miss a person crossing the street because its rubbernecking at am accident or gawking at a hot girl or answering a test or trying to turn up the radio because its favorite song is on.

   There won't be a blind corner as the thousands of other autonomous Google cars passing by would already be napping and sending live data.
 Probably even some of the children crossing the street will have Google devices aw ding the data to Google and through to the car.