The basic thought experiment is this: "If the robot car that hit me on my bike did so with provably the best intentions, should I still be angry?"

At least with a person driving, I can criticize their thought patterns, their reactions, their choices--I can laud them for dodging the stroller, or curse them for drinking and swerving into me (provided I can still talk).

With a machine, though, we lose any chance at morality. They act exactly as programmed, and we all know how reliable programs are.

If, following such an accident, I were to be in a wheelchair for the rest of my life, I would want the entire stack trace that led up to my collision. I would want the unabridged source code and issue trackers. I would want the core dump. I would want every scrap of information that went into the vehicle's decision to zig instead of zagging.

I'm not sure that we want to hand over so much blanket responsibility to black-boxes designed by committee and implemented by antisocial 20-something engineers working weird hours.

EDIT:

Reflecting on this a little has suggested, perhaps, that we should have one standard algorithm and package for doing this sort of thing, open-source and in the public domain.

There shouldn't even be a question about allowing closed-source autonomous vehicles on the road.