The whole thing gets interesting when you look at the judicial side of it: Who is responsible in the case of an accident? The car manufacturer (if it was a software failure)? Then even one accident triggered by it is too much.
The idea that 'even one accident is too many' is a joke when you consider how many accidents there are with human drivers, and yet we accept them willingly. If there are 30,000 deaths per year from human driven accidents, then surely the real answer to how many accidents by computer driven cars is acceptable should have an upper bound of 29,999, right? Why isn't it acceptable if it saves even one human life, rather than being unacceptable if it costs one?
Unfortunately, we as a group don't tend to be very logical, and love to spread blame, so I am afraid we'll take a long time to start saving lives. The data on the self-driven cars on the road is pretty overwhelming that they are safer than human drivers, but unfortunately there will be those (as in this thread) who believe that is true of everyone but themselves.
Its going to be a long road to acceptance.
But there's two problems here:
1) We won't be replacing all the cars overnight. The problem comes with the interaction between terrible human drivers doing wildly unpredictable, insane things and the inflexible, unadaptable automated cars.
And
2) I'm not comfortable with dying or being injured due to a software error regardless of whether its likelihood is higher or lower. The roads are far more dangerous than the air, and this is why I'm uncomfortable with the whole concept of automated cars on the road and I'm not uncomfortable with autopilot in planes.
Why on earth is dying due to a software error worse than dying due to a human error if you can chose one or the other, and the chance of a software error is lower?
There are only so many things bad human drivers can do, the physics is pretty limiting. It isn't like they can teleport in front of another car. Even the craziest things like opposing traffic jumping the median into oncoming traffic isn't all that crazy, it isn't conceptually different from a deer jumping in front of a car.
I'm not comfortable with dying or being injured due to a software error regardless of whether its likelihood is higher or lower.
Hard to argue with that.
I was thinking about your assertion that it's easy to solve, and why exactly I disagree with it so strongly on a base level. What am I basing this belief on?
And I figured it out. Driving games.
The AI in driving games like Gran Turismo has had more work than Google or any other company has done on theirs. And not only that, the AI in driving games has perfect information of the physics involved and the environment around them.
And those AIs are still fucking terrible when interacting with human players. Pack of AI driving around? Just fine. Add a human driver to the mix and it turns into a disaster.
This is why I am very concerned about automated driving in the real world alongside human drivers.
As for driving games, those have a different set of requirements and incentives. Driving poorly isn't necessarily a bad thing from the position of game play.
It would be at least as difficult, or less so (but not more so) to program AI to interact with human drivers with the primary goal being safety, not speed and exhilaration.
Those behaviors are easily improved upon by software. Dealing with lane changes is something software is likely to be vastly better at, because they can point sensors in all directions -- I only have one set of eyes.
Furthermore, a lot of horrific accidents are because of bizarre unexpected things that the human does not react to in a timely fashion precisely they are so far out of the normal scope of traffic flow.
For example, what if a car coming the other direction drifts in front of you on a rainy night? This is an unfortunately common and very lethal kind of accident in rural areas. It will take you 1 to 3 seconds to interpret the scenarios and react appropriately. (You only get 3-4 seconds before your are killed.) A computer can instantly recognize that car 100 yards away is maneuvering in a potentially lethal manner and begin slowing down right now. That buys more time for everyone.
Engineering solutions are a tiny part of getting this stuff to market.