The ethical issues when a driverless car kills someone cannot be solved in a reasonable way.
The question is which city/country will be the first to implement roads for driverless cars only ?
Given that human-driven cars, trucks, cyclists are already on the roads and will be for quite a while to come, and pedestrians already cross it, you would have to build a whole new road network, with crossing points for human-driven vehicles/pedestrian traffic. Which is simply infeasible, both in terms of money, but also simply space, especially in built-up areas where the space is already fully utilised.
Which begs the question: How are self driving cars ever going to be mainstream?
This is the present in Honolulu, as of last year: a fleet of self driving cars, operating on a closed road where only driverless cars are allowed, transports over 3000 people per day. The service is called "Skyline".
Fine for getting around different areas of the cities, but it's not going to drive you wherever you want to go though.