Self driving cars have been driven up and down various parts of the SF bay region for several years now. Google's self driving cars have driven over 1 million miless. They report that their vehicles have been involved in 12 accidents, none of which are the fault of the vehicle's self driving capabilities.
The only reason they will appear to 'pop' into the consumer market overnight is because this technology is politically charged and simultaneously so important to society that it is being overengineered to improve success and safety. The moment self driving cars has its first accident after being released to consumers, the media will likely flip, ignoring mountains of data demonstrating how much more safe these vehicles are to provide a sensationalized report on the dangers of self driving cars. The safer they are and the more data backing this up, the less likely this will derail the technology.
1 million miles: https://plus.google.com/+SelfDrivingCar/posts/iMHEMH9crJb
12 accidents: http://www.latimes.com/business/la-fi-google-cars-20150603-s...
CMU ALV: https://www.cs.cmu.edu/afs/cs/project/alv/www/index.html