You will not know how safe they are until accidents happen. I would love to see someone with money run an experiment with real drivers vs google cars. Create a worst case scenario course with simulated drunk drivers, blown tires, missing stop signs, etc.. and run this with real and fake drivers. I suspect the average driver would crash 10% of the time, autonomous car would crash 1% of the time, and a professional driver would crash never.

The real safety improvements will come when all cars are autonomous and they talk to each other.