Google self-driving cars have caused accidents, although Google claims the self-driving feature was off at the time.
I'm more concerned of what these cars will do to highway capacity and therefore traffic congestion due to the excessive lag/follow distance their algorithms impose. While they should be following at < 1sec lag due their super-human reflexes, what you actually see (http://www.youtube.com/watch?v=TRIOE1IZrq4) is sub-human follow times and it's apparently the only car on the road that's frequently pumping on the breaks (which consequently makes following cars increase their lag time as well).
I've also read that they won't exceed the speed limit, which is significantly more dangerous than driving with the flow of traffic. I hope it knows enough to move over when being overtaken from behind. If you're driving 65mph on I-280 between San Fran and San Jose with no traffic, you are the slowest car on the road by between 20-40mph. The road has long straightaways, often 1+ miles of visibility, well paved, and I'm often overtaken while going 90mph.
I expect that the experience of having to share the road with these devices, as they start to become more common, will transition from, "Gee that's cool" to something more like "Geez, get off the road Goorama!"