This about it this way. Would you buy a self driving car that drives worse than you do? Even if it drives better than the average, an average brought down by drunks and the elderly? I know I wouldn't
Everyone likes to think they're better than average (https://en.wikipedia.org/wiki/Illusory_superiority#Driving_a...).
I would rather keep the worst drivers off the road than keep the best.
A machine suddenly hitting a bug is whole new ball game though. I guess over time maybe you could build up a sense of where machines have trouble driving and adjust accordingly.