The problem is that whenever a human driven car reverses over a toddler or wraps around a telephone pole or veers across lanes into oncoming traffic or blows through a red light and T-bones someone, nobody cares because it happens all the time, but when a self-driving car does, murderbot outrage makes frontpage news all over the world.
So even though we rationally should immediately go all in on automation as soon as self-driving cars are on average safer than human ones, realistically this is politically completely unpalatable until they reach a much, much higher, possibly unachievable, standard.
It's not "fair" that this is the case, but unfortunately humans are really bad at accurately assessing the risks of low probability incidents (eg. dying from terrorism), especially if they get a lot of media airtime.