You're assuming the best possible implementation of autonomous car will be the one which actually arrives on the roads. Given the daunting complexity necessary to even make that feasible,it would surely be the first system of its kind and scale without a considerable number of flaws due to bad programming, exploits, cost cutting, unforseen consequences, etc.
The more 'intelligent' and 'autonomous' the cars are, and the more of them there are on the road, the more likely it is systemic problems will arise. This is not a criticism of the idea of autonomous cars but just an inevitable result of complex systems interacting. Just because they're computers, doesn't mean they won't fail, or that accidents will be so rare as to be not worth considering. Airplanes almost never crash now compared to a century ago, but that doesn't make worrying about air safety, or the affect of automation on pilot competence in the event that manual control becomes necessary, a red herring.