The other problem is that even if you have car that can do 95% of the driving and only need 5%...

...that makes the human driver, who normally would get say 10k miles of practice every year will drive... 500 miles on average.

It really does need to be "car can do all of the driving", or else it will be putting people that have zero practice in those hard situations it can't handle.