Tesla seems to be making progress on full self driving, but real self driving (cars without steering wheels) still seems quite distant. How will such a car respond to road workers redirecting traffic? How many times have you had to talk to someone outside of your car to obtain instructions on how to get around some obstacle, like a moving truck. I don't do such things every day or even every month, but there are occasions where it is necessary to take unmarked detours.

How would a self driving car get through heavy fog or snow covered roads when there are only very difficult to decipher hints to the road edges?

How will self-driving cars deal with humans that can perfectly predict their actions? I believe that bad drivers will take advantage of self-driving cars by cutting them off and failing to yield when they should.

Because human drivers have a sense of self preservation we can break the rules when we are about to be car-jacked or see an impending collision coming. Imagine how easy it is to obstruct a self-driving car for nefarious reasons.

I'm confident that, eventually, self-driving cars will address all of these issues to some degree, and that will mark a turning point where it is better to leave driving to the cars than average drivers. However, this doesn't mean I'm going to put my $100K car into a taxi pool to be used by (sometimes drunk) strangers; I don't buy the argument that the cost of buying such a car will pay for itself.