Bad press. Being killed by a machine is soo much worse than being killed by a fellow human. This won't be a problem. Yes, it will make the news, but if self driving cars turn out to be more reliable than human drivers, the numbers will show, and the insurance companies will follow. (Funny how talks about not controlling your car will die out in the face of monetary savings.)
Liability. Self driving cars are not the only thing that kills people when it goes wrong. Think airliner software. I think we'll get over it. Also, current cars already face similar challenges. I recall a sticky pedal problem that caused some Japanese cars to speed up until death ensues. As far as I know, unlike some unlucky drivers, the company is still alive.
Law. Not a big problem, as long as self driving cars are legal somewhere. Once they have demonstrated their worth, other states and countries will follow very quickly.
Privacy. It kinda sucks, but it won't matter one bit. If people really cared about not giving away their trip data to some for-profit corporation, they wouldn't be using Facebook in the first place. Or Gmail.
Price. It will inevitably go way down. Self driving is a software problem. A human driver could manage with only a couple web cams. So will programs, as they get better. Not to mention economies of scale.
Unemployment. Here be Unions. But they won't stop the advent of self driving cars. Driver-less companies will simply drive the Driver-full companies out of business. Unlike rail-road companies, they don't own the tracks. Now this will raise unemployment unless we do something about it. (Enforcing a 4 day work-week would be a good start.)
Overall, I expect all those problems to set us back about a decade, 15 years tops. But they won't even arise until we actually start selling those cars.