Not really. The cars only need to be safe enough to get over the potential fear of users to activate them. This may even be
less safe than human drivers, we just don't know until they are deployed. For example, it could turn out that present day autopilot in Tesla cars is less safe than human drivers, but yet people still use it. Humans aren't strictly rational based upon risk when it comes to transportation. If autocars are convenient, and in every day experience work successfully, people will use them (especially those who are young, who have the marginal cost of having to instead learn how to drive, and may just opt-out of learning to drive altogether if they can get through their daily lives by using autonomous vehicles.)
The legal issue you mention seems secondary to the human factors one. If people get addicted to self-driving, and some are dependent upon it, the laws will bend to the culture not the other way around, as they always do. Look at the current regime for an example of this: we gloss over the 30k fatalities that happen regularly due to cars, and hand out licenses to drive these death machines to people who really should have no right to do so given their poor driving skills or physical impediments -- we do so because to deny someone the ability to drive is so averse to culture and freedom that we err on the side of enabling more people to drive despite the fact that it surely increases the risk for everyone on the road.