My guess is people will change their expectations of what they can do with a car. Maybe it will not work on unusual terrain anymore. Maintenance procedures might also change and involve remote controls or other specialized types of control.
The only thing I can think that would sell me on the idea is a really low price point, e.g. sub-$10k to get a new, driverless, road-only vehicle. But I would only want that in addition to a "real" car.