If the car is driven by a computer at Google, then it can be "hijacked" by the computer. A few years after mandating that everybody have self-driving cars, the argument will be that certain roads don't need to be in the system because there's far too little traffic. After that, it'll be argued that Google should be able to divert you away from certain areas of town where there's been high traffic or other hazards, without asking you. Then it'll be argued that the government can mandate certain routes in order to preserve the environment or some such excuse. Then it'll be seen as entirely reasonable that all auto "routes" be managed by city planners who want to control noise, pollution, etc, for your own good. It'll be incremental.

Fortunately, as you say, there are still a majority of sane people who vote in the USA. Will that still be true a generation later?