It will be a great day when human drivers are banned. Humans can't handle driving, and they cause 40,000 deaths per year in the US alone. Computers will be able to do this far better.Sure!...if we all actually owned our machines and the government wasn't using the mechanisms through large corporations to spy on us and control us. The thing is, many people in the government and large corporations are going to try and do just that, because it is in their short term interest to do so.
Assuming we have driverless cars in the future that really are extremely reliable and safe, then I don't see what argument you can possibly make to allow human drivers, other than "I like driving".
"I like driving," should be enough. Maybe there should be certain regions where you can still do it, but busy city highways would be restricted. In any case, it's not so much driverless cars that are the problem, so much as cars that are controlled by powerful organizations with interests that aren't the individual and individual autonomy and freedom of movement being taken away in a practical sense.
The actual harm caused by the present state of things really isn't very much, if anything (can it be quantified?)
This is how tyranny generally goes. At first, the Soviets were bright eyed and idealistic about the project of building a new society. Heck, even Germany in the late 1930's was like this, with progressives in the USA singing the praises of the "progressive" policies and energy of the new regime.
By contrast, driverless cars (at least the hypothetical ones assumed above) would fix a big problem, and would have very few downsides I can see.
No disagreement, again. The question is, will this eventually be implemented in a way where everyone is going to report in to Big Brother and ask permission to take a trip?