There is a way to get the best of both worlds: variably-autonomous swarming vehicles. The larger the swarm, the more autonomy the car has. The smaller the swarm, the less autonomy.

Large swarms on the highway, small swarms (or solitary driving) elsewhere. Best of both!