You don't have to have the cars being "driverless taxis" to be more efficient in use of roads, there are lots of other options that might involve sharing cars with several neighbors, that the driverless feature makes much more feasible.

While there may be some people who use driverless cars less efficently than they use cars today, I have no doubt it would be more than balanced out by people who would be glad to save some money and don't need full ownership of a car.

I really don't see a scenario where cars are circling for hours in busy parts of the city. At the very least it could be smart enough to go wait somewhere out of the way. At least in US cities, people wouldn't tolerate that if it was significant enough to make a difference.