But we don't know how to make human drivers any better, and self-driving cars are steadily improving (even if that's slowing down a little).
Maybe. But human drivers already do this way more than I would expect. I’ve seen emergency vehicles stopped and blocked by human drivers who are either paralyzed with uncertainty of what to do or selfishness (no way of knowing what’s going through their minds).
How so? We hear about the self-driving car incidents because they're required to write up reports whenever something happens (and it also makes for a good news story). That's not the case with human drivers.
It's also the least structured portion with the least clear rules than any other part of the driving task, so it stands to reason it would be worse.
My solution, would be to have free or low-cost self-driving (to keep the cost down) buses and ban cars including taxis. That should move a lot of people and clear out the streets.
From a European perspective I would say that in a city the real alternative is a functioning public transport system. Trams, light rail etc are infinitely safer and far cheaper than any form of car based system.