You still need ambulance access everywhere and delivery access almost everywhere. So you need a lot of vehicle infrastructure even though it’s mostly unused, and vehicles must suddenly share the space with pedestrians instead of pedestrians having sidewalks and designated crossings. Pedestrian streets exist in most cities but the problem of making each street a safe place for pedestrians while ambulances and delivery trucks can still access them isn’t a solved problem. A couple of vehicle terrorist attacks in Europe in recent years highlighted this issue.