Yeah, self driving cars may work in Arizona but how about rush hour Amsterdam or New York? In my country it might work for long haul trucks on highways in the near future. But I dont see a use case beyond that.
We can't assume the pure chaos of Cairo or Delhi requires the same model. Self-driving might only need basic Roomba-level collision avoidance and basic pathfinding for a place where the humans navigate with the expectation that every object, moving or not, is a collision waiting to happen.
Think about how much of rules-based driving is about avoiding hitting people who aren't following the rules. If you start off assuming there are no rules, the model is simpler: everything the sensors pick up could come flying at you at any moment, so plan accordingly, and stop or slow down if you can't find a path that avoids an impact.
In any case, I don't think we should be assuming infrastructure will remain the same. If self driving does replace driving, the infrastructure will change. Point problems (eg, computer can't handle these 43 tricky roads or intersections, those problems will be solvable be changing the road. It might just be adjusting so gns or minor parking rule changes.
There's a chicken-egg problem, which is why getting some of the way on existing infrastructure is smart. It also avoids scaring finance ministers, but ultimately infrastructure will meet driverless cars part way.
That said, we redo intersections because they have a lot of accidents all the time. I'm not sure why intersections that self-driving vehicles have particular problems with would be any different. I can also imagine having some sort of RF "beacon" that gets used for construction sites and the like to flag drivers that they need to take over manual control in a mile or whatever.