I was under the impression that freeways were the easiest target for self driving. The lack of pedestrians, bicycles and unprotected left turns makes it much more predictable. It's hard for humans because the speed is much faster than human reaction time evolved for, but that's not as much an issue for self driving. Tesla for a very long time only supportted self-driving on freeways.

Neighborhoods are the complicated parts: they're filled with pedestrians, cyclists, kids, animals, and uncontrolled intersections.

AFAICT, a self driving car has to pretty much assume that every pedestrian might suddenly dart into traffic, it can't really tell the difference between a normal pedestrian and a drunk heads down in their phone.

But keeping speeds below 20mph opens up a lot of opportunities simply because stops are basically instantaneous at that speed and because car-pedestrian collisions under 20mph are almost never fatal.