Half of what keeps NYC pedestrians on the sidewalk is the fear of being run over by cars. If pedestrians know they can just safely cut in front of cars whenever they want, I fear the average speed in the city is going to drop from 12 to 6...
Even without the fear of death, a lot of people don't want to cut in front of a car and get honked at and flipped off. But if you see that it is just an empty self driving car, with no human present to get mad at you, the calculus changes as to whether you cut it off or not. And that can have cascading consequences for average vehicle speed.