Safety in numbers. There's a reason there's not an issue during the day and during heavy commute hours.
This seems intuitive to me because I would never risk fighting someone in the street over some crime, so I don't see why having me around would deter them at all. The same goes for maybe 90% of the people I know. We're weak and docile, even in numbers.