Do you think that's unjustified from them?

I, as a fairly non-descript white guy, am always worried when out walking near a single female that she might be afraid of me and what I might do and take steps to help alleviate that (walk slower, etc.).

I mean, it seems like the most reasonable course of action. Sure, I'm never going to confront/assault/rape anyone, but the 5'2", 110 lbs woman walking in front of me on the street? She has no clue. How hard is it for me to slow down half a step and give her some relief?