It takes only one bad apple to spoil the entire barrel. (Literally: the bad apple produces ethylene which spoils the rest of the apples in the barrel).
Examples of this phenomenon are everywhere. One bad cop makes the entire force untrustworthy, two to three unruly kids can make a classroom unteachable, et cetera.
And a 0.1% chance of a bad encounter with a stranger as a vulnerable person can make it not worth approaching strangers at all.