This seems awfully reductionist. I think often the intent is more along the lines of- "We want to live in a place where kids can walk to their friends house without an adult escort and/or play kickball in the street. We've observed that this becomes unsafe in places with high density populations, so lets prevent high density housing to maintain that lifestyle"
There are lots of poor neighborhoods that are single family housing (the entire south side of Chicago comes to mind). The issues greatly impacting San Francisco should not be generalized like this- most places do not have them.