There are tons of marginally-housed people who are just normal folks who couldn't afford rising rents. You just don't see them because they tend to stay in cars, motels, or couch surf.

The people you mention are a tiny visible minority.