The non-visible homeless (people down on their luck, to an approximation) get mentally sorted into the vast category of "that's unfortunate but it doesn't affect me directly," which might influence people's votes or charitable donations, but doesn't arouse much passion for most of us.

By contrast, the visible homeless are a visceral inconvenience, even a danger, so people feel strongly about how that problem is addressed. As a smallish woman, I am on high alert in certain areas of San Francisco, Oakland, etc. I'm not all that likely to be accosted, but if I ever am, it will be a big deal, so navigating those places is nerve-wracking because my threat-detection is constantly dialed up.

I would compare it to, like, road planning versus traffic enforcement. The former has far-reaching, long-term impact (like housing policy) but the latter is what gets people heated, due to the immediate impact in their day-to-day lives.