> they are inconvenienced by the sight of poverty

Or, you know, wake up to find shit on their driveways, smashed windows on their cars, needles thrown on the street, etc.

I'm assuming that you have the privilege of never living near a large homeless encampment. It really does become an all-encompassing matter for people in the area. You're reminded of it every time you breathe in air that smells like diesel and burnt aluminum foil. When you see tents at the very intersection in this article, while the bridge housing 2 blocks away has 50+ empty units due to lack of interest. Every time you go to the library and there's someone in a trenchcoat furiously masturbating. When you're followed on a bike ride by someone who runs a chop shop, looking to see if you're an easy mark. When you're walking down the street and someone screams racial slurs and threatens to kill you.

Rationally you can think "that's just the mental illness or addiction causing someone to behave that way". But over time, even if you rationalize the behavior it wears down on your empathy.