Are you saying you fail to sympathize with them because those people are poor?
The U.S. have a reputation of giving everyone the same chance to make their own fortune, regardless of status. When you're thrown into prison, unable to pay back the debt that got you there in the first place, this is like the opposite of that reputation. There's no way out.
Whatever happened to "Give me your tired, your poor?"