Debt isn't a problem in itself, but I don't feel like there's enough support for people who find themselves in rough financial situations. Dealing with money and debt isn't that hard, but you need to understand it and I think there's loads of people that don't.
Yes, if I was having a bad day, being asked for some change for food could be a nuisance (in particular the twelfth time the question was asked while you were already running late for an appointment!) - but overall, when living in a city, the people who on occasion made me feel unsafe didn't appear to be homeless.
Just out of curiosity - if being homeless is sufficient grounds for not being allowed to frequent public spaces, where do you suggest people so affected gather? After all, by definition, they haven't got access to private spaces, no? (Yes, this is a somewhat snarky question, I'll admit.)
Been homeless for a short time myself and was connected with the "scene" before and after that, homeless people are a quite peaceful crowd and often show more kindness towards other people and animals than "ordinary" people.