I think the main reason isn't the shelter itself, but that being around a large number of fellow homeless people is dangerous because of the rampant mental health issues, drug abuse, and violent crime.
The problem is similar to what happened when low income housing (the "projects") concentrated the poor into economic ghettos.