Depends on the substance, depends on the market, depends on the reason for using. I haven't seen too many people on MDMA become violent or impoverished through addiction, though their incessant hugs and platitudes can lead to a violent response if they don't have enough to go around.
It's complex, but not that complex. We have the power of observation - where we can plot what is safe to use recreationally, what can be considered substance abuse, which substances often lead to abusive behaviour, which socio-economic factors lead to use in the first place, factors in the marketplace that lead to violence and crime, how we can monetarily/socially/legislatively abate the negative factors...and yet here we are with our proverbial junk in our hand wondering what to do. Start being objective, use methodologies that we've used for years for other problems that aren't taboo. We've travelled to the fucking moon.
My non-objective opinion: Drugs are interesting. Drugs take the edge off of a less-than-stellar life. Drugs help the mind momentarily escape the trap set by forces who care little for your happiness and wellbeing. You can fix the latter, which is unlikely in our lifetime, or reduce the burden of being a drug user in the interim.