Think about it this way: the two main ways people are harmed in relation to drugs is via a) addiction, and b) the illegal drug trade. The latter only exists because of prohibition (ok, even if all drugs were legalized, there might still be an illegal trade, but imo it would likely be a b2b black market rather than street based retail, so the violence would still be significantly lowered). As for addiction, it's thornier, obviously many people are addicted to legal drugs, but the criminalization of addicts severely exacerbates many addicts problems: they don't seek medical help due to fear of legal repercussions, they're forced to interact with illegal drug dealers, they often hide their addiction even from their friends/family because of the social stigma attached to criminality.