If that's the criteria then we still have an inconsistency: tobacco and alcohol should be banned. The alternative is to be consistent in the other direction (which to some that are after "freedom" sounds more appropriate).
The drug prohibition made it difficult to know how many people use drugs responsibly. It is linked to legal weed actually: it is being legalized not because research showed it is not too dangerous (we knew it for a while), but because more and more people become aware that you can totally use weed and remain a respectable member of society.
For me it is a question of personal freedom. Everything has risks. Going hiking in mountains can kill you, for example, or almost any outdoor activity really. And some people die there pretty much every day. But I really don't want the state banning it just because some people fuck up.