Not OP, but I'd use externalities to guide the decision. After all, it isn't the drug users partaking in the quiet of their home that we really care about. It's the junkies on the street making life worse for everyone else around them, the smugglers, the violence, the people dying from overdoses, etc.

Of course the problem is that almost all of the drugs in question have legitimate medical uses so we can't just wave a magic wand and make them go away.

I'd personally try the angle of making drug use completely non-criminal (but not the associated behavior problems that do impact other people; this is a non-trivial element), and then make dealing drugs punishable with a LOT of time.

For most people it’s “get rid of the drugs I don’t like but keep the drugs I like.”