The idea that selling drugs is something special misses and should remain illegal seems to only partially consider the implications of what that means.
One side of safety concerns is consumption safety. An at home production facility is of lower quality, exemplified by many more cases of methanol poisoning during the US Prohibition era [0]. Another is production safety, cooking methamphetamines is real chemistry —- I would much prefer to have a factory doing that rather than my residential neighbours. Finally, we have access, growing plants indoors is non-trivial and processing them into something resembling a finished product also so. Given this, I’m convinced that limiting illegality to trafficking doesn’t stop the war on drugs, because it has always been about trafficking — the hard part is moving the product and selling it to people. If the “do as you want” resolves to making trafficking legal, then it seems the drugs are simply legal.
[0] https://gardenandgun.com/articles/will-drinking-moonshine-ma... (The actual source is apparently a book the writer references)