This only works in pure anarchy where behavioral negative externalities are not paid for by others and people do not care whatsoever how much other people suffer, which is defensible, but not realistic.

Also a freedom is not neutral. Legalizing all drugs signals that it's OK to consume them on a certain level, and you run the risk of endangering an entire population. Would you really don't mind if your (18yo+) kids could just buy some crack at the grocery store next door? Furthermore, the concept of consenting adult is very fluid and dangerous. Do you really think an 18yo kid has the mind power to say no to his group of friends? Most don't. And what happens when people deemed underage want drugs? They certainly will, and they will have to go through illegal channels to get them, so you haven't solved the problem of drug dealing at all by legalizing for "consenting adults".