Exchange of money and goods is not "communication" in the sense of "free speech" or "privacy". The state needs to protect my freedom to not be killed by a hitman, be threatened by guns etc.

I am all in for legalizing drugs, but don't pretend that all weapon trade, offering hacking services or other nasty stuff needs to be protected in a free society. Freedom of speech is about inconvenient opinions, not about "everything you can do by communicating with others".