That's only half the picture. There's also the 10th Amendment, which says that powers not delegated to the feds belong to the states or the people. Federal drug laws rely on the interstate commerce power, even though they also regulate conduct that is sometimes purely intrastate and/or purely non-commercial. The most natural reading of the Constitution is that the feds have no authority here. The Supreme Court disagreed in
Wickard and
Raich, but
Wickard was poorly reasoned rationalization for political purposes and
Raich does not stand except by relying on
Wickard.
I mean, for all practical purposes, the feds will do what they want. But if you're going to cite the text of the Constitution, the feds are breaking the law here. It takes case law to justify what they're doing, not just the text.
Just a thought: Why did alcohol prohibition require a constitutional amendment but drug prohibition didn't?