The federal government, if it was motivated enough, has the capability to bust your door down and hold you in a cell indefinitely with no trial if it wanted to. That doesn't mean they have a right to do so. It only means the people have allowed them to do it.
> all sorts of things we don't want as a society (money laundering, drug trade, human trafficking).
Oh, you mean the eternal bogeyman that can be used to justify any conceivable invasion of your privacy by government? Right.
What do you say to The Government requiring that your home and all new homes contain cameras and microphones to detect bad actors doing all sorts of things we don't want as a society, like money laundering, drug trade, and human trafficking?
Somehow I doubt you'd be on board with that, which is the entire point. You would draw a line in the sand because The Government does not have a right to do that, and those unsavory things that could possibly happen in your home are not a good enough excuse for them to subvert your rights.