Indeed, "if you apply General Curtis Le May to a situation and you get havoc, well, that s what you called General LeMay in for" (to use an Eben Moglen quote out of the context in which he said it); if you host a device in your home running proprietary software you ought not be surprised that it is spying on you and the proprietors determine what to do with that data, not you.
We must not forget another critical component of all this: these always-on listening devices run on proprietary (nonfree, user-subjugating) software. Therefore the user has no permission to: inspect what it does, modify it to do only what the user wants (or, since most computer users aren't programmers, get someone technical they trust to perform such modifications for them), run the modified software in their device, and distribute copies of the improved software to help their community.
If these devices ran on free software (software that respected the user's freedom to run, inspect, modify, and share) the more technical among us could help them. But as it is even the most technically-minded willing person cannot legally do this work to help them.
As Eben Moglen reminded us after the Snowden revelations came out: It's critical that we don't fall into the trap of saying something akin to 'those kids take too many darn pictures' like concluding that we just can't have these devices or their services at all. We can have all of their alleged conveniences but only if we have free software implementations.