What's not okay is for me to build a house and add recording devices all around and then sell it to you without informing you. Using your house example, that's the most direct comparison and would 100% be illegal.
There's obviously a trade off most users are willing to make between privacy and functionality, but I do believe the exchange should be 100% in the open and a conscious decision made by the user.