That rule was put in place in the early, analog, days of cell phone usage to help build confidence that J Random Somebody wasn't listening to your calls. Cordless phones at the time were well known for being easy to listen to, and the wireless industry didn't want that stigma on cell phones.

What harm does spying on my neighbour through her window cause ?

If light travels through my telescope into my house I'm not allowed to watch it ?

Seriously just because something is "in your house" does not mean you some inherent right to it.