Then they should be way more worried that their government is using any access to their wifi router at all. Like, you got way bigger things to worry about at that point, right?
There are a million surveillance side-channels that could be used to build profile-able information about what is happening inside a home from the outside. The concern isn't that those things are possible (they always will be) the concern should be that there are authorities who could be (mis)using such avenues and not explicitly being disallowed in the first place.