I'd think people would be happy that at least some remote work will never go away at this point. To argue that you should never ever even have to consider going back into an office for any amount of time is just silly. But to also argue you can never ever work from home is also silly.
Most managers were people doing stuff who got promoted into 'managing' stuff with no obvious output or direct value. It's not really surprising that a chain of these people would knee-jerk react weirdly in a situation they felt exposed as not being that useful / have their limited people a management skills tested in a situation they themselves likely didn't experience.
And regardless, that sort of thing happens even when employees work in an office. People can certainly get a part time side gig and do that work from their main job's office. Maybe a little more likely to get caught, but it's far from impossible.
One common argument is if you can’t tell their performance is impacted, they must actually be performing adequately. Except, in a different context, we will happily explain how our own performance can’t be measured by number of lines changed or bugs fixed.