I once taught a course and the first thing I was told was to look for a green glow on people's faces: it meant people were playing solitaire on the class computers instead of listening.
A whole lot of managers knows only the work equivalent of that: they've learnt simple proxies for actual management that let them think they know when people are working well from walking rooms (or having people reporting to them do so).
People working from home means they actually have to figure out how to track output, and that takes effort. Effort many managers are not used to.
I had to learn a whole lot to start managing remote and distributed teams. It takes away a whole lot of lazy shortcuts to management.
Lucky for me I was forced to deal with that many years ago, but it was uncomfortable and a big change, so I'm not at all surprised that many resist it now.