So, it is about wanting to assert control. Because they think "abusing" remote work is when workers have more control of their circumstances and don't have to break their backs or brains for their salary.
You can say "well that's the sort of thing you should catch in a performance review", but that's more-or-less isomorphic to hiring anyone who applies to your job and relying on performance reviews to fire the bad ones. I think people can intuit that that approach would not work very well.
If you want to frame "being worried people will not do their jobs, or will do them worse than your expectations on hire" as "asserting control", I suppose you can do that. But surely an employer has at least some legitimate right to try to ensure that their employees are doing the job they pay them for (notwithstanding the broader economic undertones to the employer-employee relationship, which are a much larger issue that extends way beyond RTO).
Some work might be harder to quantify, but... for a lot of work, there's deadlines. If you need to get a UI screen working to some spec by Feb 1, and Feb 1 comes and goes and it's not done, isn't that some indication that work is getting done or not?
Do they really care if someone 'does the job' or 'delivers the results'? Sometimes there's not much difference, sometimes there's a huge difference.
And why Feb 1? Who decides what the reasonable timeline is? How sure are you that they aren't playing political games? How do you measure their performance? Did the engineer abandon some on-call thing to get it done? Did they pull a more senior person off of their job to help? Did the manager who gets blamed for all this choose who to hire?
Management isn't binary in this way, and when managers try to make it binary, a lot of people (rightly) complain. And I'd bet that many of the people who complain about that are exactly the same people who are here arguing for remote work (not least because I am in both groups).
Quantifying work to such an extent that you can detect any slacking or poor-quality work is one of the fastest ways to make it horrible for employees. Unscrupulous employees abuse Goodhart's Law to hell and back, scrupulous ones get punished for doing important work that didn't make it onto the quantified metrics, and work becomes more about covering your ass than it is about getting stuff done.
Feels like a fringe belief here, and only really feasible in flat, lean orgs with semi-technical stakeholders and no BSers. [So probably <0.001% of tech industry currently]
But it isn't. There's a reason that every company wants that top 0.001%. Employees who give a damn and can be trusted to get things done effectively without supervision or managerial pressure are rare. Even at the best organizations, they're often a minority. At weaker organizations, and (typically) at older and larger ones, they're somewhere between "very rare" and "totally nonexistent".
If you're the latest highly-funded startup, maybe you can pay enough and create a good enough work environment to attract that person. But what if you're a random 30-year-old contracting firm in Overland Park, Kansas, paying $85k a year for software engineers? Do you think you'll attract that vanishingly rare talent? Can you rely on the idea that all of your engineers are so motivated and so skilled?
If you want to argue for that level of managerial hands-off-ness, you can do that. It's a legitimate managerial philosophy, and it might even be the right one! But I think it's hard to deny that many people don't think that's the right managerial philosophy, and that's all that they need to believe to favor RTO without any particular malice.
My point isn't to argue for RTO. Again, I run a remote company, and if you ask me, I'll tell you your company should probably be remote, too (depending on exactly what you do). My point is to argue that people who are arguing for it need not be doing so out of any particular malice.
You don't have to take off all guardrails which prevent abuse. You can still have a rules based office fully remote