Managing a lot of people at scale is messy and you have to use crude solutions. It's impossible to know everything that's going on.
If you were a manager you wouldn't do any better. Out of the crooked timber of humanity, no straight thing was ever made.
But more specifically ICs tend to want to say "if you just let me do what I know is right it would be fine." That's a trade-off, too, though. That solution means a lot of people will be messing around due to no accountability.
If setting the actual goal you want to achieve as a manager, and then trusting employees to allocate their focus accordingly, you will absolutely have people faffing off (as likely can't be avoided), but at least those who don't will optimize towards what works according to their actual expertise at least.