The borderline conspiracy theories on this are odd. If you're a manager, having a remote team is just harder.
It can still work well with a certain type of people, people who are self directed and take pride in their work, but let's not fool ourselves into believing that's the only kind.