I think one "secret" is that we have small-ish teams (3-5 engineers), each with a very engaged and technically savvy leader. I've known other managers to have up to 20 IC reports. I can see that kind of structure falling apart when remote in such a way that your underperformers start completely slacking off and you're too busy to even notice. It's way harder for that to happen in smaller teams where your team lead/manager is very engaged in the team's work and your relative output is quite visible.