Were they "working hard" on individual-contributor-style work, or on management?
"They can’t peer into everyday lives of the lower levels" - I'd imagine this is exactly what management should work "hard at".
If you're not keen to do that, then maybe be a high-level IC?