Why would I care about my manager not being in the office? We probably don't share a continent anyway.
On the other hand, sometimes it is culture. I had manager that had 100+ direct reports. He never had 1x1 and barely managed to get all of the payrol/timesheet/pto aprovals on time.