The manager at my company that keeps this "no full-time telecommute" policy is in his mid-40's. And he's not rich. Well compensated, yes, but not 20 million dollar home in Atherton and a Tesla rich, more like "drives a BMW 4 series from his Peninsula Condo", but he's certainly not rich enough to not need to work.
So I don't think this is a problem with "old rich people".