Then you already fundamentally have a remote job. A non-remote job would organize business units / teams by location so that you're in the same office as people you're working with. And yes, this would require relocations and imply a smaller hiring pool. That's the tradeoff of not having remote work, not just spending more time in the office. You're not comparing remote to non-remote work, you're just comparing two flavors of remote work.
ADDED: You can force co-location but, as you say, it reduces company options as well as employee options. And may not even be very practical; I work extensively with other teams both in another state and in Europe.
much less time wasted in between meetings.
they definitely also get more done
The real difference in my experience is that extroverts prefer WFH and those tend to more often become managers however plenty are ICs as well.
Why are meeting room chairs so uncomfortable?