Maybe you need to work on your communication skills? I realize that's a snarky sounding answer, so let me expand on that. Face to face communication is nice and has benefits. But there are tradeoffs. It's clear that your company has decided that when making a decision between having certain team members be remotely part of the team or not at all, the choice is clear. So maybe the appropriate course of action is to accept the benefit of continuing to have access to those individuals and work on adjusting your communication style to support distributed teams. Protip: a remote first workflow also improves documentation and communication if you have more than one office.
Personally, I welcome the chance to work with a group of minds that isn't restricted by the filter of "lives within x hours of office". The world is full of amazing projects created by people around the world working together.
I think your observation is valid, but in my experience the drawbacks of remote working are dwarfed by the benefits.