If a company is 100% remote that’s one thing, but in any hybrid scenario those that consistently appear in an office will unavoidably be at a significant advantage. Those informal coffee chats, grabbing lunch together and such are invaluable and it’s hard/impossible for someone on a screen somewhere to get the same benefits.
There will be those that dig their heels in hard but unless the company is 100% remote forever such stubbornness is likely a career limiting move.