The problem IMO is that too little work is truly remote. If the rest of your team is colocated somewhere else then you'll have to travel. If your team is spread all over but still relies on face to face meetings (e.g. planning sessions, "all hands" announcements, hackathons) then you'll still have to travel. If your job/role involves dealing with customers who aren't even slightly remote-friendly, you'll have to travel. If you're in or adjacent to academe, you'll have to travel to conferences. Most of these exceptions are not necessary. Unfortunately, most people and companies don't even try to avoid them. They just do what they've done, and the not-really-remote workers have to fly in.