The sweet spot in my experience has been for WFH orgs to get together periodically for extended offsites. A one week gathering twice a year has forged some incredible bonds on my distributed team.
of course I got hired in that place in 1/2020, so we know how that turned out.
All of your complaints are (usually) easily solved by experienced travelers. It's okay to not like travel, but it doesn't condemn the concept of a weeklong company offsite a couple times a year.
I'm the only person "remote" (work from another office/country) in my team and I am the one that always has to travel to meet the rest of the team.
It's nice meeting people and all, but I generally find these offsites pointless and not work the hassle of traveling for it.