I don't mind full remote -- if it's full remote. I also don't mind going in the local office (for the reasons you mentioned people like your company trip -- I feel more connected and communication walls are broken down).
This is one of those situations where there's no obvious perfect answer. If we didn't do regular meetups, then we'd lose out on the benefits for those who like them. If we do, we lose out on the talent that fears them.
Industry conferences can also provide a useful target (meet before the conference, go to the conference, etc).
You can make them optional I guess but the outliers who don't want to go will often end up dropping off of others' radar screens over time. Personally if periodic offsites (or for that matter attending certain other types of events to, say, give a presentation) really aren't your thing, you should probably find a company that better matches your preferences.
You can see it as a perk, but I see this as a theft.
However, please note that we have unlimited PTO (not a scam - I have not turned down a single PTO request in the last two years) and strongly recommend at least 1 month paid paternity/maternity leave.