I wouldn't say it was the deciding factor, but when I got an offer from a smaller place working in person, I took a significant pay cut to go back into a (very cool) office.
The thing is having to be the social support for co-workers is one of the things that makes working in an office so horrible for many of us. It feels like a lot of people want to burden their co-workers with social roles that are better filled pretty much anywhere else.
Some folks are looking for new relationships, especially those just starting their career or moving to a new ciry. Ideally they find other like them, rather than 'burdening' people who don't care
There's a surprisingly large population in the tech community that want to follow your approach of "don't get vaccinated and hope everyone else does". I can respect that a bit if you at least want to continue to adhere to basic social distancing and pandemic safety guidelines.
If I do return to an office it would certainly only be for companies that require a vaccine before returning to an office.