I think it depends on how you view your colleagues, and also on your working conditions and location. My entire company is remote (albeit with a small office in a co-working space a few local people sometimes use), and we have two week-long on-sites every year (recent times notwithstanding). There is no pressure to attend, but the entire company is always there (unless they have scheduling conflicts etc) - that says a lot to me.

It helps that the most important aspect of our hiring process is about culture-fit - so much so that the final stage of the (pretty relaxed) hiring process is a presentation to everyone in the company who wishes to attend (regardless of role). Those presentations typically see 40% or more of the company attend. Crucially, every person in that presentation gets the opportunity to vote on whether the candidate is hired.

I've made friends for life at this company.