An additional challenge for me is that I don't like the taste of alcohol, and having even a little makes me do/say things that I later regret.
So even when I do show up at company social events, I feel even more like an outsider. It's hard to stay engaged at a job where all of these team-building events / milestone-celebrations make me feel lonely and excluded.
I actually had to push back on our in-office boozy happy hours because a few team members were getting too drunk and it was making other people uncomfortable. My boss didn't think it'd seem "cool" if we put together a policy for how these happy hours should be handled. I don't work there anymore. :)
I still distinctly feel different whenever a VP or higher level manager joins, but when it's just my team it's a lot more comfortable and has led to a lot of friendships that extend beyond our working relationship.