I have realised some years ago that if you quietly stop attending, nobody will notice or care. On monthly town halls at a previous job I routinely went to the toilet as people were gathering, then once the event started for good, I slipped back to my desk or left for a walk. Nobody ever noticed or cared.

The next place, one of those socialising, average loser kind of a colleague asked why I wasn't attending company events and after-hour team buildings, and I told her I had a medical appointment. She didn't press the topic, and stopped asking after 2-3 times.

My strategy for pushy people who ever would ask what kind of an appointment I had (happened once) is to tell them how rude I find them asking such a question. Nobody wants others to think of them as a rude person, since everybody wants to be liked. The person never asked me anything personal ever again, and I like it that way.