Gmail also just works for me, and Fastmail did not. After FM killed two job interview cycles by not RSVPing to calendar invites I came to the realization that it would be unethical in the efficient altruism sense for me to not use Gmail.
I'm curious how long ago your Fastmail experience was. I've been using it lately and haven't had any issue with RSVPs (at least that I know of). I also often see people who don't RSVP to anything, so I'm surprised interviewers would see that as an issue.