Thought I'd have another look at mail providers, but from what I can see, none support the features I use with fastmail (custom domain, security key, unlimited on-the-fly aliases for sending).
The alternative that's looking best to me so far is Kolab Now. I don't see a lot of user reviews of it on Reddit though, or anywhere else, so it seems to not be very popular at first sight. That's perhaps not a good sign.
In any case I'm planning on trying it out for a while, with a domain I don't use it all that often, before deciding to migrate to it.