Alternaive to Fastmail (using a custom email domain)

I've been wanting to ditch Gmail for too long now. In the context of privacy, I had seen FastMail mentioned a number of times on HN and by all accounts it looked good from the outside. One of the key must-haves for me is the ability to use a custom email domain. FM has that. Great!

Note: My needs are only for me. There are no other team members - at least not at this time.

Unfortunately, I've run into a couple of things with FM that for me make it a bad fit.

1) I've set up my custom domain and have an address, me@my-domain.com. Unfortunately, in the FM email client - web and mobile - the default send from is always me@fastmail.com, and I have to manually switch the send from to me@my-custom-domain.com.

Since I'm using this for professional purposes, this is annoying and a deal breaker. I don't want to sending from me@fastmail.com *at all*. At the very least, if I'm using my own custom domain why wouldn't that be the default? Bad product design?

2) Along the same lines, apparently there's no way to "decouple" me@fastmail.com from me@my-custom-domain.com. That is, to get to me@my-custom-domain.com I have to log into me@fastmail.com. But that means - on web and mobile. - my admin "stuff" is readily available.

I would prefer it if my me@my-custom-domain.com had its own password just for email, and that if I need to switch into me@fastmail.com to do something "admin-y" there's at least a "please re-enter your password" or "please enter your admin pin". Something to prevent mistake. Something to add a bit of sec in the case, say, I lose my mobile device and someone guesses my PIN (or whatever).

As far as I can tell, the only way to get the "decoupling" effect would be to have two @fastmail.com accounts both on the $50 tier. But even so, as far as I can tell, I'm still stuck w/ their email client defaulting to @fastmail.com instead of @my-custom-domain.com - which just boggle my mind that something THAT simple isn't possible.

I want privacy.

But I also want some sense of security.

And some sense of UX.

Thanks.