Why would I lose access to my Fastmail account? That risk is a remote one. I have 2FA, they take backups and provide support, and they are a reliable company (at least, not a one-person operation).
Essentially you just have to configure mutt on a *nix host to download your email once an hour. 10 minute set up time?
Miles different from managing your own email infrastructure.