I got into the exact same problem a few years ago. So, I wrote email backup app: https://thehorcrux.com/why-i-built-horcrux-app/

I still use Google Apps at this point as it's pretty convenient compared to standing up my own server. However, I make sure my email address is on my own domain. https://thehorcrux.com/about/