My personal problem is that I'm way too deeply engaged with Google now. I was fortunate enough to get a Google Apps account back when it was a free service, and I have been providing all the Google services for me and all my family for free for several years.
Fastmail looks absolutely incredible and I've never heard the slightest bad thing about them, but it'd cost me $250/month to get the same level of service I get now for free. :/