It wasn't hard, just about 5 hours of work and then a few hours to set everything up as I like it. I pay ~$5 per month for email/calendar/contacts through Fastmail, ~$10 per year for 2 domains (each), and ~$5 for an Android app to sync my CardDAV/CalDAV accounts with my Android phone. I have almost completely deleted/disabled Google apps on Android, although I'm not ready to run LineageOS quite yet. I even use an OSM-based maps app, which doesn't work as well as Google Maps, but it is sufficient; navigation sucks compared to Google Maps, but that's the price you pay for doing this sort of thing.
I'm not super-paranoid about government surveillance and I didn't care about Google tailoring ads to me like most folks here, but after all of the data breaches and such, I decided that controlling my own data is worthwhile just to make me feel better. Now, I am able to do most of the stuff I could do before, maybe 70-80% as good as with Google for some things (like maps), but I have peace of mind.