I've been running my own mail server for about 15 years now. Getting it set up is a fair bit of work. The hardest part is just deciding how you want things configured. It has been pretty much hands-off for ten years now. Dealing with spam has been the biggest annoyance, but even that hasn't been too horrible, and it forced me to learn how to write milters. I pay for commercial hosting so my mail comes from a "reputable" IP address. I once got blacklisted by Spamhaus because I accidentally enabled an open relay, but that was easy enough to fix and it hasn't happened since.

AMA if you decide to take this on, I'm happy to help.