Still took me a weekend to get ACLs right, tune the threshold for greylisting (I greylist e-mails which score between two and seven points in Spam Assasin), set up DKIM and SPF... it's not difficult but there's a long list of things to do.
If you're doing it just for yourself, exim, spamd and mutt are enough. But I wouldn't want to be responsible for a larger installation with regular, non-technical users.
It's not just the configuration, your service will always be subpar. At least until there's a good webmail (I have high hopes for Mailpile).