mar-v-in and his µg are corner stones to build a nice free android. As for a server side I'd suggest ownCloud at the moment. I'm running one on ipv6 at home. Works great.
More importantly bitnami makes it a one-click install for all of the above, minus the IMAP server.
Only Replicant seem to be doing things the right way, but they are going so slow, and are so unwilling to use proprietary blobs anywhere that there's a space for something like you have here - blobs for hardware features only, no proprietary apps, strongly anti-Google, including F-droid by default - but it needs a bit of proper open source-style organization.
Sigh. The whole ROM scene is a big mess :(
I am still using the blobs, but I was thinking about gradually replacing parts once replicant reaches a good-enough (e.g. samsung rild, have to try it though).
Btw, if you are interested in building the ROM add me on jabber (arty@jabber.ccc.de). I want to build the next version this weekend anyway and I may as well guide you through building roms, too. Plus I would appreciate help on porting to new devices :-P
[EDIT] I'm unhappy with the ROM scene, too. That's why I started my own ROM in the first place.