Maddy: Composable all-in-one mail server
maddy.email
maddy.email
It was a traumatizing experience. It rarely worked, all my email went to spam and I absolutely hated the amount of hackery and duct tape required to keep it working. Configuration files, missed emails...
It was so unpleasant, to this day I avoid messing with mail on servers at all costs. I'd much rather pay for Mailgun or some provider.
Lately I've been seeing an uptick of these apps making self-hosted email easier. Mail in a box, Maddy, etc. I will unconditionally support anything that makes doing this easier.
I'm not switching from GSuite, but I'm happy to see progress in the space.
Maddy comes with a lot of the 'good stuff' like DKIM and DMARC ready to be used, whereas setting that up yourself with Postfix was a pain (from my experience a few years ago) and probably meant you didn't bother.
I've had better deliverability, especially to Gmail and Microsoft, since using Maddy, probably for this reason.
Now, of course, I must admit that self-hosted e-mail seems like it will always inevitably run into a blocklist problem, usually because of IP neighbours. But personally, I've managed to avoid that in the last year that I've been running Maddy -- main exception is I had to fill in Microsoft's silly form to get unblocked at the start (but I've had to do this for every deployment I've ever done).
Well, viruses are the exception.
For anyone with deliverability issues and mail going to spam, it might be worth trying this out (after you have sorted all the usual points like DKIM, SPF, PTR record etc.) - this obviously only helps if your IP is clean. I was surprised though, and only discovered it by accident.
I had good deliverability to Google hosted mail, but not to Microsoft 365. I even had issues with deliverability doing 365 to 365, until I sent HTML mail.
I'll have to give this a try. Thanks for the tip!
It was the same issue - even replies to existing threads had the issue. What got me looking into it more closely was that I had no issues delivering to G Suite tenants; it was only issues with 365 tenants.
I wonder if there's a non-trivial correlation between people choosing to genuinely self-host their own mail, and those who use plaintext outbound mail by default (and know what that means).
On the other hand, most human-generated mail is HTML-based, as it's written in a web-client or desktop/mobile "app client", that is using HTML to let people do bold and put pictures into their email.
I was as surprised as you when I figured this out!
Are you sure there is not some sort of parasitic Bayes filtering happening here?
I never found much by way of technical detail on 365 deliverability, beyond the usual DKIM/SPF/DMARC/PTR record guidance, and forms for getting yourself removed from the whitelist.
What's especially strange is that this issue applied even when sending mail as a 365 subscriber. Once there was HTML content present in the email, deliverability (to other separate 365 tenants) improved!
Very odd...
Where do I even turn? Did my startup's domain get blacklisted before I've even launched?
See also ISPmail (which uses Debian):
However, I was very specific with always going for the minimum most simple solution. I do not have a dedicated database for instance, but use just mbox and POP3.
And I did not even bother much with anti-spam, I chose a super easy filter in Thunderbird instead, sending everything with 'unknown' in Received: header to the Thrash folder.
Getting accepted by others was also not much of an issue (but lots of details of course, PTR, SPF, DKIM, DMARC).
More details here if anyone is interested:
I realise that everyone hates spammers, but having a "thrash" folder is a bit much, isn't it? ;)
I've got to write a longer article about this, but self-hosting email is incredibly liberating. I now own my entire digital identity. I can work on open source projects with developers around the world completely while relying only on my own capabilities.
Did a bit of a cost analysis here: https://figbert.com/posts/moving-to-hetzner-from-digitalocea...
I filled the form, was requested documentation from my ISP on the IP address.
I asked my VPS hosting company (since they provide the public IP and therefore act as ISP) and they proactively reached out to Microsoft, who lifted the restriction after that.
So since then no delivery issues. YMMV.
If you intend to host this from your residential address, it can be a good idea to tunnel external traffic over VPN through a VPS or similar. Not only for privacy reasons, but also to get around ISP blocks and IP banlists. I don't know how flexible maddy is, but in postfix in case the above scenario wouldn't be resolved, I could have set to use mailgun/mailroute for MS domains only and relay like normally for others.
But like I mentioned, what eventually resolved it was the ISP contacting them.
I have had a number of bumps:
1. In exchanging emails with someone with a custom domain, I found there SPF record was broken and thus my server was rejecting their emails. I've weakened my policy and now their mail goes to my Junk, which I then manually move to my inbox because I'm lazy and don't want to set up a custom rule.
2. I wanted to subscribe to the Tarnsap mailing list, and had to decrease the minimum TLS level for outgoing mail to "none." Dr. Percival believes TLS on SMTP is "silly" (which, in the sense that all email is insecure, is true, but in the sense that email with modern security measures is better than nothing, is in itself a "silly" opinion).
3. I had some server downtime recently (https://figbert.com/posts/wrong-way-to-switch-server-os/) and couldn't receive emails, which sucked. But that was on me.
Our external auditors get all upset about it every single year, and every single year, I show them the RFC's and they then shutup about it for a year. If you feel strongly enough about it, try to get a RFC passed where SMTP requires TLS now.
I have no idea why that helps (well, I could guess that some spam heuristic thinks plaintext email without an accompanying HTML envelope is more likely to be spam), but changing this took me from near-constant "your email went to spam" to no issues sending even things that actually look spammy (i.e. an email just containing a link that might be of interest to the recipient)
Generally if you have a static IP and you don't go being all stupid with spam, it's not THAT difficult, but you do have to jump a few hoops and then occasionally play wackamole with their spam prevention junk for the month.
It seems to come in waves, like email will be fine for a few months and then 1 provider after another will be all upset about gosh knows what that day and you have to visit URL's and push a few buttons.
I've been to lazy to track it, and the various reasons for that particular day, but this has been my experience. A few times a year you have to go babysit SMTP so email can be delivered again.
We host and manage the SMTP server(s) ourselves(We currently run Postfix). If you outsource your email to Google, etc, then they have to babysit the email logs for URL's, not you.
Do they care? How would I know? =(
But then I also very much prefer a hierarchical/directory tree approach to organizing e-mail than labels&search as Gmail does it.
More similar UX-philosophy can be found in Mailpile[1] and Cypht[2]. Both still have decent amount of moving parts but are continuously progressing.
[0]: https://github.com/roundcube/roundcubemail/
[1]: https://github.com/mailpile/Mailpile
[2]: https://cypht.org/
I would recommend it based on my use and experience.
I’ve been using it for my personal email and for automated no-reply messages in a few projects with no delivery issues. I’m hosting on a Contabo VPS.
It handles spam with rspamd, sets you up with DMARC, DKIM, has an admin interface, web mail interface.
Maddy (sqlite) + https://litestream.io/ might be simple enough to get right.
What replication lag could tar-on-a-cronjob reasonably achieve for a moderate 5GB mail store?
Even tar'ing isn't trivial: I've seen casing of people having missed preserving file permissions, making restoration difficult.
(Backups are done by rsnapshot pulling from a snapshot, so we have another copy outside ZFS just in case.)
But proper backup tools like rsnapshot, borg, etc. pp. have no trouble correctly backing up email servers even at 10 or 100 times your mail store size.
It works great with Haraka[0] on the outside proxying email to maddy instances on the inside. I've also combined it with SES provisioned with Pulumi + k8s[1] (for those on AWS), which is a little bit more involved but is what I use on projects now.
Another entry in this space is chasquid[2] but I've only used/can recommend maddy. Maddy is so good it makes me think anyone could run a hard-multitenant email provider with ease (I'm essentially doing that for each of my projects).
[1]: https://www.vadosware.io/post/setting-up-ses-with-pulumi/
[1] https://jschumacher.info/2021/05/running-a-private-mail-serv...
Steep learning curve indeed, but after initial setup it's just working and necessitate no tending except when I do major upgrades (of Debian so every 2.5 years on average), in case some config needs updating.
But this is really a private email server with no users except me, thus probably not encountering the issues related to mail servers with many users that might or might not have a good email hygiene.
It is pretty awesome to see new projects that try to improve the user experience for the system admins, for example, the Caddy web server and now the Maddy mail server. I'm not sure whether i'd have been able to set up a mail server the "old fashioned way", seemingly in line with the concerns of the other people here who have tried that more extensively. Somehow, while e-mail is super widespread and works decently for getting text information from place A to place B, it also feels a tad overcomplicated (the different components for sending/receiving mail, SMTP, IMAP, POP3, the sad need for anti virus scanning, SPF, DKIM, DMARC, the similarly sad need for anti spam protection).
That said, the containerized solution above has also worked nicely for me - i'll still probably use GMail/other hosted solutions for personal e-mail stuff and communication, but right now i've already switched over to using my own mail server for all of the automated e-mails that i need, such as from SonarQube and Zabbix, as well as GitLab.
I do approve of such a project existing though, and if I wasn't running other things on the host I would probably use it.
Resource consumption is basically zero, no problems with sending or receiving emails at all.
It did take me quite a few hours to properly set up a Kubernetes + Terraform config though. Getting all the bits and pieces for DKIM, SPF, DMARC right, including issuing certs via cert-manager, is quite finicky.
However, i'd say that it's less of a matter of choosing a VPS host and more one of making sure that the IP addresses that you get are not in any blacklists. This seems like a good starting point for those sorts of checks: https://dnschecker.org/ip-blacklist-checker.php Or maybe this: https://mxtoolbox.com/blacklists.aspx
I think the actual question would be: "Which VPS hosts have the least blacklisted IP addresses for hosting mail servers?" But i'm afraid that i cannot answer that one, because after a brief search i could not find any articles, which would check the IP blocks owned by different hosts and what percentage of them are in blacklists, which would probably be a large undertaking.
Until then, most of the answers will probably be subjective, along the lines of: "I use $HOST without many problems." (like my answer above, though even then someone could get a blacklisted IP address and their answer would be the exact opposite to mine).
I've not had problems with Hetzner and Scaleway. Before that I used to host email on random vendors I saw on Low End Box[1] and never had problems.
I think the advantage with Hetzner cloud is that you rent the IP address separately. If you get an IP with a bad reputation you can stop renting it and get a new one.
Edit: I have not actually tried changing the IP, its just an idea. Also, I hope the spammers don't start doing this and spoil it for everyone else.
I do use SPF but I don't use DKIM or DMARC. On the inbound side, I do use greylisting and anti-spam. I don't tend to change my config between releases of debian stable.
At least Vultr and Linode will also communicate with other big providers (e.g. Gmail or Outlook.com) in case you have delivery issues due to history of the IP addressor subnet before you took it over.
I wonder how it compares in multi account management and spam, phishing, & virus blocking, is it extensible?
maddy implements de-facto standard milter protocol so it is possible to use any third-party filtering software supporting it.
But it's a total mess. So many different components glued together, so much that can go wrong. The configuration is scattered into hundreds of files. There is no clear split between the product and user-configuration. To change some settings, you need to make adjustments in the core product (the config files that hold everything together).
Replacing a lot of components with Maddy would probably make projects like mailcow way slimmer.
Edit : typo
I am severely tempted to give maddy a try now...!
[edit] I found the FAQ:
> It was intended to be one but developers quickly acknowledged the fact email cannot be easily abstracted behind some magic.
So it can't be as simple as caddy, but they're doing their best. :)
It is more or less one binary and one data folder, available for all operating systems, providing a lot of functionality (SMTP, IMAP, ActiveSync, Webmail, Groupware, ...). Everything can be configured via one uniform Web UI. Multi domain, multi IP, and you could even cluster together multiple servers.
It's closed source, but offers a free license for up to 5 mailboxes.
Sadly this product died during the last 10 years and is now in an unusable state.
But if I was setting up something new, I'd definitely use one.
I guess that needs to be in FAQ...
There are no known issues with popular clients but performance may be a bit rough. Both due to missing extensions and storage implementation quality e.g. SEARCH may get slow for large inboxes.