Maddy: Composable all-in-one mail server in Go
github.com
github.com
This looks like something worth combining with alps[0] for easy self-hosted email. I might give it a go and ditch gsuite!
I recently switched to Fastmail, and one of the features I like about them is that they support wildcard sender aliases, so if I sign up for a service with someservice@mydomain.com, when I hit reply to any emails to that address, it automatically sends from someservice@mydomain.com even though I've never specifically registered that identity with Fastmail.
[0] https://thedigitalnonprofit.com/use-gmail-own-domain-free/
Also, while aliases in gmail work generally, I find they leak the main gmail account address, which makes them significantly less desirable.
You can just head over to the "External DNS" page where MIAB will tell you what DNS entries you need and what each entry does. Then you just copy the DNS entries you need.
Intellectually, I don't like domain DNS on the same box; practically speaking? It works and works well -- so well that it's often easier to disable per-domain DNS where my sites are hosted and just point A records over to the sites from MAIB.
Original reply:
Buying domain hosting gives you "unlimited" email addresses, so domain hosting is cheaper as soon as you need 3 or more addresses.
3 addresses x $2 = $6/mo for email vs $4.95/mo for any number of addresses. I have a friend who is married with 4 children. He pays $4.95/mo for 6 addresses. You don't have to actually host an http domain to get email hosting that comes with domain hosting.
"unlimited" in quotes because obviously there are some limits and service terms. You can't start a competitor to gmail with a $4.95/mo hosting account.
Though I admit that for someone who did not tinker with UNIX for years, configuring this kind of setup might be daunting.
Personal mail could be solved by some kind of daemon which provides SMTP, IMAP, built-in spam detection, DNS checks, some simple administration web interface and works with 0 configuration for typical use-cases. Maddy could be this project, so I would be happy to replace all those daemons with a single binary. I hope that they'll implement spam detection, as rspamd is a separate daemon which requires redis, so in the end it's still a complex setup with lots of moving parts.
https://battlepenguin.com/tech/how-google-and-microsoft-made...
I still prefer to host my own though.
Great article btw!
P.S. As far as I am aware, redis is optional for rspamd. I certainly run it for @hexanet.dev email without having redis installed.
I used to be a Kolab [2] user and "promoter" for many years but the project seems stalled a bit recently and although it is still being worked on and developed, it is lacking documentation for the latest releases.
I have moved to mailcow recently and am amazed at how easy it is to setup and maintain as opposed to Kolab.
The thing I miss from Kolab in mailcow is the integrated LDAP storage and custom built UI (if anyone knows a good and reliable LDAP administration UI as a web app, please let me know!), which I got used to using for user authentication in other services. Luckily, there are some efforts in LDAP integration into mailcow [3], although, still not mature yet and not that tightly integrated.
[0] https://mailcow.email/ [1] https://sogo.nu/ [2] https://kolab.org/ [3] https://github.com/Programmierus/ldap-mailcow
Another good alternative is migadu (who sponsored the webmail you linked above, and IIRC are already using it). They do charge about $20/year for a micro plan that lets you have any number of addresses and domains , with some throughput limits, and $90/year for a full plan without those limits. Something similar to the micro plan was free until recently - personally, I'm on the $90 plan with a few domains and emails, which would have cost about 10 times on much on gsuite and fastmail.
[0] https://github.com/tomav/docker-mailserver [1] https://github.com/containrrr/watchtower
It is not me. Just wanted to give a little tribute to the amazing person who put a lot of work into email libraries for Go: https://github.com/emersion/
I'm hoping to use a Golang-based application as a simple SMTP Relay, which also allows for ranges of IP addresses to be whitelisted to connect openly to it from within our campus network.
Basically, the scenario is as follows: a number of internal services are older and may not support the authenticated route so we can use Sendgrid directly and send emails securely, so instead we currently have them routed to an internal Windows server which is running an older piece of IIS functionality (it's an SMTP Server in Windows that can be turned on). That piece is configured to be open (so it can receive messages on Port 25) but is also limited to a set of IP address ranges that internal servers are using (so that it's not generally usable by just anyone inside our network) and is configured to send those received messages out using Sendgrid.
I found the following Goland project just a few months back that seemed promising: https://github.com/wiggin77/mailrelay
And I'll probably need to follow up with the developer again (or try and make the contribution back to the project on my own), but the main issue it had was that the IP range whitelisting wasn't available (and even whitelisting by specific IP addresses seemed problematic, so in my testing I had to keep it completely open for it to work as expected).
If Maddy can fulfill the need instead though I can give it a shot ;-).
That is rather than a timeout or connection refused, an smtp 550 or 530 error with exlenation (access/relay denied; client ip not in whitelist)?
There are tons of other options. Mailbox.org, Migadu, Protonmail, Tutanota and Mailfence for example.
Great service, E2E for everything, including contacts and calendar, and it's quite cheap (I think I pay €12/year).
This seems good, however for 5+ it becomes expensive too.
Is this service related to what you linked?
Is that still an issue?
So prereqs - 1) Valid reverse DNS on your sending IP. Preferably with a hostname that is also forward resolvable. 2) SPF Records 3) DKIM
I do in fact want this. It may not be possible, but I definitely want it to be.
(I eventually went with a Tutanota account, fwiw)
I have had my own email server sending emails which were not marked as spam. also I have had cases where email from Gmail addresses/ips would be actually spam.
I urge everyone to start using their own mail servers so the ecosystem around self hosting emails becomes super smooth.
why do receivers still rely on ip addresses when we have dkim and spf?
Lists are like the DRM kind of tech, where the genuine user has the real headaches (pay a service to filter my mail, cant self host etc) while the spams are still flowing through.
SPF and DKIM are pretty much required now, as are TLS/SSL. From there, it turns into a dice roll. Gmail is terrible about this; they have a totally opaque and very frustrating engine that sometimes filters messages into a junk folder and sometimes doesn't. Outlook.com uses a somewhat more traditional internal RBL, but they are happy to block network segments and they don't offer any way to query their blacklist or request removal from it, so you could do everything right but a neighboring VPS or IP will get you blacklisted anyway. Comcast will simply accept delivery of messages and then disappear them depending on an arrangement of the stars that I haven't quite figured out yet. And those old sbcglobal/Yahoo services users... just, uggghh.
A popular solution is to rely on a third-party service to handle your outbounds. Sendgrid specifically is really bad, they carry way too much junk traffic, so don't go with them if you decide to try this out.
I have a post banging around in my head that's titled "Email is fractally broken", and getting outbound non-spam messages to reliably land in other people's inboxes would be a significant part of that writeup.
In their activity feed right now, SendGrid traffic is being blocked by comcast, GoDaddy, iCloud, and a cornucopia of smaller services.
Here's one of SendGrid's outbound IPs, listed on multiple RBLs used by other services: https://mxtoolbox.com/SuperTool.aspx?action=blacklist%3a167....
On the receiving end, I've had to deal with waves of spam from SendGrid for years, and it's especially difficult because they carry a mix of spam and legitimate traffic, so blocking them causes complaints and not blocking them also causes complaints.
Never had any issues with sending mail. But I've been careful with the initial setup and have all the expected requirements covered : no open relay, SPF, DKIM, reverse ip, TLS,...
Been using the tutorial at workaround.org as basis for building the stack (postfix, dovecot, rspamd,...) and, while the initial learning curve was high, once its been set up, its been a really very stable setup requiring practically no maintenance whatsoever.
Not saying that's for everyone, just saying it works for me.
I guess I was lucky at the dice roll mentioned by other comments.
Ah yeah, and testing your outgoing emails using a service like mail-tester.com is very very useful. I aimed for and reached a perfect score 10/10.
However, when I set up my first email server most emails were delivered just fine, however emails to a Microsoft domain (hotmail, outlook, etc) all were marked as Spam.
Then I switched to a new IP and since then everything has been great.
It is easy to get an email server running initially, but hard to get all the auxiliaries (SPF, DKIM, DMARC, MTA-STS, TLSRPT, BIMI, etc) set up correctly. It's even harder to debug deliverability issues, since there is 0 feedback on why your email is not being delivered. To a point where it inspired me to build an email hardening monitoring/validation service [0].
At the risk of being downvoted by people who are frustrated by email deliverability (and it is frustrating, I know): when you have email deliverability issues, it is almost always because you did something wrong on your end. Remember that false positives in spam detection hurts the $EVIL_BIG_CORP receiver as much as it does to you as a sender.
> Status update: JMAP is a big complex protocol that is not used by any popular clients and has no server libraries available for Go. IMAP is a big complex but also widespread protocol that is well-known, supported by any email client and has server library available for Go. Some of IMAP disadvantages come from incomplete client implementations, not protocol flaws, as JMAP developers want to convince you. I do not expect wide JMAP adoption in next several years, therefore the decision was made to prioritize improving IMAP implementation.
> If somebody insists on having JMAP, I recommend looking at Cyrus email server, perhaps write a Go library for its SASL delegation protocol so it could be used with maddy just as easy as Dovecot.
https://www.npmjs.com/package/mime-to-jmap
Its a bit of a mess and it needs a write-up, but it should be a reasonable start to making more jmap servers. (This does not implement the jmap application protocol, but it does generate fantastic json from any email blob, incl attachments, snippets, html and text output. And thats crazy difficult to implement correctly.)
Maddy looks great though, I may start playing around with it.
Putting aside issues with sending mail to recipients using third party providers, have you been able to receive mail reliably from (a) senders using third party providers or (b) senders running their own SMTP.
If you run your own, you'll want to get some log checking tools to make sure things are tidy.
We send mail from a Linode VPS in Frankfurt, which works fine most of the time but a couple of times a year Microsoft puts us on a blacklist, most likely because of a neighbouring IP sending spam.
Project site says "all-in-one" (one daemon) to replace postfix, etc. Like sendmail?
Please say more.
For 20+ plus years I've been advocating for postfix instead of sendmail style architectures.
I don't know what to call the "one task, one process" style of postfix. What's the opposite of monolithic?
This tension is replaying (again) in the web services vs monolithic architecture debate.
Ironically, I prefer monolithic web apps over web services based designs. Contradicting myself.
I'd like to better understand when each style is better suited to the task at hand.
The web's answers for a suitable antonym seem inappropriate [0] (all referring to "small" rather than "many parts". I reckon megalithic is the way to go.
Although I've been hosting my own mail server (postfix/dovecot) for years I must admit that I treat most of its parts as a single black box. My head just doesn't grok the entirety of processes and the various sub components (spam, grey listing, filters).
I prefer monolithic every day. It's not hard to provide plugins that allow custom filtering/ spam etc, so the end result can easily be the same.
Especially for personal /small business servers, simplicity wins everytime IMHO.
Mono - one, lithic - stone. "One stone". It's in the name, right?
Default configuration runs everything as a single daemon. This has been done to minimize any management overhead, avoid the complexity and performance overhead introduced by IPC.
It is definitely possible to split things apart though - this is not something of a hard design decision. This is what LMTP is for, right? maddy can work as both LMTP server and client and also supports both server and client parts of Dovecot's authentication delegation protocol.
So you can do something like that: 1. maddy instance running SMTP on port 25, running inbound filtering and then doing transparent LMTP forwarding to ... 2. maddy instance running LMTP on some unix socket, delivering to local storage and providing access to it via IMAP, authenticating users using ... 3. maddy instance running Dovecot auth's protocol on some unix socket providing authentication service using some DB. 4. maddy instance running Submission, managing queue of outbound messages, trying delivery by forwarding them to ... 5. maddy instance running LMTP on some Unix socket, actually attempting outbound delivery.
In fact, you can also put any of these on separate VMs/containers or even physical systems. And if we add some load-balancing capabilities to SMTP client then it can be used to scale message processing (though a single daemon can already handle quite a lot of emails and users without problems).
So it's not a monolith, it's a composite. If that makes sense.
Sure, the components of postfix are not standalone tools, they are very much like systemd. One codebase (monorepo, yaay, old is new), sharing a lot of common internal code.
In the end multi-process composite (like oracle and postgresql RDBMSes) or a multi-threaded monolith (like MySQL or anything that runs on a JVM) is not that big of a difference nowadays. Both can be performant, both can be maintained well/efficiently by big teams and by a one-man-army (see how postfix is mostly maintained by Wietse Venema).
That said, Caddy's killer feature for me was automatically configuring certs, that's what made me switch from Apache back when we were moving everything to HTTPS. I still don't fully understand how certs work, but fortunately I don't really need to. Until Maddy does this, it won't be a good comparison.
Also I would really appreciate some documentation for making this work with Caddy handling TLS certificates for me. I guess I'll file a bug about that.
Only SMTP though, for the IMAP part you need to use something else like Dovecot.
Obviously only usable for NixOS zealots, but just import the class with your desired parameters and good to go.
But Maddy-like consolidation seems like the way forward. Setting up 100 moving parts is not always easy.
[0]: https://gitlab.com/simple-nixos-mailserver/nixos-mailserver
There are some messy bits related to licensing because it was MIT licensed in the past (back then when I did not take this "little pet project" very seriously).