Your own Debian Mail Server (part II): how to prove you are not a spammer | Hacker News Reader

Your own Debian Mail Server (part II): how to prove you are not a spammer

scaron.info

276 pointstastalian96 comments

My number one product I wish existed is this: A complete email server package that is easy to configure, setup, manage, is secure, and is accepted by other email service providers (gmail, yahoo, etc) out of the box. And with easy I mean as easy as apt-get install or just downloading a binary.

For anyone that has configured email servers, you know it is a headache, this tutorial makes it look easy but its only adressing a a tiny portion of the problem (albiet a important one) - email spoofing. (EDIT: it does mention spam assassin at the end so there's a little bit of info about spam filtering)

https://yunohost.org/

https://yunohost.org/#/whatsyunohost

YunoHost is a server operating system aiming to make self-hosting accessible to everyone. It is based on Debian GNU/Linux and is fully compatible with it.

Basically YunoHost automatically installs and configures some services around LDAP, and provides tools to administrate them.

It can thus be considered as a distribution, including the following software:

    Nginx: a web server
    Postfix: an SMTP e-mail server
    Dovecot: an IMAP and a POP3 e-mail server
    Amavis: an antispam
    Metronome: an XMPP server
    OpenLDAP
    Bind: a DNS server
    SSOwat: a Single Sign On (SSO) web authentication system
    A backup system (not yet implemeted)

The tricky bit here is "accepted by other email service providers". This depends a lot on the IP you are using, on the reverse DNS, DKIM/SPF settings, your ISP and "neighbours" reputations, RBL listings etc.

It's not just a case of distributing postfix and a nice UI on top. That's what makes email difficult nowadays.

I tried doing this a long time ago, and eventually gave up exactly because of this. I self host a lot of things, but self hosted email has always taken up a disproportionate amount of my time. At some point it's just not worth the headache any more.

I ended up ditching the self hosting route and went with an Exchange Online hosted email subscription at $5 per month and never looked back.

https://products.office.com/en-ca/exchange/exchange-online

Well, it's not THAT hard. Find a good host with clean IPs, check the IP against RBLs before embarking on running your own email, if it's listed demand a clean IP instead.

It's not the easiest thing in the world, but it's far from brain surgery.

Mail-in-a-Box [1] is very well-done, with a clean and modern mail server configuration. However it is not very customisable and requires a dedicated vm.

[1] https://mailinabox.email

A VM/VPS running Ubuntu with one GB of RAM to be precise. Had to recreate a droplet when testing it because I only provisioned 512 MB of RAM.

So far, it has done its job quite nicely. Just make sure you don't plan on installing it next to a webserver. Its installer hijacks port 80/443 with its own nginx webserver.

Can I add "mailing list with a decent UI" to the tick list? Something like discourse's UI but with proper mailing list functionality, like Mailman. I know discourse has fairly theoretical mailing list mode but it's atrociously bad...

Check out Hyperkitty (https://gitlab.com/mailman/hyperkitty)

I also wonder if it might be workable to deploy the D-lang forum just for the mailinglist bits: https://github.com/CyberShadow/DFeed

I can say that configuring opensmtpd is surprisingly easy, comparing to postfix or exim. My config is 13 lines, and it supports dovecot integration, DKIM signing of outbound messages, TLS and authenticated submission. I wish every server has so simple configuration. But that's only SMTP part of mail processing.

I've became a big fan of OpenSMTPD lately, although for two specific use cases. I manage a number of mail systems, some small and some large, and used Postfix exclusively.

Recently, after discovering and playing with OpenSMTPD, I've started replacing Postfix with OpenSMTPD on null mailers and internal relays. My Postfix configuration is pretty much identical on all of those hosts and I've gotten it pretty polished over the years, but there's something about the simplicity of OpenSMTPD that I really like.

I don't envision moving from Postfix anytime soon on my main mail systems (which serve many, many domains and users). If I had a much simpler mail setup I would definitely consider it, though.

> is accepted by other email service providers (gmail, yahoo, etc) out of the box

What would stop spammers from abusing this trust?

The installer would make you solve a captcha.

Can't be open source then, or spammers would just delete that code.

And how exactly would they do that on your server?

Zimbra has packages for Ubuntu and CentOS which are really easy to install and upgrade.

Pros: It provides a good enough webmail, a good (collaborative) calendar, LDAP integration, etc. Configuring SPF, DKIM and https/smtps/imaps was pretty easy (good official wiki docs).

Cons: Since it's a very "industry ready" kind of project, with lots of corporations using it, so the community forums can be very hit or miss. "Community" (free software) downloads are harder to find on their website, and the binary package is a bit annoying to keep up to date (does not use apt/yum).

Overall, I find it sufficiently turn-key and it "just works", while still being based on free software. I also feel comfortable recommending it to clients who want to move away from MS Exchange or Google Apps, since there are lots of Zimbra commercial providers out there who can support it.

ps: Zimbra/mail is not part of my main business focus, I only manage our mail server because I don't like depending on a 3rd-party for something as critical as mail.

The minimum system requirements for Zimbra feel excessive.

Fair point. The web UI and (iirc) imap/pop daemons are Java based. The smtpd is postfix. It also uses MySQL for some of its storage, and OpenLDAP for authentication. On the other hand, it scales well.

There are a few docker boxes which look quite promising, and once you have docker on your system it is actually pretty much as easy as apt-get install. (docker pull / docker run)

http://www.iredmail.org/ ?

iRedMail is very quick to install and offers a large selection of OS and backends including OpenBSD, MariaDB, PostgreSQL, LDAP, ActiveSync.

The only thing it doesn't do automatically is buy an SSL certificate.

The last bit should be easy to fix once letsencrypt is operational.

we have a pretty easy to use multi-domain mail server in a Debian Wheezy container [1] based on LXC. It's an integrated package with Postfix, Dovecot, a GUI admin (Vimbadmin) and Roundcube webmail preconfigured so you can start with a GUI admin right away and add domains and users. It's like a small head start and works out of the box and there is a comprehensive guide on how to use it and configuring a mail server including a screencast [2], that could be worth exploring.

But the thing with a mail server is there are number of steps beyond installation. You need to configure SSL and spam filtering, then your DNS and SPF records. Then you need to test and ensure your mail is making it through. And a mail server requires constant attention. It's not a configure once and forget. You need to take care of spam filtering, anti virus, and ensuring your mail server is up pretty much 24/7 without break, as email is important and cannot be down. This can become a huge time sink, hence folks leaning towards hosted solutions like Gmail etc which take care of all of that for a small cost.

[1] https://www.flockport.com/apps/mailbox/

[2] https://www.flockport.com/using-the-flockport-mailserver/

I use virtualmin/webmin for this purpose.

I work on Virtualmin. I'm glad it works well for you in this role. The mail stack is probably he single most complicated portion of the stuff Virtualmin manages (it certainly has a long dependency list). As the person that maintains some of it, I also wish there were a simpler way! The number of components we have to keep up in order to make it easy is mind blowing...

That said, I'm really surprised at how many mail sending services there are. Sending mail really shouldn't be hard (and it isn't if you understand all the components, but it's still time-consuming enough to be a challenge for many). I am not one of those folks who believes email should be replaced by a whole new thing, but I do think a simplification of the stack would be lovely. How we do that without introducing even more new mail related standards is the conundrum.

Sounds like you know enough about this process where you know it can be a lot of setup and careful maintenance.

So if you don't want to invest the setup and ongoing maintenance time, or get no pleasure from doing such , may I ask why you want a simple way to do all this at all?

Seems like you should just use fastmail or gmail or similar?

Configuring and running smtp isn't supposed to be super easy and fool proof. I think most systems that claim that will leave you with an unmaintained email solution that is likely not very secure and probably not being replicated or backed up, for starters, not mentioning all the actual things that go into making it all work reliably.

So if you don't want to invest the setup and ongoing maintenance time, or get no pleasure from doing such , may I ask why you want a simple way to do all this at all?

I'm not the person you replied to, but among the potential reasons would be privacy, reliability and sustainability.

Yeah, i hear you, but I think that its a bit of a silver bullet, to have a 1 click install that gives you all of these things.

Just the 3 you mentioned here, privacy, reliability and sustainability are not going to happen if you are doing a 1-click install, and don't understand how it works.

I find the best way to do this is using a panel like Vesta or Webmin.

There's also iRedAdmin but it's not particularly 'free'.

You can try https://hub.docker.com/r/analogic/poste.io/ - Its complete easy to use solution installed with docker... see https://poste.io/ (author here)

Ugh, closed source.

mailserver is a part of my stack I need to be able to tweak and look behind the curtain at; one-size-fits-all is really hard with email.

haraka

https://haraka.github.io/

Best mailsoftware I used so far. And ridiculously fast

correct me if I'm wrong, but as good as haraka looks (and it looks great), it isn't really a "complete package" like the GP was looking for...

From the Manual page[0]:

> Haraka makes no attempt to be a mail store (like Exchange or Postix/Exim/Qmail), a LDA, nor an IMAP server (like Dovecot or Courier). Haraka is typically used with such systems.

[0] https://haraka.github.io/manual.html

To me, the main obstacle is not the software (I can install a server in VM), it's the implication that I have to rent a VPS, or buy a DNS record, or subscribe to a different kind of ("business") Internet plan, when I already have packets flowing, just so that the email giants and everyone else believe I'm a legitimate participant in humanity. I feel stymied by email giants and ISPs that seem to collaborate to prevent me from doing something that even I agree is simple, to the point that I don't bother. That's 100% a social problem.

I think the desire to run one's own email server today mostly reflects a longing to re-discover the highly-accessible anarchy of the early Internet. Unfortunately, if that is ever to be found, it likely won't be in the form of complying with the highly-burdensome mostly-social requirements of our modern, well-centralized, email system. More likely, it will come from painting over it.

It's not just corporations that would not think your legitimate but everyone really.

These sort of unspoken rules you mention are very largely due to combat spam. If anyone with port 25 open was trusted the amount of spam would be intolerable.

Seems sort of silly to want to use email without taking these steps to make it official as possible.

This is not some manipulative plot by google to get you to spend an extra 60 dollars a year, sorry.

Yes, treating dialup sources as likely spam sources has been done since long before Google became a significant player in email. In earlier years the main approach was to try to convince ISPs to filter outgoing port 25 by default on their dialup IP ranges. Later, people started compiling lists of dialup IP ranges (later expanded to DSL/cable/etc.) to block them at the recipient side, since there were too many ISPs who weren't filtering. Recipients disliked email from dialup IPs because ISPs seemed unwilling or unable to police their customers and respond to individual abuse reports, and so little legitimate email originated there anyway that it was easier to just cut them off.

I don't think the bigger end-user providers have been very involved in developing those kinds of policies. The NANAE crowd was/is mostly administrators of smaller and university servers, not Yahoo/AOL/Hotmail/Gmail administrators.

Most email providers block dynamic IP ranges because of the absolute torrent of spam that flowed from them before the block. Something like 90% of the world's spam used to come from hacked home PCs. Someone would install a hacked version of Photoshop, a "porn downloader", or get drive by installed and their PC would be under someone else's control and able to send out spam after spam after spam on an open port 25 from every ISP in the world. To counteract that, ISPs started blocking outgoing port 25 connections and most email server providers began blocking incoming connections from dynamic IP space. This cut down on spam dramatically.

This feeling of yours is not caused by the large email providers, it is caused by your desire to reach the billions of people who subscribe to their services.

Been doing it for 10 years, and it's still fun :) https://jve.linuxwall.info/blog/index.php?post/2015/03/11/10...

I have a mail server hosted at Linode on:

- a clean IP, not on RBL, not blacklisted elsewhere to the best of my knowledge (outlook.com tells you when you IP is bad)

- also accessible on IPv6

- both IPs having a proper rDNS on my domain

- supporting SSL on port 487 and 565, with a certificate from a known authority

- with DKIM and SPF both passing according to gmail

Yet it ends up in gmail spam folder. And I'm only sending email to myself and 2 other persons, so it's not even mass mailing.

I think there are other factors at play.

Google wants you to do special things for them. Have you gone to https://postmaster.google.com and set up your domain there?

This was first mentioned on HN here: https://news.ycombinator.com/item?id=9905767

(I'm hoping I won't have to partake in this.)

Yes, and even if the tools are not available perhaps the domain verification would help.

I really would have hoped that Google would try to live without asking people to link domains to Google accounts. At least that's what I understand they are doing here. If so, this is a step towards their interest, not ours, and their slogan "don't be evil" pales more every day. Yes, of course some newer competitors of theirs are leading this course by attempting to win over people to their own closed messaging worlds, and users at large don't care, so all of this is sad. (The language on their blog post about this (linked from the mentioned post) is laden with a disappointing amount of weasel words, too.)

So what I'm saying is, no, I don't want to link my domains to my Google account just so that I can send mails to them. And I'm going to hold out hoping that their systems are collecting enough trust in my domains and/or IP addresses in other ways so that it won't be necessary.

Depending on who it is, maybe you can give the people you want to email mailboxes on your own domain?

Thanks, that is super helpful!

I discovered the following for outlook.com: https://postmaster.live.com/snds/index.aspx?wa=wsignin1.0

Are there similar tools for aol.com, yahoo.com and icloud.com? (to basically cover the big 5)

If you do find a tool for aol, let us know. They are the most obtuse when it comes to rejecting emails.

No. Creating an account for every domain I want to send email to is not scaleable.

I was in a very similar position and found all my email bounced by every Cisco Ironport user. Contacting Cisco gave me the usual "please stop sending spam" sort of answer.

I then had a customer who actually had a Cisco support agreement log a case, and was promptly informed that I needed to create an account on abuse.net and register our domain to resolve the issue. I did, and it immediately resolved the issue.

It's a frustrating, terrible situation, where you follow every possible best practice you can find, and you're at the mercy of a third party you've never heard of. I get that RBLs are a similar situation, but you can usually identify when that's a cause.

I've had this recur over the years with 5-6 other domains, but there doesn't seem to be any pattern to it, it certainly isn't an issue with every domain.

That must be heart-breaking and very frustrating. You've done everything you can do to be a good internet citizen and support all the proper technology, but you're still treated badly.

If I am correct, gmail now defaults to a "don't trust an IP by default" procedure. If enough people get your emails in a spam folder and mark it as not being spam, the system will start trusting the IP address. It also has to do with volume.

This might be a good service offering.

If someone could operate a few hundred gmail accounts, they could let people send them messages and mark those messages as not being spam in order to train the filters.

Such accounts would be permanently disabled. Spammers have been trying to use sock-puppet networks for years.

I haven't checked in a while, but sometimes a free service like Sendgrid can help. Send 12,000 emails per month free and the stats are pretty cool. PS: I have no affiliation with them :-) I just looked and they have source code posted on the homepage, "You can send email over SMTP or HTTP..."

Check PBL too

Avoiding spam filters is unfortunately not as easy as setting up rDNS, SPF, DKIM and DMARC. Reputation is also becoming a big issue: http://liminality.xyz/the-hostile-email-landscape/

Again: SPF, DKIM and DMARC are no indicators of spamminess of a source. These systems have a completely different purpose.

Well, I understand their purpose may be different, but it is nonetheless having my e-mails ending up in "Spam" GMail boxes which triggered me to write this post ;)

The title is confusing: even if it helps your messages to be delivered properly, it's not a mean to prove that you are not a spammer.

You have to configure them in order for your mail to be successfully delivered and not put into Spam folder on major email providers.

I've run mail servers for decades without configuring them and have never had issues. Reputation is probably the most important (note that my domains and even some of my servers were online before these technologies existed) and it's extremely important to get your DNS right, especially Forward-confirmed reverse DNS (FCrDNS). Strictly enforce authentication on submission port 587 and segregate user submissions from application generated submissions so you can tweak each configuration appropriately. Keep in mind that marking messages as spam involves a complex chain of weighting, so if a minor adjustment gets your messages accepted, you could still be straddling a line and would benefit from fixing the basics. And never launch a server on an IP without first checking it against blacklists (demand a new one if it's listed anywhere).

Reputation is everything, but when you need to setup a new server on a new blacklist-checked IP for (non-spammy) mass mailing, without SPF and DKIM your emails will most likely go to the Spam folder, in 2015.

Of course, those things are not guaranteeing delivery, but they play an important role.

Google is particularly insidious: gmail will happily throw away email (not just mark as spam) to "new" recipients, while your own account, which will usually already have a "relationship" with your domain, might receive email just fine.

I just recently had an issue where I tried to send an email to a someone I'd just met. The cc-part that went to my gmail-account got through fine. He didn't even receive spam. After I set up spf, I successfully sent an email to the exact same gmail address.

If gmail had rejected the mail, there'd be no problem -- then I'd know that I'd have to take action. Quietly eating the mail... not cool.

I wonder how long until the only way to send email into gmail/outlook is to set up routing rules that send email to gmail/outlook addresses by logging in to those respective services, and sending directly, bypassing traditional unauthenticated smtp... presumably setting up one "major" delivery would be enough, as gmail can't ignore outlook.com and vice-versa...

Yes but you can't start from scratch without them and do a moderate amount of traffic.

If you have the same clean ips from pre dkim/domainkeys days then don't lose them, or it may be an uphill battle which I would be surprised if you didn't engage dkim to aid in fighting at that point.

This is demonstrably not true.

SPF and DKIM are neither necessary for mail delivery nor sufficient to assure delivery.

There is, in fact, nothing you can do to guarantee delivery of your mail once you offer it to another mail server. If the recipient doesn't like it, it will be dropped.

You can do lots of things to help. The most important is to not be a spammer. Don't send substantially the same mail to lots of people who haven't asked for it.

> SPF and DKIM are neither necessary for mail delivery nor sufficient to assure delivery.

This statement, while true, is completely useless. Without SPF and DKIM email providers will view your emails with more suspicion, so that a larger percentage of your email ends up in spam even when it really isn't. SPF and DKIM do not guarantee delivery but they reduce the chance of your email being inappropriately recognized as spam.

It's somewhat true. DKIM checks the authenticity of a email domain. So it definitly helps. SPF does barely the same. that's why you got into spam if you not set RDNS OR DKIM OR SPF. Since the other server can't be sure if the server is allowed to send mails with the provided domain.

Mailservers are simple, basically you can send with every domain available, however that won't work since other servers will handle that via SPF, RDNS or DKIM.

If it's demonstrably not true, why is it then that multiple reports of mail going into SPAM folder stop coming in, once I setup SPF and DKIM? While it's true that reverse DNS is probably a bigger factor. Not having these 2 setup is going to increase your odds of ending up in the SPAM mailbox.

I have reverse DNS and SPF, but not DKIM and I don't have that many issues getting my email delivered. I did have issues with AOL, but once I registered as the contact for my IP address with them, those issues went away. I've also had one or two issues with GMail over the years, but last I knew, it was okay (I normally don't deal with GMail addresses that much).

Then again, I've had my domain for 17 years, self hosting everything for 16 years (with the occasional IP change, but I think I've had the same IP now for almost ten years so go figure).

Existence of SPF and DKIM will not necessarily keep your message out of the "spam" folders, but absence of them will significantly increase the chances of being delivered there.

I've been running my own (personal) email for a while now. And for a while, both hotmail/outlook.com and gmail have been eating my mail. Google is enough of an asshole to not report anything to the sending smtp, while outlook.com/hotmail at least gives you an error, so you know they got the mail all right, just didn't like your sending ip.

My ip/domain name was in no (public) black lists, however - when I finally set up SPF google stopped black-holing my email.

After I managed to get hold of admins of outlook.com via (I think, there were a few redundant hoops I jumped through):

https://support.microsoft.com/en-us/getsupport?oaspworkflow=...

Outlook.com/hotmail.com provisionally started accepting my email again. All this without my domain sending any spam the past few years.

Personally I think SPF is rather silly, but apparently it's considered an important filter-knob by certain services. I'd much rather gmail/outlook require valid certificates for smtp, and turn of plain-text, than all these add-on protocols that are supposed to avoid "forged sender"-type stuff.

Then I'd have to move over from cacert to a "real" cert, but hopefully that bar will be easier to clear once letsencrypt is up and running.

Slight OT/marketing, but I'm constantly having deliverability issues with Yahoo and spent weeks figuring out whats wrong, when spam-tester shows score 10/10, SenderScore 97, and yahoo keeps automatically marking my messages as spam, even when many times I contact client via phone and they swear they never clicked "mark as spam", which I have no reason no to believe.

The spam complaint rate keeps being broken (around 0.3%) because of yahoo, and utilizing Sendgrid as an EPS, I'm afraid of losing ability to send.

At this point I would love to hear an opinion of a e-mail deliveribility expert/veteran, or someone who can help me get off the cloud and host own email server that will be well-configured and maintained. I'm aware this service might come with a hefty $ bill. Please contact me via my email in my profile.

</shameless plug for help>

A different and less comprehensive recipe to compare with for Ubuntu:

https://www.exratione.com/2014/07/setting-up-spf-and-dkim-fo...

If you like Docker you can try Poste.io https://hub.docker.com/r/analogic/poste.io/

Is anyone else peeved at the fact that SpamAssassin is still the de-facto standard for spam filtering?

Works pretty well for post-queue but as soon as you try to pre-queue filter anything you're in deep trouble.

Has anyone tried compiling SpamAssassin or writing a faster version of it in another language? It was a long time since I played with perllibs but I seem to remember being able to load perl code into c programs.

    >Has anyone tried compiling SpamAssassin or writing a faster version of it in another language?

How fast to you want it? I've been pumping roughly 20,000 emails per day through Spamassassin via maia[0], on a relatively moderate VPS. greylisting in front of it handles a huge portion of the load.

[0] https://github.com/technion/maia_mailguard

Edit: Been a while since I checked. Actually yesterday's number was 98,000.

It's been made clear to me now that I need to look closer at greylisting in postfix before mails are passed to the proxy_filter.

But if you're interested I'm talking around 62k mails a day. My experience with this amount has led me to use 64G RAM on each MX but that's only to handle a certain incident with very high load. Usually RAM usage is much lower than 64G and there's plenty of IO cache available.

Spamassassin is old and outdated. It's like Apache and rspamd is like nginx. Give it a shot.

I have used assp for a few years, it's tricky to setup but does a great job once you get there. http://sourceforge.net/projects/assp/

http://dspam.nuclearelephant.com/

It's a bit picky and horribly documented, but extremely fast and low on resources.

Except database size. Will grow to gigs and gigs and gigs and gigs. Dspam can be very difficult to manage with a 400gb token database when you have a large system. ( last version I used was 3.9 maybe they have improved this?)

I use SpamAssassin in a prequeue SMTP proxy and it works pretty well, but it's nearly the last check after many other lightweight tests.

Greylisting is the #1 standard for spam filtering.

Um ... maybe. I run a greylist daemon and I wrote about its effectiveness a few months ago (http://boston.conman.org/2015/04/12.1), which also goes into how effective SPF would be had I actually used it in accepting incoming email.

I also wrote about the effectiveness of the various blacklists out there as well (http://boston.conman.org/2015/05/11.1).

(The TL,DR of the following: SpamAssassin still works very well, and can be used on the incoming SMTP connection just fine ("pre-queue" as you prefer to call it).)

I used SpamAssassin ca. 2000-2008, then moved to Gmail, and recently went back to maintaining my non-@gmail.com addresses with my own mail setup, again with SA. In the last 30 days I got 43 spams delivered to my spam folders, 16 spams delivered to non-existing addresses at my domains (captured to prevent backscatter), and 101 spams rejected right away on the SMTP level due to high enough score, and IIRC zero delivered to my inbox (IIRC even across the last 2-3 months). I'm not sure why so few spams are even being sent to my handful of non-@gmail.com addresses, I've been using some of them on mailing lists for about 8 years now, too. (My single @gmail.com address gets about 40 spams in the Gmail spam folder per day.) I got 2 false "half-positives" in the last 30 days from the same company before training them (and 0 fp to the spam folder); I say half-positives sine I'm filtering mails with a low enough score to a "possible spam" folder, with the idea of reducing the amount of work for checking.

It took some effort to get everything working well (not the fault of SpamAssassin, except for pulling some hair about its relatively messy setup (quite complex and not very clean, so needs a calm mind when doing a non-standard setup)). I wrote some software[0] for this, though some people will shake their heads that I'm using djbdns and Qmail (my motivation for the latter is that I know its workings and that it's the original backend for qpsmtpd).

[0] https://github.com/pflanze/better-qmail-remote, https://github.com/pflanze/mailmover, (and https://github.com/pflanze/tinydns-scm for generating tinydns configurations programmatically using Scheme, including SPF records, once I get around cleaning up the code and pushing it here)

> as soon as you try to pre-queue filter anything you're in deep trouble.

I'm using qpsmtpd as the incoming SMTP server, which is also written in Perl, but it doesn't matter, as SpamAssassin offers a daemon approach (spamd) which even qpsmtpd uses, thus you just execute the small "spamc" program and pass the mail on stdin (or use a library that reimplements the protocol in the language of choice). Given this I see no reason to be in deep trouble, and rejecting spams during the SMTP stage works just fine for me.

PS. yes, you can also embed Perl in a C program, but why deal with the complexities of mixing languages in the same process when scanning a mail takes several seconds of real time and a sizable fraction of a second of CPU time, thus the IPC overhead is completely negligible.

Thanks for the extensive response but with around 62k mails a day, on each MX, I think pre-queue filtering requires too much RAM.

It's been made clear to me, thanks to this thread, that I first and foremost need to explore greylisting in postfix before mail is passed to the proxy_filter. Secondly that I should explore dspam and rspamd as faster alternatives to SA.

I'm not sure I'd put much stock in the linked mail-tester. It has a very incomplete SPF implementation that'll heavily penalise perfectly valid records.

(eg, my record is simply "mx -all". So if a host is listed as an MX for my domain, it's also a valid sender. mail-tester doesn't appear to understand 'a' or 'mx' entries, so fails this entirely.)

This is more or less a problem that you're never going to solve as a normal person running their own mailserver. Hell, apparently Google has problems with it. They mark emails I send from gmail.com to others within my own GApps org as spam because the headers don't match. What??

All this is moot if your server is in a blacklisted ip range. I found it out the hard way. Not just any VPS provider will do, make sure you get one from a company with a clean ip range.

Lots of people think it's a great idea to make packaged all-in-one mail systems, but what you're really doing is making it easier for spammers to get up and running.

Been there done that. But in the end I went with http://mandrill.com/

Kinda off-topic, but I was wondering if anyone can recommend some setup for a CLI based mail client that works with IMAP and Gmail?

I'm on a Mac now.

I used mutt on a Mac and was quite satisfied with it, although I'm kinda "GUI-averse" anyways.

mutt itself is kinda slow if you actually use its internal IMAP and SMTP features over the Internet (such as with Gmail). Instead, I used offlineimap to pull down all of my mail to the Mac and pointed mutt at the (Maildir) directory where it was stored (which was incredibly fast, as you might guess).

In addition, instead of having mutt send outbound mail directly to Gmail itself, I fed messages to msmtp (locally) and let that take care of sending them off to Gmail in the background.

That setup isn't for everybody, of course, but I was very happy with it. In particular, {processing/catching up on} all of the mailing lists I subscribe to was much, much quicker.

I'm using kolab.org for few years, great product overall.

what about os x server?

OS X server mail setup is easy. But, if you need to customize anything it starts to get scary, i.e. If you don't change the settings the OS X way then you risk an update blowing away your customizations. The OS X way is a minimally documented serveradmin tool from terminal.

Oh and OS X server doesn't support anything newer than tls1.0.

So for these reasons my next mail server will be something more mainstream.

Yes---you're absolutely right. Thinking back, I have run into similar problems with the os x server apache implementation, where (a) anything slightly non-standard is not possible from Apple's GUI interface, but (b) editing config files by hand works UNTIL an update wipes it all away, etc.

Reply on news.ycombinator.com