How I Stopped over 1000 Spam Emails/Day from Reaching Me in 5 Minutes | Hacker News Reader

How I Stopped over 1000 Spam Emails/Day from Reaching Me in 5 Minutes

merricklozano.com

31 pointswiredml21 comments

The obvious downside of greylisting is that the first contact takes some minutes.

Another downside is that some poorly written applications try to deliver mail on their own, instead of using a real mailserver.

It is somehow required that the mail user knows about the greylisting setup so that he can request the other application to send the same mail again after some minutes. One real world example are those registration links you get via mail to confirm your account. It happens very often to me that I have to re-request the mail, to get it into my mailserver.

Good points. I used to do enterprise IT consulting and we couldn't integrate many spam cutting techniques without getting a large dose of complaints saying "where is my spam".

In our case, it's just two of us, and the level of spam was overwhelming. I accept the trade-off.

Heh, I always assume I've broken my mailserver when I don't get any spam. Unfortunately, I am usually right.

I don't think many (if any) web services work that way. I know that at my company we set up a Postfix daemon on one of our servers which all of our web servers connect to to send mail. So, when you get your password emailed to you, it's Postfix doing the actual sending, and Postfix will retry for hours, let alone 5 minutes. Same thing for all my personal projects.

One real world example are those registration links you get via mail to confirm your account. It happens very often to me that I have to re-request the mail, to get it into my mailserver.

Oh, you'll get those in an hour or so, when their mailer retires. I have never had a problem.

I did kill greylisting, though, because it breaks when the mail server that first tried to send the message is not the one that tries to send it again. Gmail does this, for example, and unless you keep up to date on Google's internal architecture, you are going to lose mail from Gmail users. Combined with being tired of waiting an hour or more for every email, this killed greylisting for me. Now I let the spammers deal with spamassassin, which is quite nice if you bump up the scores on the content and URI blacklists.

The Gmail issue you highlight I believe would only happen when the original outbound smtp server goes dark before it reattempts delivery. I say this because I have been receiving emails from Gmail with greylisting on for a week now.

Look at the logs and you will notice a lot of initial attempts that are never "followed through" on, and these are from Google's IPs.

And BTW, spammers have figured out RFC822 now, so this doesn't even prevent a lot of spam.

I was talking about registration mails from 'poorly written' applications. Mail comes after one hour. I am cool.

I am aware that most prominent site do this the right way, but just wanted to mention that some do not.

About the problem with gmail, I was not aware of that. Thanks for the info. I will go grep my logs now. :)

Aaahh.. google is inside my whitelist. :) That explains it.

btw: anybody needs to see a list of 'no-retry' servers, check out:

/etc/postgrey/whitelist_clients

The issue there being that if Google ever changes their internal network structure, you lose mail until someone notices. Personally, I don't like coupling myself to other networks that tightly, and I also don't like losing email.

So no greylisting for me.

Spam from @gmail.com is increasing nowadays, AFAICT.

I used postfix-policyd for a while. I whitelisted the major email providers. You can also run the policy daemon in 'training mode' for a while.

http://www.policyd.org/tiki-index.php?page=Greylisting&s...

I still had the same problems others have mentioned though, where crappy registration systems were sending their confirmation mails directly from their PHP application, rather than relaying it through a smart MTA that actually understood the SMTP protocol and would do the retry.

Ultimately, too many users complained about mail delays and missing email. This is one of those scenarios where the cost of not delivering an important email may outweigh the benefits of blocking a few more invalid ones.

I used a hosting provider who had this turned on for a while and it often had odd behavior. There are a ton of email servers that are not configured properly or are configured in complex ways. The worst case I saw was with a large corporate email system. They used some kind of round robin/load balaner. Incoming mail went through one mailserver, outgoing through multiple (different) mailservers. The greylisting would therefore always get caught in an infinite loop. Here's a rough example:

1) The greylisting would initially reject the mail from bigcorp.com. 2) Bigcorp.com would get the rejection, but the re-send would come from a different mail server: mailserver1.bigcorp.com. 3) Since this is a new IP address, the greylisting would bounce the new email. 4) Bigcorp.com would re-send, but this time from mailserver8.bigcorp.com. 5) rinse and repeat

Now, you can say that the guys at bigcorp.com are boneheads and they should hide these details from the outside world. But the reality is, as an end-user, I don't care. I just want to get my mail.

Hate to say thing, but I too had similar problems. Turns out that postgrey works for a while, but the spammers know how to work it better than you think. So for a few days it worked, then slowly started to degrade back to where I was getting 50+ spams a day that should have been greylisted...

Gave up, postini... $1/month -- more than worth my $$/hour.

This is exactly why I find running your own mail server does not pay off. Fighting spam is a full-time job and should not be taken lightly. Even large enterprises move the majority of their e-mail hosting to cheap providers like Google. (Unless some other company threatens to stop giving you their business if you go that route)

I've worked on a couple of antispam solutions that did something akin to a more intelligent grey listing. Think quality of service meets IP reputation.

The problem with braindead simple things like postgrey is that they are incredibly easy to circumvent and the more attractive a target you are, the quicker the spammers will get around the solution. For instance, if one of the major providers (Yahoo, Hotmail, Verizon, Comcast) were to implement this, spammers would work around it within the hour. If you're low enough on their radar they won't care that they can't get email through to you but as soon as it starts to bother them enough to care, it stops being effective.

One very useful trick is to use www.dnswl.org (a DNS whitelist) before greylisting. It lets trustworthy mail servers right through, which almost negates the downside of greylisting (delayed mail).

For our small software company's email, we use Google's Postini service that runs $12/mail account/year. Emails are routed to Postini which filters the spam and then passes the good messages on to our mail server. It was pretty straight forward to setup by updating DNS and configuring the Postini account. I don't recall having do do anything special on the mail server or mail clients, but it has been a while.

Yeah, the grey listing. The anti-spam technique that works only because it is not widely used, so... shhhh :)

Gmail will handle those final 20 :)

I think you're required to post a video interview with a title like that.

Reply on news.ycombinator.com