Spam blocklist SORBS closed by its owner, Proofpoint | Hacker News Reader

Spam blocklist SORBS closed by its owner, Proofpoint

theregister.com

60 pointsBrajeshwar41 comments

So, while the loss of SORBS is troubling (I mean: yet another initially-volunteer-driven and widely-used project burning out...), it, like other DNSBLs, is not very relevant to modern spam filtering anymore.

With, like, 80+% of inbound SMTP traffic coming from Google, Microsoft, Amazon and assorted non-malicious transactional/list-based SaaSes, a simple 'I either like or dislike the sending IP' approach has been infeasible for many years.

DNSBLs have been effectively the same as turning your mailer off, for at least 15 years.

Sorry, "turning your mailer off" seems outbound-related to me, whereas DNSBLs are typically used for inbound filtering?

And in 2009, filtering inbound SMTP traffic using a popular DNSBL or two was definitely effective (as was greylisting). Alas, no more.

But I probably misunderstand what you're saying?

> With, like, 80+% of inbound SMTP traffic coming from Google, Microsoft

Just do a challenge-response on incoming mail from new addresses on Gmail and Outlook. Send an auto-reply telling them they can get their mail through if they copy a code in the reply. That will kill most spams from those sources.

isn't this essentially handing police powers to a duopoly of USA commercial corporations?

I think they mean that gmail/outlook are highly abused and you can reduce spam by demanding a challenge response from new addresses on those domains when they email your domain.

Yes. I'm talking about spam originating from Gmail and Outlook, not received by them.

We already did that.

A few years ago, I would have said that the challenge-response approach was doomed to failure, but in light of the concentration of spam and phishing emanating from Gmail accounts, Gmail addresses are the new IP addresses. How long before someone starts distributing a Gmail address blocklist?

That's a good idea in theory, but considering IPv4 addresses cost money and Gmail addresses are infinite for practical purposes, that wouldn't work. I don't know that I've ever seen the same Gmail address used more than once for spam and phishing.

It'd be nice if Google did something about this. Until they do, I tell everyone that uses Google for email that they have to accept that they're hosting with one of the biggest sources of spam on the Internet that, as far as I'm aware, does absolutely nothing when spam from them is forwarded to their abuse addresses.

> Send an auto-reply telling them they can get their mail through if they copy a code in the reply

Never auto-reply to any email ever. You're only making the spam problem worse.

(Plus, if you think there are not actual persons behind most Outlook/Gmail spam, I've got news for you. They will reply, and beg you for another chance, sometimes in highly emotional terms).

"550 5.7.1 The recipient has set a policy that prohibits email from this sender" at the SMTP level is the only way forward here.

It certainly doesn't solve the spam problem by itself, but as someone still DIY hosting their email, I find the Spamhaus Zen DNSBL has non-zero usefulness. Mostly filtering out some of the botnet spam that gets past the initial "must have RDNS than matches forward DNS" Postfix check (which gets a bigger chunk of it).

I still love using DNSBL as part of a score-based approach. Add Bayesian filtering, spf/dkim scoring, a greylist, and postfix options like RDNS checks.

I see a few spam a week, have never seen a false positive. I think greylisting made the biggest difference after sane-sender checks.

Where do you get this 80%+ spam coming from MS etc from? My mailserver still gets heaps and heaps of spam from random IP's that an RBL helps block. I use rspamd which takes a number of RBLs into account. Yes, RBL isn't the only thing it uses (there's Bayes, Neural Net, dmarc/dkim/sfp, razor/pyzor as well as others). Very little spam comes from the big players. I DO see a lot of spam attempts from *.onmicrosoft.com domain names, but no one legit sends from that domain so that's just blocked outright. If the argument is 80%+ of email is _handled by_ MS/Google etc then yea, I agree. But us losers out here still doing our own mail still see masses of spam from random IPs that RBLs still help block. All that said, seeya SORBs. Your data was always hot garbage and so full of false positives as to be useless.

Strict SPF enforcement will get rid of most of the random-IP spam. Then, you'll identify a few hosting providers that actually get SPF right, but are very easy to block based on rDNS or name servers. And then it's really mostly Amazon/Google/Microsoft and assorted transactional/list SaaSes...

I disagree.

Looking at just a random spam I have here, I see it passed both SPF and DKIM happily. But it was marked as spam in HostKarma, Spamhaus, Truncate and flagged as Bulk by Razor.

So I dropped it. Looking at heaps of my attempted Spam emails I see the same. It seems most spammers setup SPF before attempting, or are hijacking legit sites/mailservers to send the spam.

Pretty much the only Spam I find I reject based on DMARC as well is just the "I hacked your webcam" blackmail spam that tries to "prove" they hacked you by spoofing my email domain.

I think if I disabled all RBLs the other huristics rspam gives me would still catch 90% of the Spam, but the RBLs certainly help me still catch 99.9% of the Spam attempts I recieve.

> With, like, 80+% of inbound SMTP traffic coming from Google, Microsoft, Amazon

Factually untrue. You are a teenager spouting self-important garbage.

I think SPF+DKIM+DMARC has taken over most verification of email online already.

I've had it setup with protonmail for a year now and it works very well, easily automated with providers such as AWS route53 for example. So technically I could have hosted my own email that whole time, just replace the protonmail senders with my own.

As long as I get a VPS IP without a tainted reputation.

SPF+DKIM+DMARC are a classic case of Goodhart's law, the amount of spam they stop these days (at least anecdotally) is minimal. Most spam I get seems to come via SalesForce infrastructure, and a variety of similar bulk email marketing providers

SPF definitely stops most 'stupid' spam (with the second-most valuable metric being EHLO-to-rDNS correspondence). Now, Salesforce and most other non-malicious transactional/list-based SaaSes present other challenges, mostly solved by applying SPF to their content From: header in addition to the SMTP 'mail from' address.

This also involves promoting sender domains from 'DATA reject' to 'MAIL FROM reject' based on behavior, since most spammers see 'MAIL FROM accept' as a win, and won't check any further results.

Proper SPF/DKIM/DMARC at least prevents brand reputation abuse via spoofing (in many cases), which at least blocks a good amount of bullshit phishing and BEC efforts.

They kind of solve a different problem though - verifying that the email sender is really authorized for that domain.

Nothing prevents spammers from setting up their own domain with valid spf, dkim, dmarc.

DNSBL are good to have in addition to checking SPF+DKIM+DMARC. This is because spammers now mass produce domains with valid SPF+DKIM+DMARC using Cloudflare and GoDaddy for domain registration, WHOIS and DNS because neither company gives the slightest care about abuse (and actively go out of their way to make reporting abuse arduous), and the source IPs could be anywhere in the world.

One thing that does work to combat these SPF+DKIM+DMARC valid sources, I've found, is to reject all email from supposed email servers which either don't have working reverse DNS or which have reverse DNS which has a name that differs from the HELO / EHLO name.

It used to be enough to just insist on working reverse DNS that has matching forward DNS resolution, but recently I've been testing requiring that "Return-Path" has the same domain as the HELO / EHLO.

Another thing that I've noticed but I'm not sure how to check in sendmail is to reject email that has a "Return-Path", "From", "To", or "Reply-To" that's a Gmail address when the email isn't from Gmail. I'd like the same test with Outlook / Hotmail and Yahoo. I've found almost no instances of legitimate email sent this way.

Many things to consider, now that SORBS is one less tool to use!

sorbs sarved me well for many, many years. RIP :/

What happens with (not very frequently maintained) mailservers or appliances that used it? There are some alternatives, but not too many of them, and having one less makes the whole system more vulnerable to DDoS or legal actions (honest or not).

If your mail server or appliance still relies on DNSBLs, it's high time to upgrade to a solution that doesn't.

These days, a list of 'compromised Azure tenant IDs' (or domains abusing the Gmail API, or dodgy Salesforce senders: Microsoft is by no means the only issue here) is way more useful than anything source-IP-related...

> a list of 'compromised Azure tenant IDs' [...] is way more useful than anything source-IP-related...

Where do I get such a thing?

A large portion of my incoming spam seems to be coming from "xxxxx.onmicrososoft.com", which sounds like it might be what you're talking about.

Well, mostly from your incoming spam. The header you're looking for is `X-MS-Exchange-CrossTenant-Id`. If it ends in `aaaa`, don't touch it, as that's the freemail-outlook.com, but otherwise, feel free to (test-)reject it.

To get started: 41a71966-4fa6-4839-a87d-034d66bdda33 d931cb4a-3984-4328-9fb6-96d7d7fd51b0 e85f2c00-2730-4ca5-b8d8-609b15bd4746

(all seen-in-the-wild compromised instances in the past 14 days)

Hey there, you should start a blocklist!

Yeah, no, I'm good -- having seen what happened to the ones that came before me, I'm quite happy to limit myself to policing (nah, gardening) my own little corner of the Internet...

About 70% of my spam originates from *.onmicrosoft.com. Unfortunately I can't easily block the whole thing because there is also legitimate email traffic from Office365/Azure.

I have tried sending Microsoft reports, but have not heard back, and the spam continues.

Yes, Microsoft is very slow in blocking their customers from sending spam, yet very quick in blocking external senders for that reason (same for Google, Salesforce, Amazon, etc. BTW). Funny how that works...

But, if you can, record the `X-MS-Exchange-CrossTenant-Id` header value for the spam you receive. If it ends in 'aaaa', that means it comes from the public outlook.com/hotmail.com service, and you'll need to do text content/from-address filtering to get rid of spam.

But otherwise, deny-listing the GUID you get, will do wonders to eliminate future spam from that source...

For anyone interested, here's the list for the last month or so:

https://gist.github.com/digitalresistor/03ea1b8798c519a71f06...

Edit: moved list to Gist.

You... seem to get a lot of spam! Just out of interest, across how many unique local recipient addresses is this, and how did you determine these messages were illegitimate?

Single user... me. My email address is used on all my git commits/mailing lists across the web.

I check my junk folder every other day to make sure that legitimate mail does not go through because I've set my rspamd config pretty tight.

So all of these are classified correctly as spam by human eyes.

Interesting, thanks! For what it's worth: my multiple-thousands-of-users mail server hasn't seen any of these Azure tenants in the past 14 days.

That's interesting to say the least... it means that there are MANY MANY more azure tenants that are used to send spam :/

That doesn't seem too surprising. While my account just gets three or so "digital marketing" or "mobile app" spam a day from Outlook, Mom was getting dozens of Apple / Home Depot / Harbor Freight / Lowes phishing spam a day from Outlook. Reporting them did absolutely nothing, and there were no identifying patterns beyond the painfully obvious fact that they were all from the same campaign, so I'd wager that creating unique accounts on Outlook is trivial.

The 'digital marketing' and 'mobile app' spam is, in my experience, mostly sent via 'retail' outlook/gmail/aol/yahoo/hotmail.com accounts, and mostly by actual people pasting the address list into the BCC field.

These are not that easy to filter due to the risk of false positives, but in general, a sender with a From: header matching '.*\d{1,}@(outlook|gmail|aol|yahoo|hotmail)\.com`, no To: header matching the actual recipient, and a number of keywords in the message text can be safely rejected as bizdev/SEO spam.

The big-brand spam is actually pretty easy to filter, as there are always 'tells' in the message structure. Even just requiring a match between From: display names and domains yields pretty good results, especially if you normalize the display name to eliminate homoglyphs and nearly-similar spellings.

As a former sysadmin at a business ISP about 25 years ago, I do not miss SORBS and friends. They probably got better when taken over by a professional team, but by then I had moved on into another industry and did not have to manage mail relays. I don't miss that one bit.

Running a DNSBL is/was hardly a glamorous job. It takes a peculiar character to withstand the endless onslaught of complaints from unscrupulous marketers, clueless sysadmins inadvertently running a spam cannon, and end-users feeling the need to threaten lawsuits because their church distribution list got blocked.

Reply on news.ycombinator.com