Why you shouldn't jump on the SPF bandwagon (2005)
david.woodhou.se
david.woodhou.se
SPF is not too bad as far as forging countermeasures work. It's relatively simple both to implement and to check. I'm not against it.
What I don't like is every single other standard they pushed later. I personally think DMARC is borderline useless. DKIM is plainly horrid and breaks just about everything you'd expect from email such as mailing lists, while not solving anything both from a legitimacy perspective and from a spam perspective.
Like everybody says, the "recommended" solution is to pour each and every of these half-assed solutions into a score system. Which sucks, because when a legitimate email is rejected, the fix is never trivial: it could be just a perfectly legitimate host which decided that all these solutions are crap (and they're right).
You know my current 90+% spam and scamming source by volume? It's gmail.com. It's passing all these checks, of course. It's the reason I consider DKIM virtually useless even from a legitimacy perspective: a valid DKIM signature from any large/free email provider bears no significance to the point that even if I had a decent UI for validation in my email client, I would basically have to avoid it: "oh, right, another legit scam from gmail.com".
All the barriers are just increasing the power of gmail, making it impossible to filter it off.
It's mostly likely that the volume of spam will remain constant, whatever we do. But the current fight (led by Google, for some reason) is just breaking the email federation. By the way, now that we have all the natural language tools, what was made of content based filtering?
The threat to the federation model is clear though.
DMARC just specifies reporting and policy, DKIM is almost useless (except as SPF fail override) without it (because you don't know if it should have signature or not)
It was the only thing that really stopped the phishing directly against our domain. We built a lot of tooling for other attacks we were seeing, but that piece was critical.
The trick with DMARC is that receiving email servers have no way of knowing how strictly you’ve implemented SPF and DKIM, so they guess and make their own rules unless you setup DMARC to tell them you’ve been thorough.
BUT, there is a use for SPF. You add it as a factor in your spam filter. Deduct points for an SPF failure, or make it an aspect among many in your filters. It's still a useful way for one org to tell another org, "here are the IPs you should expect mail to come from".
In particular, Gmail completely silently drops emails forwarded by hosts that aren't explicitly authorized by the sender's DMARC policy, which sucks because it means I can't forward such emails from my _own_ addresses on other domains to my _own_ Gmail account. I understand this is because the sender's DMARC policy says that the receiver should hard-reject any mail delivered by unauthorized middlemen, which seems naively short-sighted given that anyone can deliver email, and DKIM already lets the receiver verify the authenticity. (Is this even compliant behavior?)
Does anyone have a reasonable workaround for forwarding to Gmail in such cases?
I always wondered why gmail wouldn’t have a (per account) setting to say: “I am forwarding mail from this host / domain / address”, with corresponding trust of Received: headers chain. If I could then add a header in forwarded mails requiring that to be set, somehow, then we could avoid this SPF / DMARC problem entirely. The fundamentals are there (received: headers and DKIM). All we need is some last mile config :(
But it would promote decentralisation, so I’m not holding my breath.
[1] random search result of such an implementation: https://github.com/tornadoweb/tornado/pull/1864/files
In fact, I was possibly about to next month as a way to easily relay from an about to be disused address to a new address at gmail (or whatever my "friend" chooses to migrate to.)
How so?
Compare to e.g. an outsourcing firm who wants their contractors to be reachable at duder@dudesinc.com, but doesn’t want them all to have to manage another inbox. This is very hard to achieve, due to the problem mentioned above.
This was not an actual business , just friends playing around with servers back in uni. But it made me realise this is a pretty fundamental issue with SMTP. The protocol is doomed to centralise.
I had that same problem; I ended up adding all those users and addresses to /etc/postfix/relocated.
Mind you it does nothing to protect you against the token bearer fooling with your mailbox.
Unfortunately not a realistic solution if you want to offer forwarding to your users without holding their hand and assuming even more config and management woes.
This was a shared host, with a domain, and a bunch of user accounts. In theory, every user does 'echo me@ilovemomma.com > ~/.forward' on first login and they’re all set! Old school UNIX elegance. But in practice, it marks the start of a slow descent into the spam swamp.
I had most of the traffic moved to procmail/spamassassin, and added greylisting, but there is still enough spam going to gmail that I was having deliverability problems. These show themselves as gmail giving soft fails and a huge queue building up.
My current solution is 2-fold:
1) It turns out that gmail is likely to soft-fail the spammiest incoming messages. So I have a script that deletes them from the queue, with rules that match addresses that get the most spam. That way my IP only gets 1 badness point per spam message, not dozens for all of the retries.
2) I moved the outgoing email I care about to a different IP.
The user then adds this "external account/mailbox" in their Gmail settings and Google will automatically retrieve any mail from that mailbox (via POP3) and place it in their Gmail account.
score SPF_FAIL 0 0.919 0 0.919Maybe the issue with SPF is in the messaging around it?
I don't care if it doesn't work, if 90% of every day emails are getting rejected without SPF, then I setup SPF.
Edit: For serious email delivery these days DKIM is required...
At the same time I do agree with the author that SPF might be harmful for the rest of the internet by vouching for e-mail from a certain sender but if you're relying solely on SPF then you've misconfigured.
The Alternatives the article lists reflect the date (2005). Domain Keys and Cisco's alternative merged to become DKIM. DMARC came later.
Today I would suggest for most people to still use SPF but with ? or ~ (tilde), (Neutral or Softfail), and never - (Fail). So it becomes part of the spam scoring and not decider as there are too many false positives with SPF, as highlighted in the article.
Combine it with DKIM and at least initially a reporting only DMARC and you are on your way to a good set up.
SRS turned out to be a disaster. Idea good, implementation with spam bad.
Life is too short to waste it on spam.
If you can't set up proper SPF/DMARC/DKIM then I don't email me at all.
"-"(Reject/Hard Fail) for SPF is broken and should be avoided. SPF only considers the sender and not the transport and recipient. It does not consider recipients' forwarding alias rules, backup MXs, multi relay domain setups, etc. I.e. normal email infrastructure and usage patterns.
I have subset of users that uses Gmail as their client, SPF with "-" from a random valid sender would not allow me to redirect those emails to Google's servers.
Also many aliases on my domains forwards to other addresses not on hosted by my servers (subsidiaries, personal accounts, mailing lists, etc), Spf with "-" will force the end SMTP server to block those emails as doubtful your SPF listing has listed my SMTP servers.
And if a recipient domain has a more complex SMTP setup with mutiple SMTP servers acting as incomming, outgoing, bastions, backups, webmail, sharded storage, etc then any redirecting between them would again break with a strict SPF.
As said before use: DKIM, DMARC but make sure SPF is set to not Fail as it is broken. Use Greylisting, Spamassassin etc to score spam to avoid false positives that are not obvious rubbish enough to reject on envelope details alone.
Fastmail has a very good default recommendations: https://www.fastmail.com/help/technical/senderauthentication...
My own Postfix doc's DKIM section: http://flurdy.com/docs/postfix/#ext_dkim
If that's so, this is purely a theoretical discussion (which has its own merits, of course).
Gmail will accept anything that isn't actively failing anything, even spoofed nonexistant domains, but it will likely get flagged as spam.
If the host is the (or one of the) A/AAAA records for the From domain it'll generally not flag as spam.
If the host is not, then SPF records are required to be reasonably confident it won't get flagged.
Office 365 is a bit pickier and Yahoo is a pain in the butt. Fortunately I deal with mostly business systems so telling the end user "stop using Yahoo for your company voicemails" is reasonable.
And forget about sending from a DO Droplet due to poor IP reputation. A friend of mine at an ISP confirmed their third party SPAM scoring provider (like Symantec/ Brightworks) gives them a list of IPs to block at the edge, which includes large swaths of DO’s IP blocks. Not sure what the story is for other low-end VPS providers like Linode, etc.
Testing deliverability has become completely absurd. Every ISP and ESP has a completely different method for mail rejection. And thanks to outsourcing options for SPAM scoring, is subject to change at a moment’s notice.
This is a glaring omission in all Net Neutrality discussions I’ve seen, the ability to simply communicate an order acknowledgement to a customer. Even outsourcing to Mailgun, SendGrid or the like is no silver bullet for this issue, you’ll still have interruptions in delivery because every day some spammer gets through their checks and trashes that shared IP you’re sending from. And probably you’re having unnoticed delivery failures because someone else reported it / got it fixed. Renting a dedicated IP isn’t really a solution because you have to send from IPv4 to reach the guy who is still doing email @hisdomain.com and doesn’t even know IPv6 is a thing. And IPv4 addresses are in short supply.
I have used spf many times to distinguish forged phishing email vs phishing email originating from a compromised account.
SPF is like most security solutions,its purpose is to reduce risk not eliminate it. If you really want to go down the rabbit hole of eliminating risk,your first step should be to not use email at all.