Learn and Test DMARC | Hacker News Reader

Learn and Test DMARC

learndmarc.com

374 pointstimsneath58 comments

Great way of pushing the critical email services we all need to reduce spam. While I have always wanted SPF, DKIM and DMARC to be enough of an incentive for the businesses i work with, reputation is often not enough of a driver to prioritise the investment.

But fret not! For when you are dealing with companies which want to communicate with customers in a trusted way, there is a marketer's dream standard - Brand Indicators for Message Identification (BIMI) - now security isnt the only outcome, you get a pretty logo too! https://www.litmus.com/blog/what-is-bimi-and-why-should-emai...

I have used BIMI at multiple companies now which talk about Customer Experience to drive the proper (P=Reject) implementation of DMARC.

DMARC still has some issues. From a few years ago: https://i.blackhat.com/USA-20/Thursday/us-20-Chen-You-Have-N...

> Unfortunately, neither SPF nor DKIM provides a complete solution for preventing email spoofing. SPF authenticates the HELO/MAIL FROM identifier and DKIM authenticates the d= field in DKIM-signature header: neither of them authenticates the From header displayed to the end-user, which means that even if an email passes SPF and DKIM validation, its From address can still be forged.

A lack of DMARC+ on an email domain is definitely a problem, but DMARC+ alone still doesn't solve the "is this the real sender" problem.

It inst perfect, but it doesn't need to be unless your risk is disproportional to the market. Risk will always be there, you just need to manage it inline with your corporate risk tolerance - and implementing DMARC to P=Reject is most likely going to (very likely exceed) that approach.

Yes, some companies have elevated risk here (Banks, Payment Processors, Social Media companies) - but honestly most don't.

Btw - this is as much an acceptance as a survival strategy - nothing will ever be perfect, not without significant cost & impact elsewhere. Survival of the fittest these days is managing (and please the understanding of) risk better than others

As a practical example -- it's pretty common for companies to delegate email to a provider like Gmail. Some infosec folks consider this best practice, and Google will allow you to configure DMARC to say that only messages originating from their servers are legit.

However, this does mean that anyone who can suborn Google's mail servers can use them to spend spoof emails that DMARC will rate as legitimate -- and last month, there was announcement of vulnerabilities (since fixed) which allowed a third party to abuse email-forwarding features to do exactly that. See https://arxiv.org/pdf/2302.07287.pdf

We literally just ran into another issue with SPF: the SPF Lookup Limit [1] , which could cause receiver servers to bounce your email back with an "SPF PermError".

If your SPF record causes receiving mail servers to lookup too many domains, some receiving mail servers will reject your email, even when the email itself passes all SPF/DKIM/DMARC checks.

The tricky part of that to diagnose - which [1] talks about, and links to a tool to diagnose it [2] - is that there may be additional lookups that the servers you list in your SPF cause to happen.

So you could have an SPF record with only 4 servers, but if one of those servers causes 7 additional lookups, you might have over 10 SPF lookups. 10 seems to be a growing-in-popularity limit on SPF lookups.

So even if you have SPF, DKIM, and DMARC setup, make sure you don't have too many lookups caused by your SPF record!

1. https://easydmarc.com/blog/spf-too-many-dns-lookups-error 2. https://easydmarc.com/tools/spf-lookup

Hey - Managing SPF RFC limits in large environments can be rather challenging - given the limit of 10 entries. (especially before you adopt DKIM)

Some vendors such as ProofPoint offer services such as SPF hosting, which allow you to work around this constraint. (https://www.proofpoint.com/au/resources/solution-briefs/proo...)

What's keeping me, (as an attacker) from setting up phishing domain with same logo for BIMI?

You will need Verified Mark Certificates (VMC's) issued by a Certificate Authority.

isnt BIMI like $1000/yr?

Yes, it is - well worth it for companies that perceive value for that investment, but that is all part of the value analysis.

See how DMARC, SPF, and DKIM work interactively - https://news.ycombinator.com/item?id=29869266 - Jan 2022 (108 comments)

Does anybody know open-source or, at least, free way to process DMARC reports?

I have several e-mail domains with SPF, DKIM and DMARC enabled, and it works, but I have two annoying problems with DMARC:

(1) Some sites like to send DMARC reports which says "you send us 3 messages, everything is OK, all checks are passed, you are clear".

(2) Sometimes my domains are used to (try to) send spam via other servers and I got DMARC reports like "this <IP> tired to spam with your domain in HELO/FROM and we killed it, as checks failed".

Both reports are of no use for me: I don't want to know, that my users send mail to @gmail.com and @mail.ru (first reports) and I can do nothing about second case, as these <IP>s are not <IP>s of my server, so what should I do?

Some filter or dashboard will be very useful, as unpacking & checking XMLs by hands are very cumbersome.

dmarcian have a free "personal" tier: https://dmarcian.com/pricing/

parsedmarc (https://github.com/domainaware/parsedmarc) is the one I've had starred to look at, but no first-hand use yet.

Here's a script I wrote for myself: https://github.com/hannob/rpter It gives a summary of reports, and details for failures. Not super sophisticated, but should be simple enough to extend. Also parses SMTP-TLS reporting.

Very cool.

> For DMARC to pass, DKIM and/or SPF checks need to pass and the domains must be in alignment.

AFAIK this is incorrect.

It is not "and/or" but rather "or" - only DKIM or SPF needs to pass. There is no method to require both.

This was a recent problem with Cloudflares partnership with MailChannels[1] that allowed email spoofing which was related to this.

The basic problem being that mailchannel did not require authentication - cloudflare workers could just hit an API endpoint on mailchannel to send email. Mailchannel required you to add an include: record to your SPF policy. This allowed anyone to impersonate anyone else due to mailchannel being a valid sender for all domains.

Only ~400 domains of the 2M hosted had DKIM set up but even if they did the passing SPF caused DMARC to pass.

[1] https://blog.cloudflare.com/sending-email-from-workers-with-...

I mean i dont think requiring dkim would stop attacks based on totally broken authentication. In that scenario, mailchimp might as well be signing the emails for the incorrect domain as well.

True. MailChannels, not MailChip though. They handle email for a lot of webhosting providers.

Appreciate the skepticism; it keeps us on our toes. Let's cut to the chase:

Domain Lockdown: We added this to our Cloudflare Workers integration. It mandates a DNS TXT record to authorize a Worker to send emails from a specific domain. You can't forge the CF-Worker header, so impersonation is off the table.

Pre-Lockdown Vulnerability: Yes, we were more exposed before. Thanks to the researchers who pointed it out, we've patched this up with Domain Lockdown.

SMTP Relay & Web Hosting: Domain Lockdown isn’t mandatory yet in the rest of our service for the web hosting industry. But we’re developing updates for our cPanel WHM plugin and other integrations to make this scalable for millions of domains. Note that our service has to work for applications like public mailing lists where locking the sender domain down breaks stuff.

Scale & Standardization: We service a broad range of configurations. Rolling out universal changes takes time. We're also working with industry groups like M3AAWG to push for improvements to DMARC and other standards to help everyone be more secure.

Tech docs for the curious: https://support.mailchannels.com/hc/en-us/articles/456589835...

Appreciate all the questions and criticism here. reply

Thank you for the transparency and the update.

I didn't mean anything negative towards MailChannels - I know this is a weak link in the chain for many people. I did know it was resolved and I should have mentioned that but I couldn't figure out the search query to find that post.

No worries. My initial response was horrible; trying to do better.

Dunno why you're being downvoted, you're correct that it's only "or".

And if you use IP address literals instead of paying for a domain, you get a free SPF:

If the "From:"/"Reply-To:" fields contains only the email address with the IP address literal, you get your "SPF" and it gets a much better score in order to avoid grey listing for the first transaction. And if there are no URLs in the content, even better.

But that is common sense.

I think you are just misparsing their grammer. I believe and/or just means inclusive or. It does not mean "and" is neccesarily an option.

I think its unambiguous. If they want "and/or" to just mean inclusive or why not just use "or". Its shorter, simpler, easier to understand. I also feel "and/or" in general usually means either option are possible/acceptable.

The point I was making is that "DMARC passes if DKIM _and_ SPF passes" and "DMARC passes if DKIM _or_ SPF passes" are both true - you can't specify "DMARC passes if, and only if, DKIM _and_ SPF passes"

> If they want "and/or" to just mean inclusive or why not just use "or". Its shorter, simpler, easier to understand.

The english language being ambigious or not making sense is not a new issue. All i know, is i almost always see "and/or" used to mean inclusive or.

Also the dictionary seems to agree it means inclusive or: https://en.m.wiktionary.org/wiki/and/or

I agree the statement "DMARC passes if SPF (inclusive or) DKIM passes"

I was only trying to point out that "and/or" is ambiguous (which we agree, and Wikipedia even agrees - 2 paragraphs for a defintion, 5 paragraphs of criticism) and one of those ambiguities is thinking it means logical AND - which is something DMARC does not do.

Using just "or" would have been fine and conveyed the same meaning as it does now.

I really appreciate the iterative way it goes through the process. It's been a few years but this would have been a godsend at a previous company when we were trying to move to self-hosted email sending with all the proper security measures.

I sent an email via Apple’s “Hide My Email” service [1].

> Unhandled Promise Rejection:

> TypeError: a.from.replace(/[<]/gi," is not a function. (In 'a.from.replace(/[<]/gi,"(")', 'a.from.replace(/[<]/gi,"' is undefined)

> dist.min.js:3:32767

This error occurred after the interface began displaying the following information:

> Here are the message headers and message body:

> DKIM-Signature: d=icloud.com s=1a1hai

It’s been over a year since the website was featured on Hacker News (January 10, 2022), so I suspect that the JavaScript code may have become outdated and non-functional. It’s possible that it never supported Safari browsers in the first place, or perhaps it’s a combination of both issues. Nevertheless, I’ve learned a lot from the initial [2] and second [3] parts of the DMARC test, which gives me some insight into what might be happening in the subsequent steps.

[1] https://support.apple.com/en-us/HT210425

[2] dig +noall +answer -t TXT <EMAIL_DOMAIN> | grep -i SPF

[3] dig +noall +answer -t A <HOSTNAME>

You sent an email without a "from" field and it broke. Programmer didn't think to test for bad users doing bad things. Nothing special here, no big conspiracy.

It's not a "bad user doing bad things," it's a widely-used email forwarding service.

If this is true and Apple's service does leave out a "from" header (it may just as well be a parse error somewhere on the website's side) that would definitely be on the mail forwarding service, not on this particular website.

RFC 5322 and RFC 2822 specify that at least and at most one From: header must be present. Mail services that don't add at least some kind of fake From header aren't spec compliant and should probably expect error and delivery problems. RFC 2822 is over 20 years old now.

In theory Apple's service could only be RFC 822 compliant, but this does pose a big interoperability problem for its customers.

I also get the same error (in Chrome) when testing a forged email:

  telnet learndmarc.com 25
  Trying 87.239.13.42...
  Connected to learndmarc.com.
  Escape character is '^]'.
  220 allspark.uriports.com ESMTP URIports Mail Portal 1.03.2 Sun, 01 Oct 2023 21:55:40 +0000
  HELO there
  250 allspark.uriports.com Hello <my host> [<ip address>]
  MAIL From: me@example.com
  250 OK
  RCPT To: ld-49101f55f6@learndmarc.com
  250 Accepted
  DATA
  354 Enter message, ending with "." on a line by itself
  .
  250 OK id=1qn4QF-00CUhd-5j

It's funny because while I was typing this, it's like "you don't have to write a love letter". Maybe not, but you do have to repeat the From: and To: header in the data segment, I guess.

I remain amused at how much email I've sent over the years with "HELO there" instead of my hostname in there. I also wonder what % of internet traffic is "Enter message, ending with . on a line by itself".

As DMARC relies on the RFC5322.From address, omitting it will lead to errors. To avoid these errors, emails lacking this address are currently being ignored.

It is absolutely astonishing that we rely on layers and layers of shims/compatibilities/hacks to keep a technology that was well-meaning and ideal 30ish+ years ago running in the 21st century.

Same thing in the VOIP/telecom space.

Microsoft recently had issues with mail deliverability - most of our O365 tenants had a notice reminding us to check SPF, DKIM, DMARC (we're configured properly already) - some of our tenants were having issues mailing smaller mail providers (ISP-level) because the small provider is outright blocking IPs and IP ranges due to spam coming from the same IP address/mail server we're trying to send from.

Fun fact: sns.amazonaws.com still has no DMARC record. This is where AWS SNS messages originate from unless you use a custom domain, and it's where all CloudWatch alerts come from (no-reply@sns.amazonaws.com)

This is how email is supposed to work. In reality, there are whitelists ...

And blacklists. And predatory blacklists. And blacklists that amount to organized extortion.

Hello UCEPROTECT. The only blocklist that matters is SPAMHAUS. Other blocklists at the inbox provider level are behind the curtain.

Whitelists for what? Having a pre-determined whitelist for domains allowed to send to your domain is not common. That would defeat the purpose of e-mail.

People, don't forget to properly set all these checks for DNS failover.

I saw companies got scammed, because they used default settings in Exchange Online.

And attacker just made the DNS "unavailable" for brief moment and all phishing emails passed. Because MS server responded with DNS "temp error" and pass all emails as not a spam. (detailed: received-spf: TempError (protection.outlook.com: error in processing during lookup of <phished domain>: DNS Timeout) and DKIM is checked on domain of sender's SMTP server, in this case attacker's server used for phishing )

Then I had the great experience with MS IT/security support, people there can't even understand how emails works, very funny and sad experience. I hope outsourcing works for them.

This is so cool! I would love to see this for other protocols actually, maybe SSL or something!

For TLS, there’s https://bytebybyte.dev/ and https://tls13.xargs.org/

(For reference, the original is here, https://tls13.xargs.org/)

https://www.ssllabs.com/ssltest/

SSLLabs is awesome, I use it regularly for work. So much information returned that it is like a school day reading through the results.

Dmarcian

dmarcian.com

https://internet.nl

The UK Government provide a similar check which is a lot more user friendly.

I use it for sending reports to upper management, who just want to see big green ticks next to our domain names:

https://emailsecuritycheck.service.ncsc.gov.uk/check

DMARC is and has always been...fine, save for the fact that most phishing / exploits are sent using cousin domains. Is a DMARC policy necessary and a great security measure? Sure. Is it a domain identity security game changer?... no way.

Is a DMARC policy necessary and a great security measure to stop most phishing / exploits being sent using your cousin domains ...

Bunch of discussion from 2022:

https://news.ycombinator.com/item?id=29869266

Appears to be operated by uriports.com, in case anyone wondered where their email was going..

Is that a bad thing or what ?

Dope af

Reply on news.ycombinator.com