Show HN: Deep Email Validator | Hacker News Reader

Show HN: Deep Email Validator

github.com

51 pointsmfbx9da434 comments

Presence of an MX record should _not_ be used as a litmus test for a valid email domain.

RFC 5321 section 5 specifies that you lookup the MX and, if absent, use the A/AAAA record. I'm simplifying a bit, but that's the gist of it.

Using just the A record is more common than you might expect.

[1]: https://tools.ietf.org/html/rfc5321#section-5

The first line mentions:

> Validates email looks like an email i.e. contains an "@" and a "." to the right of it.

But what about emails coming from a hostname without a dot, or a TLD without a subdomain?

Even the HTML email field supports emails without a dot after the @ because it's technically valid.

You could do it for a TLD without a subdomain. The TLD "tt" does in fact have MX records. But it's a very rare case that a TLD does, and prohibited for any of the new TLDs.

For non-FQDN hostnames, it's kinda pointless, as you can only use that on your own domain. So if you were to do that, just make up your own pseudo-TLD like ".x" with an alternate DNS system. You could even have it point to the regular DNS system through CNAMEs.

Wow, indeed: https://www.nslookup.io/dns-records/tt

I understand it's technically correct but this is a solution no one wants. I recently spent time fixing this in application.

Sendmail also still supports uucp bangpaths as an email address: https://www.sendmail.org/~ca/email/doc8.12/cf/m4/uucp_mailer...

Yeah, looks like they're doing super-naive address validation:

https://github.com/mfbx9da4/deep-email-validator/blob/master...

OTOH, that code looks as if it'll find the vast majority of the problems more careful analysis could find, both simply and without bad consequences.

All that happens if that naïve code fails is that it does an MX lookup that's bound to fail, and then the code returns a bad error message ("couldn't find MX" instead of more properly "syntax error in domain part").

If an MX record isn't present you can fall back on the A-record.

(If, and only if, no MX record for the domain is present, treat the domain as if it had an MX record with the given domain as the target hostname and a preference value of 0)

If you want to be pedantic... not in the relevant case, no. In the relevant case (the domain is syntactically invalid but the naïve check didn't discover that) the response without MX RRs already tells you that there won't be an A (or AAAA).

foo@2001::01 is a valid email address, no? So is using the alternative dotless representation of the IPv4 addresses.

Per the RFC 5322 addr-spec grammar production, it’s actulaly not: colons aren’t permitted in regular domain name syntax; you need to use the domain literal form: foo@[2001::01]. Same deal with HTTP URLs, IPv6 localhost port 8080 is http://[::1]:8080/.

foo@[IPv6:2001::01], since it's an IPv6 address. Without the IPv6 prefix, it must be an IPv4 address.

Ah hah, https://tools.ietf.org/html/rfc5321#section-4.1.3 defines this. I had tried briefly to find something, but not very hard, and so incorrectly guessed it was like HTTP URLs. Thanks for the correction, this is good to know. With that, I also note that exotic IPv4 address formats (such as the 32-bit integer representation) are excluded in that grammar.

At least they aren't invalidating email addresses with the plus sign like some registration forms I encountered out there..

For most applications email validation is really not nesecarry. instead a service can rely on email verification. If the address is invalid, mails sent to it will never arrive at a destination.

It is good UX to let user immediately know if they enter a typo or an e-mail that is bound to fail. It is a real waste of time if user enters wrong e-mail and keeps refreshing inbox and spam for the following 5 minutes and still unsure afterwards whether they put wrong e-mail or the server just wasn't working correctly.

Also you will end up fucking your reputation by sending out emails to bad mailboxes all day long

Notably, this doesn't work for sending out purchase receipts.

One of the big problems we had at my old job. We were selling event tickets, the even we were selling for were sold in batches and usually sold out in minutes, so people would rush to purchase. We saw so many gnail.com addresses and people complained about not receiving their tickets

You might save a little bit of confusion by calling it an email "address" validator.

Hopefully this doesn't get used in some kind of registration flow in full since for example if you register with an aliased address, or you have a catchall address or email is just proxied the SMTP checks would fail.

Why would they fail?

It opens an SMTP connection and runs through the handshake. If this fails then, it will fail with a real delivery just the same.

It's still a bad test to do, but for other reasons.

What are the other reasons?

On signup for my app we validate like this, with MX and SMTP check - before sending a account verify message

SMTP checks need to be run from the actual mail server/cluster that will be sending emails, using the exact same EHLO greeting, etc. Otherwise you may get both false positives and negatives due to IP blocks and mail header filters on the receiving end. That's without getting into timeouts and remote-side enforced TLS.

There is exactly one right way to validate an email: send a message to it.

Email is a very old protocol with a lot of hacks and backwards compatibility. At this point, pretty much any string could be a valid email address. You can certainly run a bunch of tests and get a decent signal, but it's a lot of work, and if you reject an email address without testing it you're going to cause someone some trouble at some point. It's a lot easier and more accurate to just accept the string and try to send email to it.

It’s not a lot of work if you use a library. You will still need to do verification but saves you destroying your email domain reputation

Interesting, I wrote a similar tool some time ago, in Rust: [link redacted]

I'm using a similar technique to check deliverability, i.e. parsing SMTP responses.

I'm not a big fan of denying disposable email services. For that reason, I'll pass.

Lol

I bet this would fail on my catch all email since it checks if the mailbox exists, right?

Not 100% sure, but I expect it asks the server if mail to an address is deliverable. I recently was doing some testing directly talking SMTP to a server and recall a special check command. IIRC it said it check if the server would accept mail for an address. So I don’t think it would fail on a catch all.

Assuming this uses that technique.

How does it handle timeouts?

It has a hard coded timeout. The user can try again if some unrelated timeout error

Reply on news.ycombinator.com