It's Impossible to Validate an Email Address
elliot.land
elliot.land
If I gave you an email address that you think is invalid, rest assured I did it for a reason. I'm not an imbecile: I know how to type my address correctly (especially when you make me type it twice). For all the imbeciles who don't know how to type their address correctly, the phone system still works fine.
I may have given you my real email address with a plus-sign for a filter. Don't tell me it's invalid.
I may have given you a fake email address, because I know you're just going to spam me. If you tell me it's invalid, I'll either spend an extra few minutes cooking up a better fake email address, or I'll leave your site.
I wrote something about this a while ago. I'm no wordsmith but it fits here.
These systems aren't optimizing for someone like you trying to be clever. They're trying to handle the more common cases of someone typing a street address in the wrong field or forgetting the ".com" or putting in an HTTP address or some such.
They're trying to catch that, not every possible pedantically legal string you could throw at it. Maybe they could improve the validator, or maybe they'd rather be spending programmers' time on things with actual business value.
The sum total of imbeciles may well represent a bigger market than the sum total of people like you.
Now we can talk about HOW validation is implemented, and I think it would be completely fair to raise a warning: "Hey, did you mean to enter this?" instead of "sorry, nope." when non-trivial addresses are encountered.
https://github.com/mailcheck/mailcheck
I agree with the idea that it's impossible to validate. But, mailcheck takes the approach of seeing if the email is potentially wrong, then prompting the user with what it thinks they meant. It's usually right, but if not, it allows whatever the user wants.
For example, if your user types in "user@gmil.con", it will suggest "user@gmail.com".
There were a number of other checks (being close to yahoo.com or gmail.com or other common email hosts, containing surprising characters, etc) that would trigger warnings, but still allow the check to pass if the user assured us it was correct.
Technically, the RFC(5321) says MX records aren't required. You may be throwing out some small number of valid emails.
"If an empty list of MXs is returned, the address is treated as if it was associated with an implicit MX RR with a preference of 0"
Like DRM and a lot of other efforts to “control” things, aggressive validators invariably punish people who are just trying to do legitimate things. Don’t piss off your real customers.
I used to live in a town with a 12-letter name, and more than once a form decided that it knew the Universal Sensible Maximum Length of Town Names and wouldn’t let me type the last couple characters. And it usually doesn’t stop there, because once a site is incapable of storing things sensibly it invariably starts to have trouble matching things, giving errors that are just plain stupid (e.g. “this other thing doesn’t match what you entered”, well no shit...).
There is also too much thought put into what constitutes a person’s “name”. Generally, to work across all possible cultures, use a single, very long text field that can contain whatever the person decides to type. After accepting their input as-is, feel free to internally perform parsing logic to try to allow additional database queries but under no circumstances should your page make any assumptions.
The real “you should be fired as database administrator” mistake though is to store modified data without telling the user. This usually happens with passwords; I use a site for months and then one day accidentally hit Return too soon and my password works anyway, meaning they just CLIPPED whatever strong password I entered and stored whatever they felt like (usually 8 characters). NEVER do things like that without telling the user.
Can't the spammers simply strip the subaddress/label after '+' ?
Perhaps the reverse is more effective. Use a subaddress for all your mail and ignore that without a label coming in.
Generally, that's configurable; e.g. in postfix it's set by the optional |recipient_delimiter|. Suppose it's set to ‘-’, and you, John Public, sign up for SuspiciousService using the email address <john+public-suspserv@example.com>. Normally, the mail gets delivered to your 'john+public' mailbox. But if Mr Spammer strips everything from ‘+’, he sends to <john@example.com>. And your system knows that that is not only not a valid local mailbox, but also that it matches a plus-stripped address, and therefore consigns the message and its sender to the fiery pits of hell.
They don't.
It's extra effort for them for nearly zero marginal gain.
I wouldn't say it's nearly-zero gain; by applying a tiny sed expression they obtain a basically unblockable e-mail address.
It's easy to blacklist johndoe+amazon@gmail.com but very few people would be willing / able to blacklist their top-level johndoe@gmail.com. So the spam keeps coming.
Spammers are annoying but the progammers behind them are smart.
It has an MX record too. There is nothing wrong with this.
What is wrong with it is, amongst other things, the real-world possibility of colliding with internal hostnames.
tk has address 217.119.57.22
cf mail is handled by 0 mail.intnet.cf.
to has address 216.74.32.107
io mail is handled by 10 mailer2.io.
gg has address 87.117.196.80That's the easiest and most accurate way to do it by regex. Sure, some invalid addresses may still get accepted, but that is unavoidable. Even the most thorough validation[0] is going to accept nonexistent addresses.
[0] Except those that validate by sending a mail to it. Sending an email is the only way to be sure.
.+@.+
That's it. The name part can be anything; so don't validate it beyond having a character.
We don't have one-character domains yet, but there is no reason to exclude them. It is already possible to arrange your own top-level domain if your pockets are deep enough (.google), so don't be surprised if some megacorp starts using a top-level domain without any subdomains. Don't validate the domain part beyond having a character.
Absolutely this. The check for an '@' and something before/after it is for sanity, and anything beyond that would involve actually trying to use the address.
How about "one@two@three@four@example.com"?
# get email addresses
grep -EiEio '\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b'
# censor email addresses
sed -r 's/(<)[[:alpha:][:digit:]\._%\+-]+@[[:alpha:][:digit:]\.-]+\.[[:alpha:]]{2,4}(>)/\1--removed--\2/g'
I was neglecting this quote for a long time, until I started using regular expressions in real projects...
I realize they could save me some future potential grief, but I'm usually more concerned with the present actual grief they cause me.
I feel like I'm letting down the side.
The weirder it is the fewer web forms or software you'll successfully put it into.
I think we can just say no, functionally, you cannot put comments or additional @ symbols into your email address. It hasn't worked for long enough, people know you just aren't supposed to do it. I'd be surprised if you were allowed to create such a thing signing up for bing for example. You probably need to be the administrator of some chaotic UNIX server with full DNS, in order to force it to happen at this point.
Even Google Chrome's built in email field validation doesn't allow you to do it.
I shouldn't be expected to jump through the hoops necessary in order to allow "technically valid" email addresses that someone went out of their way to make, when I could more easily suggest they use a normal one.
I really hope you don't work on anything important if your stance is, "I shouldn't be expected to implement specifications correctly because it's easier to only implement part of it."
Why are we even having this discussion? Implement it correctly once, put it in a library and never worry about it again. You don't have to jump through any hoops, you're only making more work for yourself by implementing the standard incorrectly and then having to deal with customers that think that the ITEF standard is more valid than your personal definition of what an email address should be.
If your code is passed down the line and eventually hits someone who writes unit tests for actual valid email addresses then your name is going to come up on the git blame when it fails.
I stand by my position, failing to implement the spec correctly should be considered an error even when Google does it.
If you want a compromise, how about printing "we don't support email addresses with X" if you want to be picky for SPAM detection, simplicity, or something rather than "this email address isn't valid." Google does this for some valid-but-not-accepted addresses on their signup page but it's not sophisticated enough to catch everything.
If you want to send email to that address, you're probably going to want something that can suggest gmail as a replacement for gmial. You can also check that the domain exists and has a MX record. If you run your own mail server you can probably even check that the mailbox exists...
If you want emails to be unique, you'll need to apply per-site logic like gmails optional .'s and strip the + segments. That's important if you're combining multiple lists of emails, or importing an existing mailing list for a user.
The gist is the real world is complicated, but you can pretty easily set up something that handles 90% of it.
[0] https://mail.gnome.org/archives/evolution-list/2002-January/...
Perhaps it is nice to ask a question one step above the email verification: How much responsibility does//should the user have, how much responsibility does//should the designer have, in ensuring the user's email address as valid?
[0] https://documentation.mailgun.com/api-email-validation.html
Almost anything is valid within quotes, but quotes can not appear everywhere.