Email address validation: please stop
blog.sinjakli.co.uk
blog.sinjakli.co.uk
A plausible explanation given to me was that the web application worked just fine, but it sent the data to some COBOL-ish back end written long before the advent of web bookings, and the integration code mangled my email address when it stuffed it into some data field that was never intended to hold an email address.
This is why any real-world system needs end-to-end integration testing and not just unit testing :-)
People sometimes misread labels and enter their name on the line for their address. This would stop that.
One (very important) site properly validated my "+" email address on the front end (gave me no errors), but the backend failed and I never received the required confirmation email... all resulting in a customer service call. Arggg.
return Regex.replace(email, "([^\+])°[^@]°(@.°)$", "$1$2")
° should be * but HN eats it as markup.
P.S. Trivially A/B testator at high volumes if your CS infrastructure is capturing sufficient data.
The single most compelling reason to send people activation emails is -- I kid you not -- to remind them that they signed up for your website and how to get back to your website. A secondary consideration is not proving that they got their inbox right but proving that they didn't get someone else's inbox wrong.
I've been working on a large project that takes this approach for more than just email verification. We try to validate required information of multiple types (emails, accounts, etc.), and provide a "Move forward without validating" option after a third unsuccessful attempt. It's made for quite a better user experience overall, reducing the frustration of users sure they're submitting correct data while "the computer" thinks it's wrong, allowing them to complete the process nonetheless.
I always hated having to log in to my email after signing up, so I just create an account and login users without any upfront verification.
My email to the user says I will disable accounts that are not activated in 4 days, but its just a bluff :)
Or sometimes I get no e-mails at all. When I installed Rapportive (<3!) I found out that "I" had a profile on hi5, which I promptly deleted.
"Error: please enter a valid email address."
One: validating addresses to catch typos. A common example is typing a comma instead of a dot or typing just a username instead of a whole email address. Flagging these errors is a good thing.
Two: some developers believe that they can make people enter real email addresses by being very clever about only accepting strings that look like real email addresses. This is stupid, doesn't work, and often blocks legitimate addresses.
When I participate in some kind of online community, I want to chose if I receive emails from them at all. And if not, it should be my choice if I provide any email address at all.
I have a small site where you can participate anonymously or log in, and when you create an account it's your choice if you provide an email address at all. If not, and you lose your password, you're out of luck.
And telling your customer that they are out of luck may not be acceptable.
If you're looking for a startup idea how about a service that creates an anonymous ID (to me anyway) where the user provides that id to me, I send it to a service and get back a 'reputation' bit which says if you're a good guy or a bad guy (person what ever). And a way to report you've not been co-operating so that others can benefit.
Ebay reputation model but nominally anonymous. (at some point in some server somewhere there will be a way to link token a to token b but I'm totally ok if it can't be resolved into an actual person.)
I do often fall into the trap of trusting the framework's built-in email validation to be correct.
Apparently, this is the regex to match RFC2822 (?:[a-z0-9!#$%&'+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'+/=?^_`{|}~-]+)|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])")@(?:(?:[a-z0-9](?:[a-z0-9-][a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-][a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])
I was registering a general purpose domain name a couple of years back and asked a friend if he had any input on a good short name. He replied "nope", and being a Swede I registered nope.se.
There was a time I still forwarded all nope [at] nope [dot] se emails to my primary email, but as it turned out (not that unexpected), this was an address frequently used by Swedes to register "anonymously".
Anyway, it was an interesting/alternative way of keeping track of popularity of new communities etc. Clearly not all users expected that they had to verify their email addresses.
Naturally I reset my password, and the temporary password arrives in my account, but when I went to put in the new password, it puked on the email I was using ... then I remembered what I had done, but till today whenever I go to put in the new password with the correct login (name+fandango@gmail.com) ... I keep getting sent back to the reset password screen, over, and over and over.
It knows its me, because my credentials (Hi xxxx) are displayed in the top right hand corner, but it simply won't reset my password correctly.
Fandango support is well ... worse than useless.
Its all very maddening ... an account I've had for lord-knows-how-long containing my entire theater going experience, inaccessible. Thats what I get for trying to be clever.
Usually, in forms, you would ask for the real name and for the email address in separate fields and you wouldn't allow lists to be passed in.
I do agree though: The email address format is (even without the real name part) too complicated to really validate. In my applications, I check if there's an @-sign (I usually need a proper SMTP email address to deliver anyways), but leave the rest to the mail server on send-time, handling the bounces.
The author does say you shouldn't use it -- it's a crazy regular expression after all -- but it IS the RFC :D
This way, email validation is not even important anymore to avoid spam.
Using part of what was suggested in your post, if we do both, use recaptcha and send an email validation link before sending any emails, we avoid spam to our servers and to the people from us and we save everybody a little bit of time. :)
The next issue arises with email delivery. How do we then ensure that our validation emails don't get filed as spam? Because if the user never sees it, then it becomes a hassle for them and chances are, unless they really, really wanted access to our site, they're not going to spend time contact us to help them with validation so that they can login or otherwise...
Epic fail. Its this sort of approach that ends up resulting in cross site scripting bugs. Oh just take what ever the user typed in, and send it to the server they told me to send it to. Boom!
The perl code is perfectly reasonable for validating RFC compliant addresses.
Check for an @ sign and possibly a top level domain name (at leasr one dot) and be done with it.
Agree that anything beyond this may be a little overzealous- the real purpose and value of email validation is to help the form-filler discover typos/mistakes, to make sure you reach your willing customer. I can see though how some programmers would be tempted to distort this goal into the goal of having "a clean database." (Although worthy, but not the end goal)
So there may, theoretically, be some value in checking for the presence of exactly one @.
Requiring that the user have an email on file is not as necessary a requirement as a lot of people seem to think. It seems like half the time or more, they just want to spam it anyway.
This is probably an unacceptable solution for some sites, but I can definitely think of cases where the only reason you need email is for password recovery, and account loss isn't the end of the world.
By including super-special characters and whatever extra features GMail or whoever provides, you're just asking for it, sorry.
Especially if you're a coder yourself, you can already assume that even if it passes the initial validation, it probably won't be properly stored or escaped when the actual mail is sent, when you try to log in with your address later, etc.
Validating against the RFC is more, not less permissive than the position held by the grandparent comment.
And I am not encouraging people to be lazy and sloppy, I am just saying considering the 'real world', you are better off with an email-address that does not contain any too unusual characters (e.g. - _ . should be fine, as those are commonly used).
Seriously comments (nested even) in email addresses?
Are hyphens allowed? What about dots, underscores or numbers? Do any of those count as "super-special characters"? Just like "+", they are all permitted in standards-compliant email addresses, but I have no way of knowing whether they are permitted in addresses that you consider "legit".
Which characters can I have in my email address, that won't cause you to tell me it's my own fault when they get rejected?
I agree that you have no way of knowing what all the code along the way does, but you can at least hope that it behaves in something resembling a standards-compliant fashion. Otherwise, what's the point of email addresses at all?
My whole point is that there is a standard for an email address, outlined in a freely-available document. If an application claims to handle email, that claim implies conformance to that standard. Any deviation should be documented.
Your claim appears to be that there is some other definition of a "legit" email address, that you can absolutely 100% guarantee will be handled by absolutely every single email-handling application ever (without relying on hope).
Please answer these questions -
1) What, exactly, is the format of such a "legit" email address, according to your definition?
2) Where does this definition come from?
There's no such thing as a 100% guarantee when it comes to email (Interwebs 101) but it should be completely obvious to any sane person that you are getting much closer to those 100% if you don't use any "super-special characters" in your e-mail address as opposed to people "asking for it" by using an address like {^|~!}@gmail.com - which will obviously get you into some kind of trouble, sooner or later, whether it's Facebook's validation rejecting it or mail applications which can't handle it properly.
So of course, while there can't be 100%, from the perspective of an application that deals with e-mails, you'd want to get as close as possible. And like I said in an earlier comment, I wouldn't expect problems with characters such as - _ . but yeah, who knows?
And actually, there's no way anyone could ever have that obscure example address used above, as hey surprise, GMail won't allow you to register it (same for Hotmail). So, I am not sure what this means now:
GMail only allows alphanumeric characters and dots in mail addresses because...
a) ...their coders don't know the RFC and hardly anything about that whole e-mail thing in general, so they provided us with a heavily flawed product, according to your definition.
b) ...their coders have already been doing this whole development and e-mail thing for a week or two and it was obvious to them that "super-special characters" could lead (and have previously led) to trouble, so they're saving less 'techy' people from registering addresses that are basically "asking for it".
Ever since the first comment I left here, my whole point is that everyone who has built applications sending a lot of e-mails just knows that it's insane to assume that everybody else "does it right" and that combinations of special characters, escaping and UTF-8 often result in 'lots of fun'... not. That's far from being fully RFC-compliant but that's how it is, out there in the wild.
One last example: according to Wikipedia, Hotmail "refuses to send mail to any address containing any of the following standards-permissible characters: ! # $ % * / ? ^ ` { | } ~". (And being aware of this, you'd be "asking for it".)
2) Gmail does allow you to send mail to non-legit, but standards-compliant addresses like {^|~!}@example.com, because they know that their own mailbox name rules don't extend to other providers.
Regarding your point b) evidently they're not smart enough to grasp that addresses with + in them only work on the theoretical internet, and not the real one.
To reiterate - the mailbox names that a email provider will provide are subject to the naming conventions defined by that provider. Whether the convention is firstname.lastname; 5-7 characters only; alphabetic only; or 13375p34k transcriptions of characters from Lord of the Rings. None of that has anything to do with whether they are standards-compliant.
The test is whether they will send email to any RFC-compliant email address regardless of whether it conforms to their own mailbox naming convention. Gmail does.