Free email address validation API for web forms
blog.mailgun.com
blog.mailgun.com
They say up front that email validation is hard, and yes there are tons of edge cases and obscure tricks and rules and probably there's no guarantee that even they managed to get it right with this service, but ultimately the customer either puts in the correct address or they don't. If they're going to make a typo then it's far more likely that it would be a legal typo, and if they're going to intentionally enter a false address then it's likely it'd be a simple asdf@asdf.com.
Edit: This was a bit of a knee-jerk reaction to what I at first saw as a redundant overcomplication, however as russjones points out below it has already proven its value in reducing bounce rates by a significant percentage.
So, it might not fit my own limited use cases, but it certainly can't be ruled out entirely. Best of luck to the Mailgun team and I hope people smarter than I am can put this service to good use.
The domain may exist, the mailbox may exist, it may match all the rules for that particular mail exchange, it can pass this validator with flying colors, but you still have to send that validation email because I could just as easily have entered your email as my own, and that makes this a moot step.
What this allows you to do is give feedback to users quicker. If a user signs up for your service and mistypes the domain name for their email address, they'll never hear anything from you. And if they don't know they made a typo, they'll just think that you have a flaky service. This lets you prevent that from happening.
Simply because we're an ESP and have a huge database of valid MX hosts and the most common misspellings for them that are NOT valid MX hosts. And we give you suggestions to implement autocompletion/autocorrection.
Besides, many of our customers reach out to our support asking for regexp suggestions. So we figured it would be cool to offer this to everyone for free, so we launched it.
And yes, your service is a pretty big step up from regular validation regexps. I didn't see the value in that before, but that's my own fault. I now see there's a lot of potential for a service like this, and I hope it's put to good use.
This doesn't even count domain validation, what with new TLDs coming online constantly, etc.
Given an arbitrary address, we will validate the address based on:
Syntax checks (RFC defined grammar)
DNS validation
Spell checks
Email Service Provider (ESP) specific local-part grammar (if available).
[0] http://documentation.mailgun.com/api-email-validation.html
You can't truly validate someone's email address because you don't know what their email address actually is. Your only two options are:
1. Be omniscient
2. Send a validation email
Only Blizzard tried to confirm the email address. The other two created the account, and now won't let me disown the accounts. I can go through the forgot password process and turn them off, but I'm not sure how kosher that is.
Something like this greatly reduces the number customer support emails saying "I didn't get an order confirmation email from you!" when they typed in joe@yahoo.cm.
EDIT: I was wrong. Interesting.
The big three providers all have OpenID or OAuth endpoints that return email addresses, so you also have the option of asking the domain directly.
(You can outsource that whole thing, too... * coughs something about Mozilla Persona *)
I wonder if they will also be able to tag domains/addresses as potential spam-bots.
The email spec allows for missing MX records, it should fallback to use the A record to deliver instead.
Hopefully they aren't invalidating domains that don't have an MX record.
Say, for example, that example.com has only a single MX and for whatever reason (server is down, network connectivity is broken, etc.) it is unreachable or unresponsive at the exact time you test. Would that constitute a "fatal error" in your validation process?
If there is a single MX RR but it "expands" to multiple hosts (multiple A RRs), do you test a second one if the first doesn't respond?
TIA!
We've been using this service at Mailgun during testing and we've reduced bounce rates by 5%. That might not seem like a signficant number, and it may not be for a personal blog, but it can be significant number for a larger ecommerce website.
For someone like us, a service that sends billions of emails, it's huge and we wanted to provide additional value for our customer to help them reduce their bounce rates and improve conversion.
Plus we don't correct typos on local-parts, just domains. So we won't correct Jooohn@gmail.com, but would suggest a correction for john@gmaill.com.
Maybe I'm an idiot but it took a while to see the 'user' actually entered "gmall" (I assumed it was correcting the spelling of Michael since the vowels in that name get mixed up so frequently).
Can you talk about why reducing bounce rate is important? For me, ensuring that the email address entered belongs to the user who entered it is the most important.
I think I've been a bit harsh and quick to judge, sorry about that. Obviously the service must have value, and it's my own failing not to have seen it.
I do wish you the best of luck with this new service, and I hope that many people are able to benefit from all the hard work you guys have put into it.
I can confirm that we do not store email addresses. The parser runs completely in memory and no address is persisted after the request is complete.
You should also stop using EC2 or linode, stop injecting the google analytics JS into your pages, stop taking payments via Stripe, and so on.
Not a useless service, to be sure; you have to make your customers aware that their email is submitted to a third party for validation though. Usually people don't care all that much (and have that same emails addresses listed in public profiles), but a relevant bit of fine print should be there.
[1] http://www.codinghorror.com/blog/2005/02/regex-use-vs-regex-...
Honestly, to you and me, it may seem trivial, but your customers have a really had time entering their email address. Our customers are end users, really end users, these are people for whom their email address means VERY little, but at least they remember that their email isn't at gmalr.con, if you remind them.
I'm not a Mailgun customer, but I would pay for this service (but not to much).
EDIT: That being said, making a potential customer wait for thorough verification, on top of "spamming" them before they've (technically) bought from you would be unwise; I can see how this would have merit.
Also, we need to find a name for "give me some personal data in return for a minimal value add service" offerings.
"No addresses submitted to the guardpost service are ever stored on any Rackspace servers. Nothing is persisted after the request is complete."
However, the API is using GET.
So unless you turned off logging, you are storing all tested emails in your web server logs which most places gzip up and archive indefinitely. In other words, I'd imagine these logs "persist" after the request is complete.
ADDED: As an end user, you may or may not be considered that these logs will contain:
1. Your email address
2. The place you are making an account
3. The time you created the account
And this information will be stored likely indefinitely (whatever the server log retention policy is). These logs also give mailgun and/or Rackspace a great resource for the membership rolls and adoption rate of any site adopting this service.
I have been using the same regex[1] for years it gets the job done adequately--at least, I've never received any complaints from users or clients.
I've used the regex on some relatively high profile sites--the kind where if someone was unable to signup with a valid email address we would've definitely heard complaints.
Someone actually built it. I forget the name, but yeah. User accounts as a service.
If this is true: http://i.snag.gy/RSwiG.jpg
Then explain why the e-mail is arriving: http://i.snag.gy/gWPn5.jpg
Sure this is an extreme edge-case, but this was my second test. Who knows what else it rejects. Actually, why do we validate email addresses anyway? Whynot just try and send that validation email that you're going to send anyway? And on top of that, why would I ever trust a free third party service to check all my user's addresses?
I appreciate your point that email addresses are allowed to include all kinds of wacky characters, but if the check simply triggers a warning to double check the address, and doesn't block registration, I don't see the harm in doing it - it reduces a lot of unnecessary problems for ordinary users signing up and making mistakes who don't have addresses like the one above.
Unfortunately you didn't read the blog like most other commentors posting about ridiculously unusable but otherwise valid addresses:
"""Our goal is not to make a perfect address validator that can validate every single address that has ever been created. Our goal is to build a realistic address validator for the types of addresses we see everyday."""
/@/
But most of the times it's just overkill and you shouldn't care.
It's free and you don't need mailgun or anything else.
You are welcome
We have a web form builder service. (Jotform) We serve 3 million forms and process hundreds of thousands of submissions daily. Every form pretty much has an email question and most people enable validation feature.
We first started with a very very smart and long email validation check that was going to be perfect. Every time users reported a case where our regular expression didn't envision we had to reduce it. During the years we had to change it so many times, I am pretty sure we have left with something like this: Does it have an @? Does it have a period? Great! You are validated!
Technically a period is not required, you can have an email address directly on a top-level domain. Though, of course, that is a rare enough use case that I'm sure anyone who has one has long since given up on expecting it to be validated correctly by most webforms.
They've failed in their own example. gmali.com has valid MX records and accepts mail. Just because they don't accept billions of email per month, why should this service block any mail?
When I enter fred@gmal.com, I get a 'Did you mean' message, with the red cross error icon.
I presume this means that it's detecting that gmali.com is a valid domain and can receive e-mail, but for most people it's not what they actually meant, whereas gmal.com is both a probable typo and a domain that cannot receive e-mail, and therefore invalid.
In other words, I think it's doing the right thing, in that it's detecting that fred@gmali.com is a valid address, but warning you that it's not the correct one. I think there is definitely a usability improvement though, as it's easy to miss the yellow warning icon, and assume that it's actually telling you that you can't use that address.
Again, due to the robustness principle, just because a host does not define MX records does not mean they can’t accept mail. Mail servers will often fall-back to A records to try and deliver mail. That’s why we go one step further than just a DNS query, we ping the Mail Exchanger to make sure that it actually exists."
Plenty of boxes don't respond to pings (icmp). Can I assume you're doing a tcp scan on mail ports?
Nope. We're an ESP ourselves. Mailgun has delivered (and accepted) many billions of emails since our launch a few years ago. We have a lot of data on 99.99% other ESPs in the world and the most common misspellings for them. We also have ESP-specific data on what kind of RFC-compliant email addresses they do not allow.
The longest part of validation service is DNS checks, but once the DNS server warms up and starts caching lookups the roundtrip time is going to be the longest part of the request.
We're still collecting reliable statistics and once we have them we can follow up with you.
https://github.com/Kicksend/mailcheck
At least you are not sending email addresses to a 3rd party.
1. Parse a chunk of text and salvage any email addresses from it that you can find. Use case: my users upload spreadhseets with email of their other team members, but email field would often contain more than one email (separated by slash or space or coma or god knows what), or other stuff like Skype account etc.
2. Actual validation service. I'd pay for it at standard mailgun rates, it would be easier for me than rolling my own as I do now.
Thanks again!
"email with spaces"@gmail.com
"very.unusual.@.unusual.com"@mail.yahoo.com
"very.(),:;<>[]\".VERY.\"very@\\ \"very\".unusual"@strange.example.com
n@ai
Another server-side alternative is isemail.info, which does validate all of the above.
You can't actually register "email with spaces"@gmail.com or "very.unusual.@.unusual.com"@mail.yahoo.com with Google or Yahoo.
Both addresses pass pure syntax checks but then the validator kills it when it notices that Google or Yahoo won't let you register addresses like that.
The first two are legal by rfc but not allowed by the individual providers. The third is not legit by rfc 2606 because example.com is a reserved domain.
If it was just a RFC validation then it should validate all of those (except maybe example.com). But they go beyond that: "Furthermore, the validator is ESP specific, so we can go way beyond valid syntax checks, bring in specific requirement for Gmail vs. Yahoo vs. Hotmail."
I'm not sure that anything other than basic checks adds a lot of value, and I'd worry about sending off users' email addresses to an API on a third party website before they've even agreed to terms - I don't think most users would be happy to find out that was happening.
It would be interesting to experiment with different levels of checks, and see which ones provide the most value though.
I'm sure there are special cases all over the place. It would be nice if Mailgun differentiated between 'this address is just malformed' and 'from what I know of [ISP], this address oughtn't exist'.
Yesterday a company invited 7 new users to their account using their email addresses. 3 of those addresses had typos in the domain names which this service would have caught. As it was, this error was only discovered when the service tried to send invitation emails to the new users and that's not a great UX.
Validation emails, particularly those with a confirmation link, are a horrible horrible solution. They interrupt the user's process flow, taking them away from their web browser, possible delaying the process, and you'll also get users searching through their emails and clicking that link just to access their account (yep, really).
I think I'm going to implement something like this Mailgun service plus sending a welcome email (with no confirmation link). If the welcome email bounces then I can handle that case but it should happen less often with the Mailgun live-validation.
I work for an ISP and we, of course, provide e-mail access via webmail. Right this moment, I can see dozens and dozens of e-mail messages queued up on our outbound relays that will never be delivered because the user typo'd the recipient's e-mail address.
An amazingly high number of messages bounced back to our users (the original senders) are due to typo's like this. Some people, despite not being "techies" can skim over a bounce message and realize they misspelled "live.com" and will resend. Others, well, they call support wanting to know why they suddenly can't e-mail Aunt Sally.
So if a user enters someone@yahoo.cm , I validate it first then send someotherperson@yahoo.cm to the the mailgun. Now your real user is protected.
Now you don't send them the real user name but get most of the benefits.
I was just implementing email and name validation checks for a project myself. Luckily email addresses can at least be validated by a confirmation email; it's the real name field I have no clue what to do with.
It's funny that after all these years, we still don't seem to have cracked these basic problems.
The ASCII guardpost in the API docs is also pure gold.
Note that a regular expression is a formal grammar too.
but i am concerned that they don't mention RFC 3696 when describing their grammar. it's all there (and implemented in - the now unsupported - lepl).
"foo@"@…, "me@google.com"@google.com
"me@example.com"@example.com
Complete fail. /s
Seriously tho, I like the design. It's refreshing to see someone avoiding another ugly perl-style regex.
A few more ideas: String Concatenation API, Number Addition API, String Length API...