Dear American Website Owner
jan.rychter.com
jan.rychter.com
(yeah, this works both ways)
I do have a state in my postal address, and if I have to enter my address as 47374 Richmond instead of Richmond IN 47374, it will probably go to Virginia first, and maybe they'll get it to me eventually.
Just saying. Internationalism is hard. If you want to go for the global market, it would behoove you to try to do it right.
That way you can enter your address any way you see fit. You really know best how to address mail so that it gets to you, why would I want to pretend I know better?
If you're doing business across state/province or national borders, getting that structured address data (and getting it right) can be awfully important.
you should remove the State field if the country isn't set to U.S.A. or at least make it optional
He doesn't say "Remove state completely".
Some of us don't even have postal codes. Those profane-looking ones we enter aren't real.
Love, Ireland.
The interface for a search engine is a single input field with dynamic suggestions and people were in an uproar when Bing didn't give suggestions that were localized to the user's country or language. That's to say, if one of the biggest software companies in the world can't get it right, then what hope does the average web developer have?
There should be a library for this. We have libraries that can handle the insanely capricious details of DST for all countries and subnational entities on the planet; we should have no problem providing one list of states and two regular expressions per nation.
(I know it's more or less impossible to reliably catch (e.g.) invalid UK postcodes, but we wouldn't have to. We'd mainly want to catch obvious typos without making things needlessly hard for customers overseas.)
And then it wouldn't even touch on all the issues.
You have to start somewhere — and let's face it, dealing with the two silly problems I mentioned would get us 80% of the way there.
The problem was with international users. The system was simple: it would call the exact phone number the user entered. This meant we really wanted to get it right the first time. The other problem was that we had a single call center setup out in New York. This meant international callers needed to be called with special country codes and what not.
Now, I feel it's safe to say if you ask someone in Europe to give their phone number, they are going to give their phone number like they would to any of their friends. They aren't going to enter in the country code, and they aren't going to know to prefix it with a special code so that an automated system from the US can call them.
The thing is, it's not fair for me to simply provide them with the requirement "Give us your phone number so we can call you from New York, USA." It really isn't professional.
So, I spent some time (lots of time) reading up and learning about international phone numbers, and coding together a system that went a long way toward fixing this problem. A user could enter in their phone number, and if they didn't enter in a country code, we'd be intelligent about it and add it for them. How did we know where they lived? Two sources: CC Info and the IP address. We could be intelligent and assume the two should mostly match up. Obviously, if the CC address was the US, and the IP was somewhere off in Asia, red flags beyond just the errors for phone numbers would popup (but, even then, you had to be careful!).
I spent a lot of time fine tuning the system, working hard to make sure that a phone number would get through however the user entered it, and we could call. We had a really good success rate with numbers outside the North American norm. Enough that the cases that did fail I couldn't even figure out manually.
I was a bit saddened that it was all for nothing when we eventually removed the 'feature.' and the requirement for a phone number. I understand the reasons, but from a problem solving point of view, it was a lot of fun.
To send a text message you have to use a full international phone number, so everyone uses them. You can't deliver an SMS message without a full number.
At least, that were my thoughts at the time.
http://hicks-wright.net/blog/stupid-american-website-owners/
http://www.jasonlotito.com/programming/what-i-know-about-des...
Anyways, you mention in your post:
"Making something a simple form that is fully internationalized and has useful validation is no where near as easy as they would like us to believe."
It's not easy, but it's not as time consuming and as difficult as you think it is. Getting the phone numbers right doesn't take years. A week is all you need from design to testing, and you are done.
Postal Codes are easy as well, considering their are numerous services that allow you to verify the Postal Code to the address. However, even taking something as simple as the common case, a user from the US entering a 6 digit code has probably made an error, and that solves a common problem.
House numbers and other addresses never presented a significant problem. I've seen more problems with not accepting different encodings then with the actual input.
And why is this important, even if you just want to serve an American audience? Because not every American lives in the US. Consider just the Armed Services, stationed all over the world.
No.
If you don't want to do it, that's fine. But don't complain that it's too much work or not worth it.
The question is whether or not that time is well spent, or if I would be better off using that time to improve whatever product/service the form is attached to. Especially when you're on a lean (i.e. startup) budget, the decision becomes pretty clear.
If you are providing a service that needs to be programmed. If you are short on funds. If you are only focusing on the US.
The problem isn't that. The problem is this:
When you are providing a service that relies on processing international orders... When you are no longer a startup and earning money... When you want to accept orders from the international community...
If you don't want to support the international market, don't. But if you do, do it right. I've seen it happen: "Launching in Canada!" and they still have a restriction of 5 characters for their Zip Code, or require all numbers.
It's silly. That's what the article is referring to. Wanting to accept international customers but doing it poorly. You'd be better off not offering the service and doing it right then offering it up poorly.
I hate street address validators that insist you live in a house or apartment with a number on a given street.
Because?
I live in flat m at street number n on Foo Street.
Here in Scotland -- where I live -- there are three ways to put this on an envelope, all recognized by the post office:
Flat m, n Foo Street
m/n Foo Street (this is the one in the Post Office postcode lookup database)
Flat xFy n Foo Street (where y is floor number and x is apartment number on that floor, e.g. 3F2, 12 High Street)
... So why do so many Javascript address validators throw out m/n or xFy format addresses for having an illegal character in the middle? Including British ones, that back onto the official Post Office lookup database?
Address formats aren't standardized internationally. They aren't even standardized within countries with a unified postal service.
Bad reasons given:
- Safeguard against users who can't be relied on to format their own address properly. Fix 1: Don't worry about that. Fix 2: If you really want this, leave the rest of us an option to use a freeform address instead.
- "Out webshop database has these fields." Fix: Change the database, then.
Or let the app first sort out the raw input, make sure it makes sense, then pass it on to the database.
I can see wanting to have fine-grained table fields. But I don't like seeing the schema drive the model or the UI.
A single textarea really is a strong idea (and like others here have been advocating it fr a few years with mixed results).
Seems like these are the sort of mundane tasks computers should be good at. Let humans be quirky and let computers sort it out. Prompt for confirmation as needed.
Much harder when the "database" belongs to a third party such as UPS or Paypal.
1. If you need to integrate with third party systems who require you to break up the address. In case you've never tried, parsing addresses is very hard -- if not impossible -- depending on how many formats you need to support. All it takes is one third party library that requires you to break them up to make your life miserable.
2. If you need to categorize your users by country, state, zip, etc. For example, if you need to handle different tax laws for different states or if you need to generate reports on how many users you have from country XYZ.
Honestly this seems like a problem that could be solved once and we all would benefit... And like any good problem, there is probably a profit to be made from it.
It's a terrible system really, and the only solution that I've been able to rely upon is having the user parse their own address and supply it to me. At least that way I don't have to spend hours looking through regex statements and edge case detection logic to figure out why a street address was parsed in a particular way.
Then you stick them altogether and print them on the envelope, thus covering chapter2 = report generating.
My suggestion is to give the option to enter generic information but don't let that become the default.
* allow the user to enter a free-form address, normalize it and then prompt the user for confirmation that the normalization was correct.
* Allow the user to edit fields on the confirmation page.
* Log any differences between the normalization and what the user changes for further refinement of the normalization process.
* Don't require fields that may not be relevant to all addresses that you plan to capture.
* Allow some sort of contact avenue on the address confirmation page so that they can complain when the interface doesn't allow them to properly enter their address. (If people hit a brickwall while trying to submit their address, they are more likely to just give up if they have to hunt down contact information. At the very least, if there is an easy way for them to complain, you get some sort of feedback even if you lose them as a customer.)
Is there some huge performance advantage in having the state field be an Id into a states table? Perhaps it would make updates easier if they ever rename Washington?
Why not normalize the name? There must be lots of Johns in your DB
It makes more sense to normalize it at the beginning in order to give the user a chance to approve that it was processed correctly. You could just do the normalization at the point when you need to interact with the 3rd-party services, but then it's hidden in the back-end so you have to be confident in your normalization processing.
You could roll out a solution where the confirmation page is just there to allow the user to approve that normalization works on their free-form address, but still store it as a chuck of text in the database. Then when you are confident enough in your normalization process, you could just remove the user confirmation part.
Sure, there are other ways to normalize an address from freetext input, but they often require using a third party system from FedEx or UPS.
Unfortunately, one big text field doesn't cut it when every other system you interact with requires separate fields.
It's not that American-run web sites don't want to support international formats, but when 95% of your business is from America, it just doesn't make business sense to deal with the complexity of international addresses for 5% of your revenues.
I'd love it if someone can point me to a library/book/tutorial of the appropriate way to handle this situation. NOTE: I haven't looked that recently, but did about a year ago.
Maybe someone should start a library for this problem, and then we all lay back and watch it grow to 50,000 LOC.
And then, in 1996, "regions" were replaced with "council areas:" http://en.wikipedia.org/wiki/Subdivisions_of_Scotland
that adds to the chip on our shoulder that suggests folk in Engerland don't really care what is going on in the rest of the UK
I suspect most in "Engerland" care as much about Scotland as the average Scot cares about English affairs (well, except for holding positions of power within our parliament).
The Royal Mail are happy for old curmudgeons and the like to use traditional counties in addresses, if they wish: http://www.abcounties.co.uk/bpa/bpacontents.htm
All that Royal Mail cares about is your house number and postcode, so you could probably even get away with telling websites that you live in Wibbleton in Foobarshire.
It's not just "close enough" - it's entirely correct. York is in the "ceremonial" or "geographical" county of North Yorkshire. And addresses are geographical references, after all: http://en.wikipedia.org/wiki/Ceremonial_counties_of_England (though, technically, you should probably just go with whatever the Royal Mail determines is valid)
The address "52 High Street, Northwich, Cheshire West and Chester" would reflect the correct local authority name but is incredibly confusing - are we in Northwich or Chester? "Cheshire" is still correct.
About the phone numbers, my favorite annoyance is when companies, who always are fond of making stupid mnemonics out of the letters corresponding to their phone number, don't let me enter my mnemonic for my phone number, which I picked from Google Voice specifically because it makes my name, but insists on numbers.
No. Zip codes may cross state lines.
Edit: Sorry, There are a few that cross state lines, when using the first two digits as the state code. However, each ZIP code is unique to a town; therefore, one could use it in the above scenario.
Example, please?
Redundancy. The postal service can usually deliver if you screw up one of city, state, and zip code. If you got rid of city/state then any error in zip code would result in failure to deliver.
There are plenty of zip codes in Missouri that have a dozen small towns that all have 1001 Broadway as valid addresses.
There are at least 5 zip codes (possibly more, there used to be a list on wiki) that cross state, county, and city boundaries. So it's entirely possible that without the zip+4, you'd have to have both the city and the state to correctly locate an address.
As suggested in a comment on a rant I wrote a few years ago, one strategy is to 'validate' against something reasonable, but instead of rejecting anything that fails, ask the user to confirm that they really do want something unexpected.
I'm not sure that 'reasonable' would be for an international phone number, but whatever you pick based on the country they choose, let them override your validation.
Also, wouldn't a phone number in effect be validated for most online credit card purchases? I thought credit card processor often match the phone number you provide to the phone number on record for your account to prevent fraud. Skip the javascript validation, and let the cc processor validate the phone number.
...but I have no idea what I'm talking about, please correct me if I'm wrong.
If you under-validate, you risk sales you can't complete. Mind, that's less dire when you have email addresses - but we know the problems with trying to validate those on forms.
It's much better to realise that your information will not always be 100% accurate no matter what you do.
I'm very much for allowing people to enter any format, but seeing how badly people mis-enter NANPA phone numbers, it is simply wrong to not acknowledge that you will see far more mistakes from an open input box.
Just let people enter numbers with all the spaces and the dashes the wish. After all, if a human will ever need a phone number, he will be able to read it in any form. If these characters are a problem for you to store, simply strip them. The routine to do this takes the same effort to write as the one to check the input.
And note that if you overzealously validate, you'll get garbage data anyway. What do you think I enter as a phone number if I can't enter my number at all?
My take is that we need to validate (people screw up and often), ask them to verify when we can't validate and proceed after verification.
Ideally you'd be able to parse it into a punctuation format based on local practice, but just about everyone punts on that.
That said, once a company decides to do business outside their country of origin, some serious rethinking of customer-facing forms (and fields, if applicable) is in order.
I could have written the same thing as a nerdy techie article, pointing out flaws in input validation. Do you think this would have gotten my point across? How many people would have read that article? Would it have made it to Hacker News?
Look: this is not a rant about Americans. It just so happens that US has developed most of the internet as we know it and it just so happens that most of e-commerce happens there. I'm sure any number of websites in other countries are guilty of similar sins, but you have to start somewhere.
The point is, we should start thinking more carefully about how we validate forms and store user data in that "global economy" everyone is bragging about.
Heaven knows, we never see nerdy, technical material on this site.
Are you trying to "clarify" or "justify"?
EDIT: Serious question. At the same time you're saying "this is not a rant about Americans", you're also saying but none of you would have paid attention if I hadn't made it one.
Maybe this is because Indians, Germans etc. are not socialized to see their way of running a nation as the only way, at least not in the way Americans are. Maybe not. In any case it's a nice demonstration of how collective cultural ignorance leads to annoyed visitors and, one might presume, lost sales.
Apparently not: http://news.ycombinator.com/item?id=1232278
Americans definitely don't have a monopoly on ignorance when it comes to other countries' address formats.
There is more than one service offering a geocoding API. Use any of them to do the hard work for you.
Your form should have one single field, named "Address". Let the user type whatever s/he wants. Get this input and pass it along to Google Maps, or Yahoo Maps, or WhateverMaps. Your geocoding service will return things in a way that is obviously more parseable.
I agree with this article, if you set a country other than USA, the state field should go away or better, don't preselect the US and add the field later.
As the author is in Poland, I'd argue that voivodeships are equivalent, in terms of being the highest level subdivision of the country. As a Brit, I'd tend to go with the county instead.
Edit on second thought: how about the vast majority of forms of this nature, which have a helpful dropdown list of the fifty valid states, and sometimes DC for the District of Columbia, but almost never PR for Puerto Rico, let alone Guam, the Virgin Islands, and the other areas of the world that the US Postal Service (and law) considers part of the United States?
I lived in Puerto Rico for years, and this is an utterly rampant problem for millions of actual American citizens living in the United States, let alone the rest of the world, all due to the fact that most Americans don't know jack about the world they inhabit.
"State" is really just equivalent to "second-level subdivision" (a bit like how "ZIP code" can mean "postal code" in general, as long as the box allows freeform input). Most countries don't call their second-level subdivisions "states" but counties, municipalities, council areas, and the like. Puerto Rico has municipalities and Japan has prefectures, for instance.
the other areas of the world that the US Postal Service (and law) considers part of the United States?
The places you mention are all unincorporated organized territories, meaning they are not considered to be "part of the United States proper." The word "proper" is somewhat important in this distinction, of course, and the USPS's definitions do nothing to help clear the waters on this one.. :-)
Moreover, Indiana has counties that correspond to Puerto Rican municipalities. Drawing this sort of specious parallel is entertaining, to be sure, but doesn't help people place orders.
It does if the order form requires this information. A little extraneous information on the postage label isn't going to cause significant delays for your mail as long as the underlying details are right (street address and ZIP code for most of the US - everything else is filler). Mail delivery systems are designed to deal with this (hence the introduction of the ZIP code in the first place) because people don't always address their mail to technical specifications.
Seriously, how many products have you shipped to Afghanistan or Albania but you make everyone scroll past these to get to the only country you actually deliver to (+ Canada if you're lucky)
You do not live in the United States of America. You think all American website owners are the same. And then you decide it might just be a good idea to lump the 95.4% who know what they're doing with those who don't. But you know, most American website owners are no more alike that most American <anything>.
The moment you whine about the few to the many in a blog, two things do happen:
- you irritate the people who know what they're doing
- you don't reach the people you should
If you don't like something about an American website, tell them, not the rest of us. That way, you're more likely to actually accomplish something. Which is, after all, the American way.
If this had been pitched as "sites do international addresses wrong" as opposed to "Americans do it wrong!", there'd probably be less defensiveness, less snideness, less "typical American can't read", etc.
I don't know where you're from, obviously, but having grown up in Indiana, I'm all too aware of the oblivious American nature, and I find the original post entirely justified in its tone. As far as I'm concerned, the defensiveness here is good ol' American exceptionalism at work, and anybody who wants to have global business had better get over that.
I second or third the notion that a nice library would be a Good Thing.
Correct, but it's only somewhat less parochial to assume that one's own country is uniquely oblivious to the rest of the world than to be oblivious to the rest of the world. After all, you were the first person in the thread to start pointing out how sites from other countries around the world fail in their own ways.
The simple truth is that without being whapped in the face by alternate standards for things like address formats, people assume everyone else does things their way. That a Pole runs into this much more quickly than an American is a natural consequence of living in a smaller and less populous country with many nearby neighbors.
"As far as I'm concerned, the defensiveness here is good ol' American exceptionalism at work"
Please! I've dealt with too many people from other countries to buy that. When "outsiders" start criticizing people who identify with any nationality based on their nationality, those people usually react with defensiveness and hostility. Part of dealing with people from other countries is learning how not to provoke such reactions.
And if you alienate them, does your support cost go up or down?
"The moment you put a "country" field in your form, two things should happen:" - that sentence should give you a clue.
It helps, when mocking a people, to convey such mockery through proper use of their language. ;)
More seriously, there's no need to douche up the conversation by throwing around slurs against nationalities.
EDIT: Apparently people disagree on that need. Ah, well.
- handling international times (say, Feb 1 2007 rather than 01/02/07 or 02/01/07)
- allowing people to enter their normal phone numbers and addresses
- knowing that United Kingdom is preferred for addressing and Great Britain is not.
Okay it’s probably time to come clean. In reality the problem you highlight is all a secret plot to frustrate you and hence make you less effective and unable to compete with us in the global economy. We were all told to do so through coded messages that are embedded into episodes of American Idol (I bet you thought all that stuff Paula Abdul said was just drunken babbling)
It’s all part of a far reaching plot by our government to subvert the rest of the world. Hence Google pulling out of China and the creation of Microsoft Windows (we secretly use a highly stable open solution and just use a keyboard shortcut to make it look like Windows when you walk into the room much like that fake looking spreadsheet that was embedded into the DOS version of Tetris)
Sorry for the inconvenience (though not really as pointed out above),
America