Who uses Accept-Language header?
serpapi.com
serpapi.com
Not only languages change, but many other things like date formatting, mathematical formatting (using commas rather than dots breaks few things around) and currencies too! And it's a pain to fix as settings often collide and with many applications abusing microfrontends you can even end up in conflicts on the very same pages.
Google's the worst offender on all of its services and not just search, youtube, etc (even for work, their forced localization breaks Google Sheets in multiple ways), even my goddamn portoflio prices get converted in the local currency or using google flights shows me prices in the local currency forcing me to open more tabs to convert non stop.
And changing it to my own, on so many different services is such a pain.
But I know why this happens as a developer who advocated multiple times with my managers to not forcefully try to compute the language/region/market of users without giving them a simple button flag to change locale: product people don't listen and would rather have thousands of "ifs" that eventually will fail rather than listen to reason.
Hell, as a developer I have all my software and preferred language to english, even though I'm italian. And sometimes I use VPNs!
Even Amazon at times gives me some text in a language, and some other in others (completely unrelated like German) on the very same page.
Just keep it simple.
The most egregious instance of this is Google search with the news articles at the top. When I’m in Thailand, even if Google correctly figures out that I want English, it displays the news articles with the year in the Thai calendar! How many English speakers know that the current year in the Thai calendar is B.E. 2567‽ This is the correct configuration for approximately zero people in the world!
They're not tracking you for your benefit.
Though one does have to think that clickthrough rates and all similar metrics do go plunging when you serve a user a page in the wrong language.
I went through a lot of effort on one project that had something like 90+ country stores to make it work as expected. We used IP-based location to select a store if you hadn't already selected one. We used Accept-Language header if you hadn't selected a language. If we couldn't determine a location via IP address, we had a world map landing page you'd get redirected to so you could select a store where you'd be directed back to the product page for the correct store. Language display was best effort. All of the 'chrome' of the site could be displayed in most every language. Products had limited subsets of language. I don't recall how we resolved that disparity.
> I don't suddenly start speaking a different language if I look at a your site from a different country.
As someone that traveled a lot, I always hated this. Sorry, maybe I'm not fluent enough in German to read your site, but I really do want to order from your German store. Or maybe I really want to order from the US store even though I'm in Germany.
One other aspect of all of this that was *extra* fun was payments. Anti-fraud systems frequently get super suspicious if your location and the store location don't match.
I’m surprised by this. In my experience, Google are the one big organisation that I can count on to get this completely and utterly wrong. I’ve lost count of the number of times I’ve been stuck with a Google interface, popup, or some other thing that is in a language I don’t speak, just because I was visiting another country. I’ve specifically encountered it repeatedly with the very Google Sign-In widget they say works for them.
Sure, if it happens to be a Google website and you know the magic incantation of &hl=en then you can sometimes work around it, but that doesn’t help when it’s a Google login from another website or something of that nature.
For search i have had &hl=en added to the search parameters since forever.
^ I wonder how this works in places like Switzerland and Belgium that have more than one language. The notion that IP addresses can be mapped to languages needs to die.
Setting something that's very clearly custom (e.g. en,sv meaning English then Swedish) works for me, as I don't live in Sweden.
Accept-Language could be a pretty good indicator of a persons language skills, but it's just to far into the browsers settings to be something that a normal user would adjust. I can see why checking the domain is quicker and perhaps more inline with a persons intent, even if it is ignoring travelling.
We where trying to book a table at a restaurant, it's part of a fairly large chain. We where in Sweden, but the chain also operates in Denmark, so we know that they have a Danish version of their website and ordering app, you just can't use the Danish language when you are in Sweden. You can get English for some reason, but not the Danish version. That to me seem like a missed opportunity.
Most users are on mobile these days, and picking your preferred language is an unskippable step of setting up a new phone.
My hunch is that accept-language is far more accurate today than it used to be.
This does not address, or more precisely it even prevents the notion of:
> I might prefer some sites in Danish, but others I'd like to have en English.
, i.e. what you set in your OS or UA (browser) is always one preferred and optionally some fallback languages. I second that granular per-site control would be preferable for me either: some texts I prefer in my mother tongue (news, non-tech encyclopedic articles), the rest, namely technical documentation I prefer in English. (Also, I have very bad experiences with localised GUIs, where 1. familiarity with English "original" vastly helps when searching and 2. translations tend to range from bad to hilariously comical.)
But I must admit that most websites serving of content in multiple languages, support client-side pref pretty well, so the situation is not so terrible, in my experience.
Yet, the problematic of content negotiation and user <-> agent <-> author (provider) information passing is interesting and I'd say even under-engineered, and currently terribly suffer from security (anti-tracking) constraints.
(Lately, there was some development in "Client Hints": https://developer.mozilla.org/en-US/docs/Web/HTTP/Client_hin... )
EDIT: Best initial way to determine a user's language. MDN is right that overrides should be possible. https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_ne...
Accept-language is accurate, but just isn't easy change it which is required to handle all the possible situations and get the right answer. Still everything else is worse.
The idea you mentioned of browsers allowing the header to be set on a per-site basis would be particularly good.
Another very good solution would be for servers to assign weight to the languages they support, and to combine their weights with the Accept-Language weights to choose the best language in common.
But even as they're implemented and used now, Accept-Language headers are already a way better approach than geo-ip.
This problem is solved already by competent sites. On your first visit to a site, it uses Accept-Language to guess what language you want to see. If you use a language switcher menu to pick a different language, a cookie is used to save your choice. On your next visit to the site, the cookie is sent and that takes precedence over Accept-Language to send you the page in your preferred language. Easy.
If/when you decide to create a user account on the site (if the site even supports that) then your language preference can be stored with your user data so you would see it automatically after you log in even if you're on a different browser/device.
As with many other things, there are already best practices on how to do all this stuff. It's just a matter of getting web devs and their managers to be aware of them and/or care enough about providing a good experience for their users (unfortunately this has turned out to be a huge ask) to find and implement them.
Google knows I use/want English. It knows this because I'm logged-in and have set my preferences, but also because of that Accept-Language header my browser sends in every single request. Yet for years, perhaps because I am 20km from Germany, it has assumed I would always prefer German when doing social sign-in.
Google may occasionally glance at Accept-Language, but it does not respect it. It even demotes my own logged-in preferences below a guess based on my location.
When there are no other indicators, fine, guess my language from my location, but otherwise location != language. But I know for fact that my browser is sending out Accept-Language headers in a convenient format that will tell you all you need to know.
(if the original version is even available, they even find ways to offer some BBC shows, of all things, dubbed-only)
Then one day it went away. Complete mystery
If you’re using the Amazon app on iOS, you can open com.amazon.mobile.shopping://www.amazon.tld/?language=en_US, which does the same for the app.
It was also easily overridden by a very large language picker in our footer. This override choice is remembered in a long lived cookie.
People complained.
They didn't understand why they were getting whatever language and the picker was apparently too hard to find.
Now we just default everyone to English. They're still free to use the language picker. No one complains.
https://en-gb.foo.com >> set cookie and redirect to bare domain
...?lang=en-GB >> set cookie and redirect without the lang portion
<cookie>
<Accept-Language Header>
You can use just language (en) instead of language+locale (en-US) as desired.
edit: also, make sure you set your cache directive to include the cookie/header in question. You can also work the other way and always use the locale for subdomain.
Here in Switzerland, I am so tired of websites offering German localization by default. We don’t speak German in Geneva.
1. Check if there is a language cookie set previously 2. If no cookie, check if the Accept-Language is present and contains one of the supported locales 3. If no Accept-Language matches, use the fallback application language
Not that hard to implement, except perhaps parsing the Accept-Language preferences with their quality value can be cumbersome.
To answer the question, yes, normally you would have some kind of manual switcher that sets the cookie. Please don’t use flags for this, because flags indicate countries, not languages. Don’t make a Russian-speaking Ukrainian seek out the Russian flag, and don’t make an English-speaking Irish person seek out the Union Flag.
Yes, why make things easy. :-)
Controversial opinion: Flags are great as a visual aid, shorthand, look neat, and give a better UX for 90+% of users, even if it hurts some linguists' and nationalists' feelings.
(Bonus point for fr_CA with the CA-QC flag)
Just don't use flags, man. It's ridiculous that this is even still a discussion.
If you use flags I would still recommend putting the language name in the language next to it, for example the French flag next to 'Français'.
What flags to you suggest people use for Cantonese, Tamil, Arabic and Xhosa?
Why use the Union flag for English? If the language is en-US surely an American flag would be more accurate? You can then use the British flag for en-GB. However, if you offer Welsh you would use the Welsh flag (and presumably the Scottish one for Scottish Gaelic, although I cannot recall ever seeing that) so it would surely be more consistent to use the English flag for British? Even for or generic English?
Many web frameworks provide a tool for parsing these for you. If yours doesn't (or you're not using one) you may be able to find a package that does. Some will also do the matching for you, so you can pass it a list of languages you can support alongside the Accept-Language string and it will pick the best option that's found in both.
For example, here's the documentation for such a method in the excellent Nette PHP framework that I've been using where I can lately: https://doc.nette.org/en/http/request#toc-detectlanguage
But I've also used this stand-alone package: https://packagist.org/packages/willdurand/negotiation
Also, form an implementation standpoint, translation is a lot of work to get right and some companies may not do it for all languages so they may support the header but just not support the language choice you have. Yelp being a weird example where they apparently have the Japanese translation on another site.
Combine all that together and it's not usually worth the effort to do it unless you have specific legal requirements or high demand from users.
So some proxies (forward and reverse) will strip this header for different reasons.
The reverse proxies sometimes strip it to improve caching and/or because they were configured to have a allow-list of headers and this (being a less common one) was excluded.
The forward proxies will sometimes strip it for privacy reasons (it can be used in fingerprinting) or for the same reason (they have a list of allowed headers and this isn't one).
> Yelp doesn't actually use the header
So what made you think to add it and why did it work?
The idea is that the website chooses which language (between those two) to use.
But nowadays it’s a clusterfuck anyway. You often get WebView-based apps were half of the UI uses the OS language, half the browser preference, and the third half the service’s native/local language.
Which the rest of the article doesn't even attempt to examine.
E.g. I prefer to view some websites in English and some in my native. Generally I prefer all the websites that are not from my country domain to be in English. Setting up Accept-Language per website is annoying.
The most annoying part is when my local website sees "en" there, and decides to show me English version.
It is easier to just add e.g. `.jp` to domain, than to find out where browser sets up Accept-Language.