Blink-Dev – Intent to Deprecate and Freeze: The User-Agent string
groups.google.com
groups.google.com
I am a bit worried, though, about how we will continue to count marketshare. If the intent were to remove the UA it would be worse, but it does seem like browser marketshare analytics based on user agents may be coming to an end.
(Disclosure: I work for Google on unrelated projects.)
I find it ridiculous that one part of Chrome is forcing me to add user agent sniffing while another part is trying to deprecate it. They need to get their act together: their decisions have massive consequences across the ecosystem and they need to be more responsible.
Perhaps you can correct me on why that's not so.
And I think the chrome team could have come up with a solution that didn't involve browser sniffing, such as sending a header in the request identifying that SameSite=Lax is the defualt, or that SameSite=None is accepted. But instead they recommended sniffing the user-agent string.
So this means that if you need to use SameSite=None you cannot without UA sniffing, which I think is what the parent was complaining about.
It's VERY useful to be able to use the UA string to do something like:
if (Chrome version X) then { do this crazy workaround because Chrome is broken }
No. People don't even necessarily update to the latest version. The "Samsung Browser" was notorious for seemingly always be some old fork of Chrome or Chromium that would never update.
I appreciate the idealism of "if your standard's compliant site doesn't work in Chrome 142 then it's Google's problem" but I can't just throw up my hands and let my site be broken.
The User Hints spec specifically says that user agents aren't required to provide them in all circumstances, and that they shouldn't be provided in some.
> User agents ought to exercise judgement before granting access to this information, and MAY impose restrictions above and beyond the secure transport and delegation requirements noted above.
The message from Google is clear: Whilst we understand you want to work around bugs, and this is the only way to do it, you really should be second-guessing whether you need to touch this API at all.
user agent prefers a "mobile" user experience.
UA Client Hints will provide: brand, major version, full version, platform brand and architecture, platform architecture, model and whether it should be treated as a mobile platform.
If you were using the UserAgent to guess if the browser was a particular version with a particular bug, now you don't have to parse something and probably get it wrong.
It seems you are using "Mozilla 5.0". Your browser is really out of date!
Please consider updating to rand("Googlebot/2.1", "Version/7", "XUL/97", "HTML/5").
> User agents ought to exercise judgement before granting access to this information, and MAY impose restrictions above and beyond the secure transport and delegation requirements noted above.
Hard problems are hard, but my experience, and the recommended approach by browser implementers, is that feature detection works better.
In 5 years you might look back and realize you no longer write browser-specific code any more.
We said the same thing when IE started losing market share a decade ago.
The problem is, everyone who makes a browser thinks they have the best ideas in the world, and then impliment them, but their users never blame the browser.
If someone is trying to access your site and it breaks, do you really think they're going to say, "dang, I should really tell Chrome to fix their incompatibility".
No, they will always assume the error lies with the site owner.
Are you old enough to remember the "best viewed with Netscape" badges that were everywhere in the 90s?
Unless in 5 years we have only Chrome left, then surely.
`if(affectedBrowserUA) { elem.style.transform = 'translate3d(0,0,0); elem.style.transform = ''; }`
Edit: I guess you can achieve the same thing through UA client hints. Just need to add a new JS file that reads the request headers and maps them into JS globals.
[1] https://developer.mozilla.org/en-US/docs/Learn/Tools_and_tes...
https://winbuzzer.com/2020/01/03/google-stadia-is-currently-...
I'm all for more privacy, but IIUC the main reason big sites like Google still do any UA sniffing at all is to minimize the latency / byte cost for the client in vending a web site that is expected to work.
Other questions include "should I tell the user to drag-and-drop?" "should I tell the user to tap?" etc.
Mobile devices have keyboards. If you mean "does this device have a physical keyboard", you don't need to know that. If you mean "does this device have modifier keys like ctrl and alt", someone might have hooked up a bluetooth or USB keyboard to their mobile device.
If you're on a large screen, show hints if you have room. (You might also show such hints in tooltips, as devices with a concept of "hover" correlate well with devices where keyboard shortcuts make sense.) If you're on a small screen, provide a way for people to get those hints anyway via a help option.
> Other questions include "should I tell the user to drag-and-drop?" "should I tell the user to tap?" etc.
19-year-old advice that remains useful: https://www.w3.org/QA/Tips/noClickHere
You can't tell from the user-agent if, for instance, someone might be using a screen-reader or other assistive technology, or otherwise using an interface you don't expect. Tell people what option/link to use, not how to activate it. Help them out with things like highlights, or tours, or sample videos.
I'll take the liberty to say yes, that is what they mean, and that is their point: there is no way to detect whether those keys are available, so the best we can do is guess.
And yes, I might actually need to know whether there is a physical keyboard or not. For example, do I auto-focus the input field? For devices with OSKs, the keyboard will pop up right away and obscure most of the site on mobile. For devices where the OSK won't pop up, I want the focus to land there immediately for usability.
A time entry input is a good example: on the PC you can accept valid [0123456789:.APMapm] characters as the user types, and have the HTML page do something sensible and show something sensible. On Android you either have an ugly time picker (they are all ugly!) or use hideous workarounds to try and detect keypresses.
Android and iOS have `inputmode=` but it is very restricted and there are wierd and sometimes incompatible differences between browsers or browser versions or OS versions. Custom data entry (e.g. time, date ranges, etc) is super ugly in HTML.
I have found many browser version and device specific bugs with virtual keyboards or PC keyboard entry that cannot be "feature detected" (browser version must be sniffed).
Ah, my favorite aspect of the w3c process: responding to the need for perfectly reasonable UX discoverability for users unaccustomed to using browsers with "You don't actually want to do that."
There's a reason so much of the web development world considers the documents they make to be "take with a grain of salt and do what works for your users."
The problem isn't that I don't know these things, it's that I'm beyond those things and trying to add subtle things to improve quality of life.
> Mobile devices have keyboards. If you mean "does this device have a physical keyboard", you don't need to know that. If you mean "does this device have modifier keys like ctrl and alt", someone might have hooked up a bluetooth or USB keyboard to their mobile device.
Yes, and this makes the experience on mobile devices worse!
On a laptop, if you open Discord, click on your friend's name, and start typing, you'll send your friend a message.
On an iPad with a keyboard, if you open Discord, click on (tap) your friend's name, and start typing, nothing will happen.
You say "there's no way to detect an iPad with a keyboard" like it's a good thing, but it's clearly a bad thing!
I go over a similar problem here: https://news.ycombinator.com/item?id=22047246
> If you're on a large screen, show hints if you have room. (You might also show such hints in tooltips, as devices with a concept of "hover" correlate well with devices where keyboard shortcuts make sense.) If you're on a small screen, provide a way for people to get those hints anyway via a help option.
This is clearly a worse experience for the user because it means hints are shown to users for whom they make no sense.
Like, yes, that's what I currently do, and it sucks.
> You can't tell from the user-agent if, for instance, someone might be using a screen-reader or other assistive technology, or otherwise using an interface you don't expect. Tell people what option/link to use, not how to activate it. Help them out with things like highlights, or tours, or sample videos.
There aren't ways to detect if someone is using a screen-reader, but there are plenty of W3C ARIA tags screen-readers understand, which, if used correctly, can improve quality of life for them. I get a lot of feedback from screen-reader users that they love my graphical online video game, because I went out of my way to improve quality-of-life for them.
And now, I want to improve quality-of-life for users on iPads, and you tell me "no! everything must be the same on everything!" No! I refuse to accept that!
GitHub has a hint in their comment boxes that you can attach files by drag/drop or pasting. That's the sort of thing I'm talking about. It's miles away from a button labeled "click here". There's no need for this smarmy "don't tell the user how to activate a feature". Some features are subtle and need hints!
I was not suggesting you didn't, but more than one person will read this conversation.
> This is clearly a worse experience for the user because it means hints are shown to users for whom they make no sense.
Or to users for whom you don't know that they do make sense. If you guess incorrectly that a device doesn't have a keyboard when it does, you'll prevent users from discovering your keyboard shortcuts.
I would also suggest that web standards need to improve, to make it easier to help figure out which hints to offer. For instance, I think it would be perfectly reasonable to offer a mechanism to detect "should I show hints for keyboard shortcuts", which desktop browsers would always enable, and mobile devices would enable if they have an attached keyboard or if the user is using another mechanism that might allow for shortcuts. And one day, perhaps we'll have a standard for keyboard shortcuts that will allow users to remap those shortcuts and allow the browser to provide "native" hints that those shortcuts exist.
I'm not trying to suggest that the current state is ideal, or that it can't be improved. I'm suggesting that user-agent-based detection is in some ways worse.
> no! everything must be the same on everything
That's not what I said. I'm suggesting that everything should be accessible on every platform. You might well make adaptations and hints based on screen size, or based on whether the user has previously typed keys directly at something other than a text box; just don't make assumptions the user can't work around or change if you've detected them incorrectly. I've dealt with sites that force the use of mobile if they catch the slightest whiff of "this might be a mobile device", and the mobile experience is far less capable. (That's leaving aside the anti-pattern of "are you sure you want to use a web browser instead of our app?".)
Personally, I would suggest detecting if someone "types at" the page without being in an input box, and offering to not only focus the input box but set a device-local preference to automatically do so in the future.
If you are using UA sniffing to determine if the device has a keyboard it will be wrong a lot of the time; you can’t tell if a Microsoft Surface has the keyboard attached or an iPad doesn't. you’d probably be no worse off trying to detect it by screen width, which is already a defacto way to detect mobile devices anyways, for better or worse.
(The right solution is probably a new kind of API or media query.)
The specific feature this came up recently is whether or not a chat app should autofocus the input box. Ideally, the answer is "yes" if you have a keyboard (lets you start sending messages faster) and "no" if you don't (the on-screen keyboard would cover up the conversation and be distracting).
There's, of course, no way to detect an iPad keyboard, so currently iPads miss out on autofocus.
Discord, incidentally, uses an alternate solution: "don't autofocus the textbox, but instead detect keypresses outside the textbox and manually insert the corresponding letters in the box". This, of course, completely fails for various non-Latin keyboards, such as Japanese: if you type "chi", it will insert 「cひ」 instead of 「ち」.
I always use the term "tap". MacBook / Magic trackpads literally have an option called "tap to click".
> should I tell the user to drag-and-drop
Both mobile and desktop support drag and drop, so not sure why you'd want to disable it for either.
Why are you doing all that? Just offer the desktop version or Mobil version and be done with it. You dont need to figure out dev_caps in such detail.
There's no reason to make multiple sites each with their own flaws, when you can just make one site that can do anything.
That is hopefully the end goal. It's not worse, it's an ideal that goes hand in hand with reducing cookie usage, removing flash, mandating SSL, etc. The UA was an attempt to see if the ecosystem could cooperate to make apps more actively intelligent. This is a failed experiment. Security teaches us to reduce attack surface and we should not be using half-measures like a header-string to trigger naive/implicit client behavior. Behaviors should be explicit for safety.
The world's biggest advertisement provider's browser arm isn't going to turn off a source of fingerprinting data unless they simply don't need it anymore. It's far more likely that Google is pulling the ladder up behind them: they can identify users without that fingerprinting data, so it's beneficial to deny it to their competitors.
This can be simultaneously a great technical decision and a decision that entrenches Google.
I think there's an implicit assumption a lot of people (not you) are making: that there's no cost in collecting data. There is: often you change things by observing them, and that's not always a good thing.
There are big costs to UA strings.
How so? There is still an API to ask for more details, so you can continue to do exactly the same thing as before at the cost of a few additional API calls
What am I not seeing? To me this feels like just adding more complexity to still end up at the same spot we are now.
> The above abuse makes it desirable to freeze the UA string and replace it with a better mechanism.
UA sniffing should have died out a long time ago. It's frustrating that it's 2020 and I'm still having my browsing experience broken for no other reason than the site doesn't like my UA.
Safari has SameSite bugs and it's infuriating that after years they don't even update the bug reports, let alone fix those.
[1] I believe FF includes in the download length the time a request spends sitting in the request queue, but I can't exactly remember.
https://www.chromium.org/updates/same-site/incompatible-clie...
What I think would be best is two things: A feature advertising string (preferably with some versioning in itself), and a new User Agent string that is just that: The user agent. As in, "This is Firefox 72.2.813". Maybe the OS in there too. I think a non-trivial part of the problem with the UA today is "Hey... I'm Firefox. Also, I'm Mozilla. Also, I'm edge 3, and Internet Explorer 6, and today I think I'm also Chrome. Also, I may be a monkey with a banana plugged in to an ethernet cable." Part of the problem is that people abuse the user agent, but part of the problem is that the user agent string is also a pack of outright, outrageous, self-contradictory lies. A truthful field may still have significant utility.
One of the things I've learned over the years in software engineering is that you should do you best to make sure that your system doesn't contain lies, or, failing that, treat them like any other issues that you should contain at the edges rather than letting it run riot through your system. For instance, if you decide to let customers have 10GB of bandwidth free before you start charging them, you should never accomplish that goal by tweaking the bandwidth counting system to report 7.5GB of usage as 0GB. That is a lie. Tell the truth about the usage and let the billing system apply the discount, which is itself a truth as well. You always pay for lies in the system in the long run. It is a very common pattern in coding as well where some function author figures out how to assemble the correct lie to some other function to make it do what the author things they want, but in the long run you're better off telling the truth and fixing the code to work with the truth. Otherwise, well, you really do end up with a tangled web of code. Which is... exactly what we've gotten with UA-detection-based code. We won't necessarily get the same mess if there's a new UA that isn't a lie.
I'm not claiming this will lead to utopia. Minority forks of popular browsers will fail to pick up the necessary ad-hoc bug fixes this way, for instance. My claim is more like in the long run, the best thing for everybody is just to have a user agent that tells the truth, even if on occasion in the short term you experience occasional problems.
(One way to at least partially achieve this is to standardize the new truthful agent string to something like (\w+) ((\d+.)+\d) (\w+), and specify it as "non-conforming true user agent strings MUST be entirely rejected and treated as being absent", so anyone who tries to be Firefox and Chrome and also a monkey ends up being nobody in particular.)
Also I think you're missing why the user agent lies now. It wasn't done on a lark. It's because it's been abused so much by web devs. Both by sloppy regular expressions and deliberately. For example, even big names like Microsoft and Google have at times used it to unnecessarily deliver poorer browsing experiences to people not using their browser.
Sure browsers could have stuck to their guns and been "honest" about their UA. But honesty is no comfort to users of the browser when they find websites break for no reason. The average user is more likely to blame the browser than the website.
The parent of my post is correct; in practice we're still going to need the occasional ability to shim in browser-specific fixes, because even if the browsers do their best, they're going to inadvertently lie in the future and claim to support WebVR1.0 in Firefox 92, but, whoops, actually it crashes the entire browser if you try to do anything serious in it. Or, whoops, Firefox 92 does do a pretty decent job of WebVR1.0 but I need some attribute they overlooked. Or any number of similar little fixups. We know from experience from the field in the real world that we're talking about crashing bugs here at times; this is real thing that has happened. Whatever proposal gets implemented should deal with this case too.
If we standardized on the format like I suggested at the end of my post, it would go a long ways towards preventing future browsers from mucking up the field. If you just get "$BROWSER $VERSION $OS" in a rigid specification, and if the major browsers are sure to conform to that, and the major frameworks enforce it, it'll be enough to prevent it from becoming a problem in the future. It won't stop Joe Bob's Bait Shack & Cloud Services from giving their client a custom browser and/or server that abuses it, but there's no stopping them from doing things like that no matter what you do, so shrug.
Culturally, you should prefer to use feature detection. Most developers would never need to use anything else. But when Amazon makes its new whizbang WebVR1.0 front-end in 2024, they may need the ability to blacklist a particular browser. Lacking that ability may actually prevent them from being able to ship, if shipping will result in some non-trivial fraction of the browsers claiming "web-vr/1.0.0" will in fact crash, and they have nothing they can do about it.
Besides... they will find a way to blacklist the browser. Honestly "prevent anyone from ever knowing what version of the browser is accessing your site" is not something you can accomplish. If you don't give them some type of user agent in the header, it doesn't mean the Amazon engineers are just going to throw their hands up and fail to ship. They will do something even more inadvisable than user agent sniffing, because you "cleverly" backed them into a corner. If necessary, they will examine the order of headers, details of the TLS negotiation, all sorts of things. See "server fingerprinting" in the security area. You can't really stop it. Might as well just give it to them as a header. But this time, a clean, specified, strict one based on decades of experience, instead of the bashed-together mess that is User-Agent.
Or, to put it really shortly, the fact that a bashed-together User-Agent header has been a disaster is not sufficient proof that the entire idea of sending a User Agent is fundamentally flawed. You can't separate from the current facts whether the problem is that User Agent sniffing is always 100% guarantee totally black&white no shades of grey mega-bad, or if it's the bashed-together nature of the field that is the problem.
Feature detection is well and good, and should be the first line of attack; but sometimes you need to account for things like certain versions of Safari having WebRTC "support" that's actually completely broken, certain versions of Chrome crashing when certain WebAssembly features are used, and Firefox-specific CSS bugs. (All real examples I've run into.)
It may not be the worst thing in the world if UA sniffing is broken for all existing web properties though, since anything not well maintained enough to migrate to a new API is probably either working off of outdated information or abusing UA sniffing where feature detection would have been more appropriate anyway.
That being said, requiring a server to use User Agent Client Hints is stupid. What are client-side libraries like webrtc-adapter (https://github.com/webrtcHacks/adapter/issues/1017) supposed to do? I don't see any goals listed that wouldn't be addressed equally well while providing an equivalent JS API.
This times a million.
Browsers have tons of bugs/inconsistencies where knowing the browser+version is the only way to work around them. Even when browsers fix them, you need to keep the workarounds for people still on the older versions.
When I wrote a library that used the audio API, I couldn't believe the number of browser+version-specific workarounds I had to code for.
And it's not just "bugs" but things where the spec is unclear. I don't remember the exact specifics, but it was a lot of stuff like "if I send a pause command, will a stop event fire afterwards or a pause event or no event at all?" There were a lot of situations where there was zero overlap between browsers. Literally no choice but to use user agent sniffing.
We had to recur to identifying IE11 to write workarounds so that it works there...
I suggest UA string be a bitmask of features. Then feature detection should stop being broken
Extra bits could be used for js-on/js-off, and is-bot/is-human
--
Ah I see they're kind of doing the bitmask, but keeping a round-trip, and making things complicated (though I realize latest http standards can probably remove those round-trips in the average case)
I'd still suggest the bitmask for non-sensitive information, and have everything else simply js-tested as it currently is
Maybe is-user-blind might be a nice bit too, since canvas based websites could switch to the dom, or whatever
If we have those bits, then the user can make a set of choices once, for every site, and we get rid of cookie pop-ups
-- Websites could still ask if they want/need to do something that violates those choices
Like what?
> every website ask for permission in order to enable basic functionality
I don't believe that purely functional cookies require GDPR permission - that's covered by "provide services to the user". It's the ones which are functionality to third parties not the user which are the problem.
Ah, I didn't realize that. Well, that does sound much more reasonable.
> Necessary cookies
> Necessary cookies enable core functionality such as security, network management, and accessibility. You may disable these by changing your browser settings, but this may affect how the website functions.
> Analytics cookies [toggle On/Off]
> We'd like to set Google Analytics cookies to help us to improve our website by collecting and reporting information on how you use it. The cookies collect information in a way that does not directly identify anyone. For more information on how these cookies work, please see our 'Cookies page'.
The implication is that a consent dialog would not be required if they weren't using Google Analytics or any other third-party.
I typically read those warnings as reminders that I should open the site in a FF container.
That one was tried with the DNT bit - of course users ended up en masse setting it to "do not track" by default. Sites won't accept that.
This is exactly what you aren't supposed to do
Either send the wasm optimistically & fallback to js on error, or send it reactively with js+branch
A http header bitmask set by javascript:testWASM(), would be equivalent to what you suggest...
But avoid the js, and making a second trip to the server
From the implementation side, I would have preferred to see an OPTIONS request style solution similar to CORS to allow complying UAs to detect what if any UA information will be required.
Could also be evil and long-term: user agents won't matter anymore once everything is auto-updated Chrome, right?
As such, I think if detection comes in some other form, it might be harder to trick some sites into working properly.
The default level of information exposed drops to just the browser name and major version, which is only sent to sites on HTTPS and with JavaScript enabled.
Additional hints are only sent on subsequent requests by the browser if the site sends the matching header in its initial response and the browser chooses to send a value. The current set of proposed hints define the same amount of information exposed Chrome's User-Agent string.
I think the only winning move is not to play: Freeze the User-Agent and do not provide a replacement. Or at the very least, make the replacement based on actual feature bits, not version numbers and device models.
Just as with adblockers users will be nagged about "please turn that dial to 11". On average nothing will improve except for users who are able enough to get around those shenanigans even now.
Just as one example, active-permissions that can be revoked after being granted aren't perfect, but are a big step up over manifests, because they're more work to exploit and often allow users to retroactively change permissions after an app checks if they're allowed.
Not to pick on the Privacy Budget specifically, but I worry that proposals like this don't really get that larger principle yet -- that it's still something we haven't quite internalized in the privacy community. If a site exceeds the privacy budget, it shouldn't get told. It should just get misinformation.
It's like autoplay permissions. Autoplay permissions on web audio are awful, because you can just keep trying things until you get around the restriction. What would be better is to auto-mute the tab, because that would be completely invisible to code running on the page.
The only privacy-conscious way would be no feature-detection at all or a very coarse-grained approach like "I support HTML2021".
You can't lie that you do support something, but you can lie in the opposite direction. And for sites that legitimately need that feature to function, you don't get much benefit -- if a site genuinely needs Chrome's Native File access, saying that you don't have it just means the site won't work.
But there's a grey area past that, which is that sites that don't need a feature, but are just using it to fingerprint, can have that feature broken without repercussion. If a news site requests Native File Access, and I say "I don't support that", then whatever.
This puts engineers in an interesting position. You can't just break your site whenever the full range of features that every browser supports aren't available, because:
A) You want to support a wide range of browsers, and if your news site doesn't work with multiple browsers you're just losing potential market.
B) A fingerprinting metric that just rejects every browser that doesn't support everything is not an effective fingerprinter. At that point, we basically have the corse-grained approach you're talking about.
The problem with this approach is that when a site requests capabilities, you need some way to figure out whether or not they're actually required, and whether or not you can lie about them. Permission prompts are... well, there are probably UXs that work, but most of them are also probably too annoying to use. In practice, I suspect that manually tagging sites is not an insurmountable task -- adblockers already kind of do this today.
One thing to push for with Client Hints is that it really, really needs to be an API that's interceptable and controllable by web extensions.
The same thing is true of fonts today -- if you lie and say you already have a font that you don't, congratulations, your text rendering is broken. But you can still lie about not having fonts, and you can still standardize your installed fonts to a smaller subset to make your browser less unique.
I often hear proposals that the web should just be for static documents, and I'm fine with that, but very rarely are those proposals followed up with alternative ways for ordinary people to run untrusted code. The assumption seems to be that if the web didn't exist, users would instead be responsibly vetting every binary on their computer, rather than downloading them en-mass from dozens of sources. And just looking at the smartphone app market, I don't think that assumption is true.
Again, not to say that a better alternative platform couldn't exist, but who's working on it? The native desktop platforms I see almost all do a worse job than the web at protecting against fingerprinting. It's almost universally better for privacy to use Facebook in a browser instead of downloading their native phone app.
What we need are standardized protocols, strict customer protection laws and trustworthy software repositories so users can get software they can trust without having to vet it.
> Again, not to say that a better alternative platform couldn't exist, but who's working on it?
~Nobody is working on such a platform because the app web exists. Remove it and there is a lot more incentive to create a replacement. Regression to the mean alone practically guarantees that it will be superior.
I don't believe this is a practical philosophy given the way that ordinary people use both the web and apps today. If you can convince me that you have a plan to make everyone en-mass stop installing the Facebook App, I might be persuaded to change my mind. But I regard the advice, "don't run untrusted code" to be a bit like saying, "stick to abstinence to avoid pregnancy". The advice isn't technically wrong, we just have good evidence that it doesn't work for society in general.
I also think that "untrusted" is being used as a really broad catch-all here. Trust isn't binary. I trust a calculator app to give me the correct answer to a multiplication problem, I don't trust it to store my banking information.
What sandboxes do are they allow us to set up boundaries for apps that we partially trust. Of course, sandboxes don't remove the need for consumers to be taught not to blindly trust everything. But they're not designed to solve that problem, just to make it easier to manage. There is no way to remove the need to educate consumers; trust is too broad of a topic to divide every app into a single "trusted" or "untrusted" bucket. So what sandboxes do is embrace that grey area rather than ignore it.
In the physical world, if I'm securing a bank or an organization, there are going to be people I distrust so much that they can't even enter the premises. There are going to be people I trust enough to walk around while supervised. There are going to be people I trust enough to be unsupervised, but not enough to give them keys to my vault or server room. And finally there will be a minuscule number of people I trust to have full access to everything.
Imagine if instead, our policy was, "you shouldn't let untrustworthy people into your building in the first place, so security measures past that point are useless." Would we be able to build a database of 'trustworthy' people who could have access to the vaults of every bank they walked into?
When we get rid of sandboxes, we're still using a permissions system -- all we've done is made that permissions system less granular and harder to moderate, because we've removed our ability to say nuanced things like, "I trust Facebook to connect to the Internet, but not to read my contacts."
And how much of a hurdle is that, exactly?
That's an embedded chromium browser and according to [1] you should really be switching to oauth for login, not faking a UA string.
[1] https://security.googleblog.com/2019/04/better-protection-ag...
"However, one form of phishing, known as “man in the middle” (MITM), is hard to detect when an embedded browser framework (e.g., Chromium Embedded Framework - CEF) or another automation platform is being used for authentication. MITM intercepts the communications between a user and Google in real-time to gather the user’s credentials (including the second factor in some cases) and sign in."
In other words, the OAuth sign in is intended for applications which want to access the Google APIs in some way.
qutebrowser (as an application) doesn't need access to your Google Account in any way. It doesn't care about your Google authentication data, and it doesn't use Google APIs.
Safari could say "Sorry - <div> tags aren't working in the current release", and all big websites would switch to other html tags in a matter of weeks.
Safari has been very wary of this break-neck speed of how Chrome imposes Google's view of standards onto the world. WebKit team has even walked away from proposals when concerns were not addressed (sadly, I can't find the link in GitHub issues right now. Edit: [1]).
And yes, lack of manpower is another pressing concern.
[1] https://github.com/WICG/construct-stylesheets/issues/45#issu...
"Now I consider this issue as an absolute show stopper. I don't think we want to ever implement this feature in WebKit unless this issue is resolved."
UA is the only thing I get in server logs to discover how much attention I should pay to support particular browsers.
It depends on your application but the only real alternative is to use a js analytics library to collect the statistics. The good news is that they're more accurate, can give you more information, and there's plenty of self hosted options like Matomo.
But it is yet another thing the client as to load, and adblockers will likely skew your metrics.
In my app Dictanote (https://dictanote.co) - which uses Chrome's speech-to-text API, I have no way to distinguish Brave/Vivaldi and user doesn't understand why its not working :/
[1] https://developers.google.com/web/updates/2014/01/Web-apps-t...
This specific API only works in Google Chrome unfortunately. So we need to stop people from trying in another browser and getting frustrated as to why its not working.
To add to OP's point, this thing in Safari also comes to mind as an example of something that isn't easy to detect and address outside of UA detection: https://github.com/vitr/safari-cookie-in-iframe
Deprecating UA Strings and moving towards UA Client Hints seems like a move in the right direction though.
Assuming these iframes will never work means degrading the experience for all users, when only users on Safari are actually affected. Detecting the UA and branching based on that is a much more pragmatic solution.
- It's not likely to work anytime soon
- "The property doesn't exist or throws an exception or does something else that Chrom{e,ium} doesn't do" provides the necessary fingerprinting
- Adding the feature back later can just be chalked up to "growing pains", it doesn't really set especially unique precedent
To determine if a browser has the text to speech API just check if the webkitSpeechRecognition object exists (if that’s the one you’re using). It will exist in Chrome and will not exist in other browsers that lack the feature.
An incorrect assumption stated as fact with such great confidence. Good job.
Even on the face of it, any engineer would know this can't be true.
The kind of testing you are complaining you can't do is exactly why user agent is broken, what if you test for Chrome and then Brave/Vivaldi start supporting the speech-to-text API and your broken website still says "sorry, you need Chrome" for no reason?
"Somethings wrong! You might be using Chrome and having connectivity issues, or you might be using Brave who have a dispute with Google about usage of their speech recognition API. In the former case, get better WiFi, and in the latter case there's nothing you can do, switch browsers."
While browser and JavaScript engines are likely to continue being detectable due to behavior and performance for the foreseeable future, it's probably possible to make the OS, browser frontend and exact engine version undetectable.
Edit: OK, this isn't progressive enhancement. But it's still a major problem you have to sniff the User-Agent for. I don't want to sniff UAs either but when Chrome is wont to change how fundamental parts of the web work like cookies it's sometimes necessary. I know UA hints will still allow you to do this, but requiring a second request is going to make things like redirect pages difficult to implement.
The "main advantages" section for client hints all seem to apply there.
UA hints solve nothing, even after reading the spec
Why not drop just that and leave nothing but the exact HTML and JavaScript engine names and versions like "Blink a.b, V8 x.y"?
And what about robots? Will the Googlebot give the UA up as well?
Using a completely new string format in the same field (or removing it entirely) breaks a lot of sites that'll never be fixed.
Freezing it prevents this. And if we're freezing and creating a new system then why not go for something queryable without all the baggage?
I expect no UA header to be even less usable now that sites are more paranoid and app-ified, so instead I use a random one. That still confuses some sites...
The arguments we're seeing are around what the replacement should be (or if there should be a replacement at all). But either of those scenarios are still better than keeping user agents as they are.
Getting rid of user agent strings is great, as long as we get a better way to determine browser capabilities that doesn't require some kind of special feature checking library...
Why does edge say "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362"? Because people kept abusing it for feature detection.
It's exactly the same as the User-Agent header we had, but worse.
UA was used for tracking? With this new standard, just ask the user agent to include all details in its Accept-CH header.
UA was used for feature detection? People will use this new standard to do feature detection.
And it's worse because there's legitimate uses of UA sniffing, and JS won't have access to it anymore - TFA wants to deprecate navigator.userAgent, so only the webserver would have access to user agent details? Why?
That becomes an explicit choice by the site to request more information, it's up to the client/browser how it responds to that. Fewer bits of information are exposed by default.
> JS won't have access to it anymore - TFA wants to deprecate navigator.userAgent, so only the webserver would have access to user agent details? Why?
I should have linked to the top-level repo with the explainer (https://github.com/WICG/ua-client-hints) as it's not immediately clear from the spec, but access to the hint values is provided via getUserAgent()
So, while UA hints could potentially supply more information than the current UA string - each item needs to be explicitly requested by the site meaning the browser can make a choice on what to return. This may depend on user's preferences, level of trust in a site, the amount of identifying information already provided to the site, etc.
You say this as though the ad industry cares.
> So, while UA hints could potentially supply more information than the current UA string - each item needs to be explicitly requested by the site meaning the browser can make a choice on what to return.
Let me introduce you to useragent switchers.
The replacement is strictly worse. Simply freezing the user agent solves things well.
They don't have a choice? The point about passive vs active is that it places control with the browser/user where they didn't have it before. You'll be able to respond to some hints and ignore others.
> Let me introduce you to useragent switchers.
And what's the adoption rate of those, I wonder... less than 1% of users? This client hints standard will make it a lot more reasonable for non-power users to control what information is being disclosed, should they wish.
So, you are saying that every time someone wants to test browser compatibility, the browser will prompt the user?
No, they're not doing that. Which means that the information is in the hands of anyone that cares. It just isn't in Apache server logs by default.
> And what's the adoption rate of those, I wonder... less than 1% of users?
About as high as the dynamic equivalent will be.
Which is why not replacing the useragent string is the only option that makes things better.
On the other hand, I did use it in combination with other techniques to provide some real information. At one point I had a hard-to-even-track kind of problem, with difficulty even looking for common trends. I created a help form for the user. Please fill out when this occurred, where you were (this mattered), what OS were you running, what browser, and so forth. What I found using the UA (and backed by other tricks) was that many users, young users who were enrolled in college, were not only unaware of their browser or operating system version but the OS itself.
Its utility began its decline when the browser makers began copying one another wholesale. It's kind of a shame. Capability-testing is better but for some troubleshooting applications, having something particular to pin your problems to is handy.
https://wicg.github.io/ua-client-hints/
You should still get this information, but more privacy oriented manner.
the proposed implementation leaves me questioning how useful the CH headers are for initial impressions of a web property if only the browser and version are sent by default, and more info is only provided on follow-up requests, assuming the user agrees to some permission popup they don't understand. it'll be the "this site uses cookies" nonsense / nuisance all over again. it'll finally push end users to install the native reddit app without those constant annoyances.
i have quite a few ux enhancements that need to know or guestimate the screen size, if the device is touch capable and OS (especially for form input styling). i dont see how it is possible without an additional delay, fetch request and some annoying permission popup to deliver a good experience in the brave new CH world.
also, a media query or any other client side method prevents me from delivering exactly the final content that's necessary and not have to do additional reflow-inducing js to get the end result.
The only legitimate use case I can think of is exception tracking, it is valuable to know which browser caused the exception.
Beyond that, a website should never, ever rely on the UA for anything.
I've been on teams a few times that tried to use UA strings to try to serve up features per-browser. Trying to do this directly falls apart quickly. There are some libraries that handle the most common problems, but that really only delays the inevitable: eventually some critical user is using a browser that you are detecting wrong, and you end up lost in endless hard-coded edge cases. Using UA strings to determine functionality isn't an effective strategy.
* Exception Tracking - The User Agent string is usually attached to exceptions to aid in reproduction.
* Outdated Client Detection - Primarily in internal dashboards in BYOD environments, I've seen the server display an error when a known outdated/insecure browser connects.
There, I just told you to upgrade without sniffing UA.
It is ridiculous for every website to be babysitting their users software, when we already have operating systems that keep software up to date, package managers that keep software up to date, and software that keeps itself up to date.
If I need to make a really important website that can't lose users like that, I just need to make damn sure it works with any relevant browser without any UA sniffing hacks that will bite back down the line.
If I'm making some bleeding edge crap that really needs a bleeding edge browser, I'm sure I can find a place for that sentence where most users will find it, and if I lose a few users, it probably isn't that big of a deal. Chances are I'd lose more users anyway due to many not wanting or being able to update for any number of reasons.
That would avoid feature detection, but still create epochs to disallow insecure browsers
--
For exception tracking, the best I can suggest is getting screen size + other features; then storing them, ready to pass with the exception
I'm surprised Google goes ahead with this without having a concrete plan to handle the first request to a page. I fear that will become an ugly mess of redirects, js hacks and broken pages. Think Google is under estimating how widespread device detection user User-Agent on the server side is...
So it is good they don't want to remove it, but just freeze it :D
- Server log analytics will not be able to provide OS/Browser stats.
- Default download pages (Detect OS automatically and redirect to the platform-specific download pages) would not work.
- "m." Sites: there are still some sites that sniff the UA string and redirect users to mobile site. It's looking for patterns like "/mobile/" in UA, and don't require a lot of updates as the post argues.
It just doesn't make sense from the perspective of HTTP. The only existing flow where an HTTP server passes context to the client and the client makes a new request based on that is HTTP authentication; but that's not intended to be optional --- generally servers don't provide good enough content with HTTP 401 and expect clients to display it in case they don't feel like authenticating. In this proposed flow, the server would need to, at least, start sending the content on the original request, and then send it again in case the client actually sends a new request.
If this is an HTTPS only feature, the server to client request could be moved up into the TLS handshake; the client could include an empty Accept-CH extension, and if present, the server could include an Accept-CH extension with the fields it wants in HTTP requests. Then the client would send the fields, or not, according to user preference. Zero added round trips.
Otherwise, it might be better if the server were able to indicate to clients what its matching rules were. If the initial response returns best case data, and a list of exception rules, the client could send a second request only if the rules matched. Then you could say if it's a specific build of Chrome where important feature is present but broken[1], or Chrome version before X, or Chrome on mobile if ARMv5 or whatever, please disregard the content and refetch with more details provided to the server, so it could provide an appropriate page.
[1] Let's say that particular version crashes when the feature is attempted to be used; it's plausible, and not really detectable from Javascript.
Let alone requiring a round trip for the server to even see the information... does that mean a 302 to itself for initial data? What about a POST? The first render is where this kind of crap is most useful for server-side rendering.
The fact is, in general, it would be nice to get the actual browser/version and os/version, and maybe have an engines list for cross-compatibility detection as a fallback.
When you come across a bug that only effects a specific browser/version (or other browsers using that engine as a base), it's a pain to deal with... cleaning it up when fixed is another issue, or worse, when you target a specific version, then the next release doesn't fix the issue and you're scrambling to update your range for the fix.
It isn't nearly as bad as the late 90's, but it's still pretty bad.
So, an extra RT if your app needs to sense which UA it’s talking to.
>the web should be is an abundant source of compatibility issues. in particular for minority browsers, resulting in browsers lying about themselves (generally or to specific sites), and sites (including Google properties) being broken in some browsers for no good reason.
Doesn't that mean every Web feature would be Google's way or the high way. Google, using Blink and Chrome would be dictating the web standards. And if it is incompatible, it would now officially be the browser vendors's fault and not the designer.
I find this intent for be very difficult to believe. Chrome's privacy policy[0] already lists a ton of information that Chrome sends to Google.
I am cynical and simply do not trust Google. I see this as a move for control rather than privacy.
I was surprised that the post doesn't contain any example user agent strings for a sample of how they are expected to look starting from the different milestones.
In the interest of fingerprinting a server would request this every time.
> It provides the information in small increments, so servers are only exposed to the information they need and request
Then the server would need and request the most comprehensive list possible in order to fingerprint someone with better granularity than a UA string could.
I'm not against this as I appreciate the value of this kind on information for the developers. It would be done one way or another. But why is this billed as mainly a privacy move? Nothing suggests it intrinsically offers better privacy when facing a website configured to fingerprint you. It actually looks like it gives even more granular info over which the user has less control than they used to with the UA.
Maybe it can be removed altogether with small whitelist of exceptions.
Firefox
about:config
general.useragent.override, String, leave empty
How do I know which (binary) download to offer my users?
Edit: How do I provide reasonable defaults when the user’s OS actually matters?
That’s a load of horseshit if I’ve ever heard it.
iOS users will almost always install apps via the AppStore. Most Windows-users are probably not interested in a DMG. Are you really going to argue against that?
I agree that taking away options based on OS-detection is a seriously nasty UX anti-pattern though.
It may be true in the case of iOS and Android, because they are so locked down. However, on more powerful platforms that Windows, Mac and Linux are it isn't. I may want to run it in a VM, or not install it, but place it somewhere on a shared drive, or anything that a non-handicapped OS is capable of facilitating and many of these things will mean I will want a binary not meant to be run by my native OS. Sometimes it happens that one of my devices will break and so I want to use another one to download something that will help me fix the issue. But now I'm going to have to go full Sherlock Holmes on a website that thinks it knows better what I'm looking for.
Regular users are not technical and if you ask them to make a choice based on technical matters (Windows, MAC, Linux, Apple AppStore, Android AppStore, Chrone WebStore, whatever), where you instead could have had -1 clear choice- already presented by default in a big nice button, you will lose conversions or increase tech-support costs.
The evidence for this is so overwhelming that knowingly trying to ignore that makes it look like you have an agenda or horse to grind.
Perhaps you should propose a feature detection for it. It could look like the media source API for resolutions and formats for web video - except the same for binary formats for software.
In what other places of application development should we also stop providing reasonable defaults and force the user to choose instead?
- country? city?
- localization?
- number for local support hotline?
- email of logged in user?
- type of currency for payments?
No? Then why is having a reasonable default for downloads bad?
But anyways, the service should not make it difficult to download if it detects a wrong computer; it should allow it always, and should not be so difficult by trying to hide it.
- Chrome is still shipping with 3rd party cookies turned on by default (Safari and Firefox have them off, by default)
- Chrome usage stats are sent to Google including button clicks. This is admitted in the Chrome privacy policy.
- Chrome on mobile automatically shares your location with your default search engine i.e. Google
- Chrome sort of forces a login …which shares browser and user details history with Google
- Google redirects logins through the youtube.com domain to enable them to set a cookie for YouTube as well as Gmail or whatever, every time you login. Naughty stuff.
So the stated reason for the change doesn't appear to make sense, suggesting that something else is going on.
It amazes me that more people aren't calling Google out on this.
That's unsubstantiated and dilutes the discussion IMO. If you read the post, the proposal outlines a bunch of good reasons to stop supporting UA strings (feature detection, etc)
This doesn't get more true by just repeating it over and over. If you login to Google it'll show up in Chrome next to the address bar but it doesn't enable any syncing to Google servers. That's a different step and it requires opt-in. You can also use Chrome without logging in to any Google services.
I don't get why privacy advocates, who often have a point when talking about Google, have to rely on FUD.
Anyway, Google is a big company. Different teams have different priorities. Does the US government care about privacy? Depends - at the very least - whether you're the NSA or the FTC. Given the many signs in the past that parts of Google are willing to fight other parts of Google they disagree with, I think a better strategy for us as the community is to call the Chrome team out, specifically, on things under their control and otherwise not be excessive cynical about the fact that they along a hundred thousand other people work for Google, and some of those other people are bad.
(Automatic login to Google is a think I think we should call them out for, to be clear.)
This all-or-nothing mindset ends up harming privacy in practice.
That would be check mate for all other advertisers.
I'm just not sure whether it's good or bad that antitrust regulators won't notice before it's too late.
I'll take that benefit even if it tilts the advertising table in favor of Google. I don't care even a little about the overall health of the advertising/marketing industry.
Holy fuck!