History of the browser user-agent string

The story needs to continue with Microsoft releasing Edge and adding the "Edge" identifier to the user agent string, then in Chromium Edge, changing the identifier to "Edg" because they need to ensure servers do not confuse the new Edge and the old Edge.

I use a text-only browser to read HTML and make HTTP requests with TCP clients instead of the browser. For many years I have sent no UA header. The results have been great. Assuming a website has many versions for different form factors, there still has to be a "default". Many times it appears to be mobile. But there is more to the UA header than a means for selecting an appropriate design for a given form factor. If it were as simple as that, then there would not be so many UA string shenanigans. Websites use the UA header for myriad "illicit" purposes. Fingerprinting users is one obvious example. By not including the UA header, we as users can deny its usage. It does not stop fingerprinting but it removes the contents of the UA header as a data point.

There are very few websites I have found that require a UA header. Squarespace hosted sites are one example. Another is sec.gov. (Usually, they do not care what the string is, they just want something.) I have the local forward proxy configured to add a UA header for these few sites. Recently I had to add wsj.com to the list. It really makes one wonder why the heck these sites need a UA string. There are incorporated for-profit entities whose core speciality is UA sniffing for tracking users, advertising and who knows what else. Perhaps WSJ wants our fingerprint. As others will point out, Google is trying to eliminate use of the UA header. One wonders what these folks who think UA is significant and reliable are going to do then.

There was one example I once found of a web site that required a certain UA string as a sort of shibboleth. This was unique because they told users what was the required string. Thus, unless a user could set their UA string to the value the site indicated was required, they could not access the site's content. To me, that is what the UA string is, an HTTP header that anyone can change to anything they want.

I am more than happy to provide whatever UA string a site operator wants. All they have to do is tell me what it is. They could provide screenshots of the different designs they have made for different form factors then indicate which string to send to retrieve pages with a particular design. Otherwise I see no reason to send this header. It is hardly ever required.

There was a post here on HN the other day that also talked about the User-Agent that I started reading and now I can't find it again. Originally I thought it was going to be this page (as I've seen it before and quite enjoy it) but I'd like to finish that blog post. I don't suppose anyone else knows what I'm talking about? My searching is coming up with nothing.

User-agent strings finally make sense now. I thoroughly enjoyed this article.

Articles like these are my favorite. Some of the design decisions about the web feel so absurd until you follow the trail. It's like a slow descent into madness.

Past comments: https://news.ycombinator.com/item?id=298844

Ha! I remember this from many years ago; when was it first published?

At this point what is stopping browsers from removing all this cruft?

“ In the beginning there was NCSA Mosaic, and Mosaic called itself NCSA_Mosaic/2.0 (Windows 3.1)”

Mosaic was for UNIX at first and wasn’t the first browser, so what on earth is this?