Why not to whitelist operating system user agents | Hacker News Reader

Why not to whitelist operating system user agents

neelc.org

147 pointsneelc119 comments

I ran into a similar problem with the website of my general practitioner. It worked fine in all cases, except when using Firefox on Linux, which I use.

After lots of testing and trying to contact whoever built the website I found that it blocked only user-agents which contained this literal string:

    X11; Ubuntu; Linux

Only when that string was in there verbatim would it fail all requests with a 403 Forbidden.

After I saw the same error with some other websites for businesses in my town I started seeing a pattern. The company that hosts/builds this website apparently copy/pastes their basic server set up, and so every website they host works everywhere, except when using Firefox on Linux. So maybe one in a thousand users gets this.

I posted my search for the cause of this issue on StackOverflow¹, and even got a reply from (presumably) someone who works for the company that hosts these websites, but alas, the websites remain broken to this day. They suspected a hack to prevent some WordPress exploit…

It's frustrating, because a general practitioner's website should not fail like this (it is a point of contact that sits just below emergency services), but the people that work there don't understand the problem, and the company that hosts is can't be arsed to fix the issue.

1: https://stackoverflow.com/questions/66185885/some-websites-r...

> After I saw the same error with some other websites for businesses in my town I started seeing a pattern. The company that hosts/builds this website apparently copy/pastes their basic server set up, and so every website they host works everywhere, except when using Firefox on Linux. So maybe one in a thousand users gets this.

Haha! Never attribute malice when a simple incompetence would explain it!

This is not straight incompetence though, as that config is not there by accident. It’s more in line with “screw that 1 per 1000 users”, for whatever reason.

Malice might be too strong of a word, disdain could be closer to what we are seeing.

I would guess it was more in the line of not suspecting that it was a valid user-agent string; just one used by bots.

"No True Human..."

Not even checking what matches that user-agent would be a deeper level of “screw it”. You’d google what you’re banning before banning it.

Yeah, it's amazing how short sighted some developers and sysadmins can be when tasked with solving a problem.

Oh, we have a WordPress exploit? Let's blacklist User Agent strings!

If you're in the US, call your doc and mention that this may be a violation of the 21st Century Cures Act, as it is preventing or interfering with the access, exchange, or use of electronic health information.

It would be a stretch to call this an outright violation (as they could satisfy the requirement by printing the information you want and mailing it to you...), but it's a trendy topic in healthcare right now, so it might be enough of a motivator.

> call your doc and mention that this may be a violation of the 21st Century Cures Act

Unless you’re a lawyer, don’t do this. Empty threats are more frequently sorted into the crackpot category than the kind one responds to.

While this is generally true, in my experience anything related to healthcare regulations are a lot less likely to get written off. Generally the failure mode is quite the opposite (people assume all sorts of things about, e.g. HIPAA, that aren't true).

EDIT: I also wouldn't characterize it as an "empty threat", as it is neither empty (I think a good faith argument could be made that this needlessly disrupts patient access to information), nor a threat (it's just making them aware of something).

More realistically, they’ll refer you to their lawyer.

I host a few hundred wordpress sites and I recognise that string by sight! Tons of bots seem to use it. I haven’t 403’d it (yet) though.

The reaponse to that is to play dumb and repeatedly report yhar the website is failing, but don't try to diagnose it for them. Just focus on your problem, and keep annoying them so that it's their problem too (write a script to email occasionally, and share it with your friends). Eventually they and other customers will complain to the website vendor to make it their problem too.

Your comment seems to be dead (along with the last four of yours) which suggests you might be shadow banned? I vouched for you, hope it helps.

One happy addendum:

Seeing this topic on HN reminded me to try to and contact the website builder again, and this time they did get their hosting party to fix the problem.

The explanation as passed on to me was:

> There was a bit in the htaccess that was there since 2019, we don't know why.

Try Japanese business banking - where you have to pick an OS and stick with it when registering (with a paper form), and must use either the ESR release of Firefox or Internet Explorer. If you don't have a user agent of either of those it won't even let you sign in.

For a while Firefox ESR was the only one still supporting digital certificate request/generation (KEYGEN). My (not in Japan) and other banks used this as one securiy mechanism. With new EU rules they've now downgraded their security to a Phone app + some SMS verification.

Always spoof your user agent string, for firefox the setting is general.useragent.override

You think that's that easy?

Maybe now, not really sure if there are now changes (hopefully, since Microsoft is dropping IE), but in a time where browser plugins are abound you can't place an ActiveX plugin inside Firefox (or vice versa).

Basically security done wrong because it will only affect users and won't stop attackers.

Try Spanish online digital administration. The digital certificates only worked in IE. And mid-process they require installing a Java-based program that required a different type of digital certificate. That of course make you restart the browser and lose all the data entered. Just wow. I couldn't even come up with such a bad process if I wanted.

Not sure if you're referring to something old, I've only been in Spain since 2012 but I'm having zero issues with the digital certificate for various government websites, from hacienda to local city governments websites. Never have I been forced to install Java either. I usually use Firefox on Linux and seems to work fine, at least for me.

It was just yesterday. To sign in you need @clave, but to actually sign the document, you need AutoFirma, that's the java-based tool.

That's strange, I've never needed @clave nor AutoFirm (and in fact, I've never even heard of AutoFirma before). I just use the digital certificate (for signing in and for signing documents) received from the government to sign everything directly in the browser, without any Java or anything else really.

Well yes, your comment outlines the issue. Why is there three different certificates supposed to do the same thing?

Oh boy are you wrong. We Spanish people love trash talking our own country, but after 3 years in Japan, I can tell you that the Spanish banks and bureaucracy are light-years ahead of Japan.

At least in Spain we have online administration, even if it's not perfect. Here? Hand written forms, hankos and fax machines. Everything is at least ten times as difficult as it should be.

Banks do stuff like this all the time - they are always the long tail of security - could be a topic in itself. I contemplated this for a very long time and decided that JP Morgan would rather take the hit for bad security then pay wages and benefits to support people to deal with password resets, lost yubikeys, etc. No other answer makes sense.

My advise to OP is to dump Chase, Citibank, Bank Of America, ASAP. Move your money to one of the millennial focused banks, or an ETrade checking account.

The big banks hate you, they think your stupid, offering you retail banking services is the bane of their existence. They are going to knock you over with $40 fees because you SHOULD pay them to put up with you — at least that is how they see it.

There are much better options these days, just search for zero fee checking.

Any suggestions?

Get a schwab account. No fees, fee ATM worldwide, and TOTP.

Unfortunately can’t change loan providers, as my auto loan when was financed through the dealer ended up at Jp Morgan & Chase.

You definitely can do that, if you want to. Refinancing a loan is not especially complicated.

And pay all those expenses to have the note bought up by one of the same large banks? Selling debt is a very common practice.

Personally, I use a local credit union that doesn't sell their loans (there are several CU's and regional banks in my area that make that commitment).

From what I've seen CUs are not competitive with the big banks for jumbo loans. I got a significantly worse interest rate on my home loan with the CUs compared to BOFA (who I ended up going with).

Why do you use a website for your loan? Autopay and never talk to them again until closeout.

ETrade is now a subsidiary of Morgan Stanley. While MS is only half the size of JPMC, they’re not really a small bank.

(Wikipedia isn’t up to date BTW. Even before the Etrade they had over $1T in AUM)

Or better yet, use a credit union.

Dude, all I know is that I was using chase for one of my businesses for 3 years, millions of dollars coming in via Intuit payments -- no problems, then I switched from Intuit for ACH to using Seamlesschex.com, and then after the first batch, they locked up my business bank account, and then after a few months talking to a call center in india, with the bank manager sitting there (there is nothing they can do when they automatically lock your account), the people in india saying they will "never" return the hundreds of thousands in the account they locked up, I filed a lawsuit against Chase in civil court the same day, and then a month later, the attorney representing the case mails me a check for the full amount they stole from the account. I understand risk, but this was months later, all ACH payments, and everyone knew they owed this money. My only regret was not charging them with theft/fraud and 3x the money back for damages. Bottom line -- don't use Chase for anything. They suck.

I had someone working at Chase telling my vendor how much money was in my account. I was a private client at JP Morgan and had a business account with them.

The vendor was threatening me and using my bank account level (down to the penny) to make the threats.

Chase identified the culprit, told me who it was, then offered me lifelock identity theft protection as a courtesy for my troubles.

I haven’t had $1k in my private client account since.

Chase closed my 10-year-old+ personal account without a warning or an explanation. I recommend avoiding them.

Biggest banks routinely do such things simply because they can.

And like most things if you're not wealthy enough to afford a good attorney, and they usually can just draw out a case until you run out of money, the only people capable of protecting you are legislators who have failed time and time again to adequately take on big business abusing their positions.

This is what small claims courts are for, if you have an equivalent in your country. No/minimal lawyer involvement, as long as the amount is relatively low. If you have hundreds of thousands or more stuck in an account though, you likely have access to a lawyer. (+ obviously winnable cases will be sometimes taken without cost since the lawyers will negotiate that in the damages)

Blue Cross and Blue Shield of Illinois (I can't vouch for any of the other Blue Cross affiliates) recently redid their website. I was wondering why the hell it was kicking me out after logging in, with a "did you forget your password?" message. Multiple password reset attempts later, I called their tech support and asked what was up. I use Firefox on Linux as my daily driver.

What was up was that on their new site, I had to use Google Chrome and only Google Chrome. Not Firefox, not even Chromium. I wonder if Edge even works.

I'm seriously considering switching providers over it.

I have similar issues. I couldn't get to the billing site for BCBSIL from any browser on my system for the past year.

Unfortunately there are no decent alternatives for a PPO, where I am. If it's browser issues vs an HMO, I'll begrudgingly accept developer incompetence.

If you want to push back against the bureaucracy on this one, find a Firefox-only accessibility addon that can’t be used on their site, and play the ADA angle by sending a polite email mentioning that their negligent browser restrictions prevent “your favorite” visibility tweaker/screen reader/etc. from being used, harming the ability to access the site. You need not disclose the details of exactly what, if any, disabilities you personally suffer from.

No thanks. Dishonesty does more harm than good.

Can anyone confirm this?

I don’t have a FreeBSD machine handy right now but I just switched user agent to FreeBSD amd64 on a Linux machine with Chromium 95 and have no issue with the front page or logging into chase.com. I have rarely encountered issues using this Linux/X11 setup on chase.com for years.

Is it possible they are using an ancient browser and incorrectly assuming it’s the OS part of the user agent?

I can confirm this is 100% false. Been using Linux to login to Chase for years, never had any problems (other than weird ad-blocker issues which are cross-platform). Just tested again just to confirm that I can log in just fine.

No User-Agent switcher required.

Small nit-pick, but OP isn't running Linux, they're running FreeBSD.

They too mention Linux so it's possible they aren't aware of the difference.

It's possible "Linux" is allowed, but not *nix/Unix?

Quite likely.

May be OS with small userbase and less popular browser combo.

I have seen the same thing, the article is correct.

Take a look at their evidence that chase "openly admits to hating linux and freebsd". It's a reddit post with 3 votes about a CS response saying not supporting linux doesn't constitute an ADA violation.

Everything in this article and it's supporting evidence is a stretch and should be evaluated very carefully.

I use it almost every day. I'm thinking The user is using a weird user agent/browser and misdiagnosed the problem.

It looks like Firefox but there's just so many small browsers these days. Honestly I'd need to see the offending code. If it's user agent testing, those strings should still be readable even in a compressed js unless they run it through an obfuscator

The small browsers thing sounds about right. Check the link he posted about the email someone received on Reddit. It's posted like it's a screenshot from Mutt or some other terminal mail editor. Looks more like they are flexing their email terminal usage not just copy/pasting the message (png for text? come'on!).

Probably using qutebrowser or something else like that.

platform.js 1.3.1 is in there (https://github.com/bestiejs/platform.js/)

I see this from the pretty-printed version

    function a() {
            return /Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent) ? "Device" : "Desktop"
        }

So let's try to just say I'm say, Opera Mini, still no dice. I tried a bunhch of these to no avail. I don't know how the OP got this.

Well that kind of matches the message he received from the website. It looks like they are just trying to notify you that there is a Chase Mobile App available (which for the average phone user, would be 99.9999% better than using a browser). Personally I wouldn't use a phone for banking but some people don't have laptops/desktops.

> Personally I wouldn't use a phone for banking

Using your bank's mobile webapp is very useful when they, for some god awful reason, decide to use SafetyNet.

Just from the Reddit post as well - doesn't feel overly user-hostile or deserving of the 'JP Morgan Chase Bank admitting to me they hate Linux and BSD desktops and actively block them' title anyway.

If there's active blocking based on OS (from replies in this thread, evidence seems to be slim) then that's not great, but this seems to be pretty one-sided so far.

> should be readable even in a compressed js unless they run it through an obfuscator

User-Agent may be determined on a webserver/proxy level and request redirected silently to a page with JS just showing the banner. It does not have to be based on JS checking anything.

The article is 100% correct, I've experienced the exact same thing. For a while I thought it was blocking me due to uBlock or something, took me a while to figure out it was just the user agent.

I’ve certainly been blocked by chase on linux with firefox, but I was only using their auto loans at the time. It was super annoying, because I first tried to get a payoff quote on the iOS app, then the mobile website, then on a linux laptop, before resigning and using my work Macbook. Perhaps other lines of business support linux better?

I have been using chase.com on Ubuntu 18.04 and 20.04 with Firefox for years without any issues.

My employer for currently blacklists Firefox from being used to launch a session in their 3rd-party remote desktop portal. I use a UA switcher. It works fine. This behavior, while brain-dead, is at least trivial to circumvent. I'm happy to let them continue to check a box on their audit preparation form saying they have control over this, and to continue to have a URL rule to change my UA for the portal, rather than having to hack my client further or keep a separate browser around to launch my daily session.

had the same with a very broken citrix setup. Inalways hated citrix itself because how srupidly it was set up, but the more quirks I was working around, I realized that in the windows world it's actually a pretty sophisticated product with a lot of tunables for even Linux guests.

Nevertheless, I left banking for good and chose a company where I have real IT engineers as colleagues.

Downloading, installing and running kernel mode software to prevent cheating is already required for a number of online games.

I wonder if/when banks will extend this idea to banking to prevent fraud?

Perhaps it'll be merely an optional thing at first, like 2FA.

Later it could become something that while optional, does get you a better price of some kind, much like the driving trackers that some auto insurance companies offer.

Before long, it could even become mandatory or there could be a penalty or higher price or fee to pay if you don't do it.

Just a random idea or conspiracy theory of what's possible I suppose, but it feels like something that could be possible in the not too distant future.

Already a thing on Android. Google "Safety"Net API is used by many apps to verify that the system is not rooted or modified. These days it's combined with hardware attestation from the phone to verify that the installed OS is properly signed by the manufacturer and unmodified. So there's no workaround to using an alternative Android distribution, or rooting your phone, and still being able to use media / banking / other apps.

Of course using the bank website with the phone's browser still works...

Can confirm it works fine for me under linux firefox. OP, just adjust your user agent string if you're using a weird browser and proceed at your own risk.

(I say this because you're dealing with actual money, so incompatibilities from your browser might cause major problems if you're not careful)

Can confirm it works NOT under FreeBSD and Firefox, with useragent Win/Chrome it works.

For anyone who works at the company who does that: why you do it?

Is it to reduce amount of testing, and only have a few "blessed" browsers with guaranteed happy experience? Any other reasons?

It’s not so much for happy experiences, as it is to place bounds on what the development team is asked to do.

They may have spotted a bot using this UA and deduces it's a pattern

I mean there's an excellent and obvious answer and that's cost benefit when it comes to QA. Anything transactional, banks in particular, want to be 100% sure that end user experiences are doing exactly what they're expected to at all times. No one is being served incorrect information or is improperly served terms or disclaimers that they can use a leverage in a lawsuit. The tech teams likely have an explicit support matrix of browsers to test against and anything not on the list is considered untested and unreliable. They can't legally indemnify themselves against defects.

Interesting point on lawsuits! I never thought this way (maybe because lawsuits sounds like a very American thing, and I'm in EU)

As a Firefox/FreeBSD user occasionally annoyed by this nonsense (not Chase but other things), but not being knowledgeable about modern web standards evolution, I wonder if https://wicg.github.io/ua-client-hints/ will fix this by killing User-Agent headers.

The fun part in web dev is to make sure everything works on the 0.5% non mainstream browser/platforms. Only supporting 99.5% is boring.

A popup "your OS browser combination is not supported, some things may not work" is a much less nuclear option.

If you know which parts will not work in advance. What if the "Send $$$" button does not appear due to a CSS misfire?

eslint-plugin-compat [0] and stylelint-no-unsupported-browser-features [1] can help you know when you're using an unsupported browser feature.

[0] https://github.com/amilajack/eslint-plugin-compat

[1] https://github.com/ismay/stylelint-no-unsupported-browser-fe...

Things fail in the weirdest ways in unsupported environments though, it’s not like the “make transfer” button doesn’t work, it’s more like it might not even show up in the first place. Having 99% of your website work and the last 1% not work is a dealbreaker in many cases, and these “the site may not work for you based on your OS” banners lead the user into thinking it does work 100% if it works in 99% of the cases.

Not saying this is the way it should be, just saying that “doing your best” to allow unsupported platforms often leads to a terrible and confusing user experience.

Unlikely. This is a browser. If it's going to fail, it won't be some js somewhere. It's going to be the whole thing.

That's not true. All it takes is using an unsupported CSS rule for something to simply disappear from the page under certain circumstances. As a recent example, I found out some of our users couldn't find a specific button in an application. It still existed, but we used the zoom property to make it stand out more, which for some reason is only supported in Chrome.

How about assuming it works until users report it does not?

I would not want a money transfer to “not work” and end up moving $500,000 rather than $500.

It is to a point, but then it just becomes painful. If you want to keep a good user experience for modern browsers while supporting ancient ones, you'll probably be writing at least all your layouts twice.

You guys are getting a good user experience from your banks?

I am since reverse engineering their mobile app protocol and developing a Python library for it :)

> Worse, Chase even openly admits to being hostile to Linux and BSD to someone on Reddit. It’s something even Microsoft, Windows PC/hardware OEMs, or Apple won’t do.

If you click through to the link, you will see that this claim is totally made up.

Seriously, it sounds like the author of that post claimed discrimination for not supporting Linux.

It's not just banks. Google Maps will refuse to work if you're running OS X Lion, even if you're using a fully up-to-date version of Chromium[1] which is just as capable as any other Chromium-based browser on any other operating system.

Google Maps work perfectly on Lion if you fake the user agent, because of course it does, it's a web app and the underlying OS is irrelevant.

1: https://github.com/blueboxd/chromium-legacy

All the UK banks used to do this about 12 ish years ago. No longer. What they do try and do is shove Rapport down your throat instead.

Ugh, I remember when HSBC pushed Rapport. Is it still a thing? I run Linux exclusively and haven't seen them try to push it for a long time so not sure if it's still a thing.

They still prevent you from running their app on a rooted Android, which is nice considering I can do much more dangerous things with my money from the web site.

I haven't seen or heard of anything related to Rapport with HSBC UK for probably a decade now.

Thanks for the upvotes.

I have updated my article. It seems Chase is whitelisting OSes, but they seem to allow Linux and not FreeBSD based on comments and using a Linux user agent.

Chase may not block Linux because does Chase exactly want to deal with angry Linux users on the phone, or see Linux die-hards switch to competitors. Even if 1% of customers leave and don't come back, it could anger Chase's investors.

They may not officially support Linux but the web developers allow it anyways since it's too big of a minority.

They still block FreeBSD. Whether Chase's web developers don't know about BSD or they're willing to let BSD users switch to Citi Bank, I don't know.

I mean, they shouldn't whitelist by OS, but I don't know what the reasoning of blacklisting FreeBSD is.

This is interesting to me. I actually left Chase a few years ago over a very similar issue: their statement PDFs would show up blank in all the PDF readers I tested. After contacting support and being told that the only option was for me to install the latest Adobe Acrobat Reader, I told them to close my account.

I never even thought about the accessibility requirements. I am sure that relying on PDF features that only the latest Acrobat supports hurts a lot of people on that front too (unless Acrobat happens to be the most accessible of readers?)

I once got denied for a credit card app with a different company even though they pulled my credit because according to the company, quote, my user agent (Chrome on Linux) was suspicious activity.

My advice is to drop the bank now, after testing a replacement- there are plenty of smaller and "neo banks" looking to have your business with real development teams. I use the big, old and stogy bank of America but I have never had a complaint using desktop Linux and Firefox / chrome there.

Has this sort of thing been argued in court as an ADA issue? I could understand why using Linux might be considered legally a "choice", but if there's better ADA compliant tooling in Linux over windows, then a legal argument might just exist..

Unless one was to claim that Tux is their emotional support animal, I doubt it. Linux on the desktop has usability issues for able bodied people. I strongly doubt it has any edge on MacOS or Windows when it comes to accessibility.

Sure, but usability issues aren't necessarily issues under ADA.

Unless we have more details about this claim we can’t be sure, but it seems like the ADA claim is probably well-intentioned but also not correct.

Chase does not have to implement a specific solution to a users problem, they have to make a reasonable adjustment - I.e. you can install a small ramp if someone asked for a lift.

Depending on the issue raised, chase may feel they have a reasonable way of providing the services - for instance if the user is blind and uses some specific Linux screen reader then telephone banking may also be a reasonable adjustment rather than Linux support.

Chase may see supporting Linux for all users because of one persons disability as an ‘unreasonable’ adjustment (I don’t see the issue, but this is approximately how the claim would work). To be open I’m not exactly sure how ADA works as I’m more familiar with UK legislation.

For an ADA claim it shouldn't matter. You're usually not asked to demonstrate your disability.

There are people who only use Linux in textmode.

Sure, but I'd be surprised if any modern web app worked in TUI browsers.

A lot of them do if the text is scraped. Lack of scrapable text is an ADA issue.

A JS app can be perfectly accessible (if written correctly), despite giving a CLI browser nothing but a "please turn on JS" page.

Both Firefox and Google Chrome support powerful screen readers and other accessibility features based on an open standard. A site using these would surely be ADA-compliant

You can't dictate a specific solution. It probably hasn't happened yet but someone who exclusively uses the FOSS software probably has grounds to request flat text. Flat text may be better with current hardware, who knows.

A business is going to have a hard time arguing that providing text is unreasonable.

You will need to state what the specific issue is for yourself/someone else though, rather than just what you want the solution to be.

A greater focus on scriptability and customizability might make it a better OS for people with some disabilities. Certainly I've heard that at one point Linux was the only OS you could use in Welsh, for example (not that that's a disability, but it's similar in terms of being a minority need).

It shouldn't be allowed to ban web access from all free operating systems ;)

This gave me the idea to ban all non-free systems:

"You are using a non-free Operation System and thus signing away you fundamental rightsas a user. Please use a free Operationsystem like GNU/Linux to access this website."

But then you run into the issue of half of websites blocking free operating systems and half (haha) of websites blocking non-free operating systems.

At that point we'll need a user-agent switcher that is website aware to know which sites need which user-agents. Like secret hand signals to get into your secret clubs.

I'll just pass and not use any of it at that point.

> At that point we'll need a user-agent switcher that is website aware to know which sites need which user-agents.

Microsoft Edge already has something like this built in to get around Google's user agent checks.

Interesting. I just tried logging in from PopOS. No issues. Does it only affect FreeBSD?

I mean worst case scenario I can always open dedicated Windows VM, but I will admit that the trend is troubling.. especially with Win11 push towards 'trusted computing'.

Huh? I am able to log in to Chase just fine in my banking virtual machine (Ubuntu 20.04 LTS; Firefox 94.0 64-bit). I’m not using User Agent Switcher, and the User agent string shows that I’m using X11/Ubuntu.

As an aside, one issue Chase did have, 10 years ago, was that their DNS servers would return “query refused” if you sent them an AAAA (i.e. IPv6 IP) query. This actually caused issues with my recursive DNS server; I had to make AAAA (IPv6) queries handle errors differently than A (IPv4) queries. I just checked, and Chase finally fixed their DNS and IPv6 issues.

Congratulations, you're part of the "Some Linux user agents get through" segment noted in the second paragraph.

So instead of moving their business elsewhere, they installed a user agent switcher?

Meh. IIRC, basic functionality is still there and maybe even a bit more, there’s just some features you cannot use.

Use a different bank if your preferred platform is unsupported. No article necessary.

Reply on news.ycombinator.com