Hacker News Reader

Comment by downrightmike | Hacker News Reader

Ooph, now the AI will be wrong and the human won't know it. And that's running our... let me see... life saving care. Cool. cool. cool.

View on HN

"Computer says no" has been a trope of terrible customer service long before AI.

"Computer says no" is about unhelpful bureaucracy. Being knowingly locked to a specific procedure and unwilling to divert from it is a different thing from mistakes being made that you are blissfully unaware of. The problem is now invisible and the patient is dying for no reason anyone can discern.

The people who say "computer says no" are also unaware that's a problem. It's invisible to them. To them the process is the point.

I think that two arent as different as you believe.

Well, possibly. To paraphrase the great American philosopher Donald Rumsfeld - there are known knowns, known unknowns, unknown knowns and unknown unknowns. And making the actual process a black box for the people managing it will create a lot more Unknown Unknowns.

The "unhelpful bureaucracy" arises precisely because the process is structured so there's no point in the chain that can actually consider and solve your problem directly. The process is the point, outsourcing work to the opposite end of the globe has long been a perfect way to guarantee no it's robust against errors like "compassion" and "helpfulness", and that's as true of LLMs performing those functions tomorrow as it was of people doing it yesterday.

Has anything really changed for transparency?

Your average devs are wrong more than the AI is at this point.

At a former job for a regional bank in South East Asia my boss used to joke that outsourcing is really great for the company, because now they can fail to deliver their software projects for cheaper.

This is not about development. This is about rubber stamping insurance denial by an unaccountable system.

If you sell something in the US, and a customer sues you, going to court and saying you are not accountable because the work was performed by someone in another jurisdiction is not going to go well for you.

The only time that works if you have no assets or presence in the US, or I guess you have an in with the US president.

It is about a.) avoiding criminal liabilities for frauds by the insurance company b.) facilitating desirable fraud by employees while keeping plausible deniality.

It is super hard to build class cast action and suing companies for denied insurance claims is also quite hard - and super expensive. Besides, the current management dont need it to go on forever, they just need it to go on for now. If company gets to pay fine 20 years later and you already pocketed the bonuses and salary, why would you care?

Reply on news.ycombinator.com