I think, more frighteningly, the potential for it to make decisions on insurance claims and medical care.
I've also vaguely heard of a large company that provides just this as a service -- basically a factory where insurance claims are processed by humans here in VN, in one of the less affluent regions. I recall they had some minor problems with staffing as it's not a particularly pleasant job (it's very boring). On the other hand, the region has few employment opportunities, so perhaps it's good for some people too.
I'm not sure which country this last one is processing forms for. It may, or may not be the USA.
I don't really have an opinion to offer -- I just thought you might find that interesting.
So many stupid comments about AI boil down to "humans are incredibly good at X, we can't risk having AI do it". Humans are bad at all manner of things. There are all kinds of bad human decisions being made in insurance, health care, construction, investing, everywhere. It's one big joke to suggest we are good at all this stuff.
What is needed from the AI is a trace/line-of-reasoning to which a decision is derived. Like a court judgement, which has explanations attached. This should be available (or be made as part of the decision documentation).
I think the safest would be for a reviewer in an appeal process to even not have any access to any of the AI's decision or reasoning, since if the incorrect decision was based on hallucinated information, a reviewer might be biased to think it's true even if it was imagined.
This would forbid things like spam filters.
Do you have a source for your somewhat unbelievable claim?
It's pretty standard practice for there to be a gradient of anti-spam enforcement. The messages the scoring engine thinks are certainly spam don't reach end users. If the scoring engine thinks it's not spam, it gets through. The middle range is what ends up in spam folders.
Most spam is so low-effort that the spam rules route it directly to /dev/null. I want to say the numbers are like 90% of spam doesn't even make it past that point, but I'm mostly culling this from recollections of various threads where email admins talk about spam filtering.
> by 2014, it comprised around 90% of all global email traffic
It is available to the entity running inference. Every single token and all the probabilities of every choice. But every decision made at every company isn't subject to a court hearing. So no, that's also a silly idea.
Even if you were right, AI doesn't change any of it - companies are liable.
Litigation is mainly a form of sport available to and enjoyed by the rich. And I mean serious litigation like taking on some corporation with deep pockets; not pick-on-someone-your-own-size litigation as in neighbor cut down a tree which fell onto your toolshed.
Second, have you ever look at rhat space? Because the agencies that done this were already weak, the hurdles you had to overcome were massive and the space was abused by companie to the maximum.
I mean, imagine that making an insurance claim with a black-sounding name results in a 5% greater chance of being rejected. How would we even know if this is the case? And, how do we prevent that?
Now, of course humans are biased too, but there's no guarantee that the biases of humans are the same as the biases of whatever AI model is chosen. And with humans we can hold them accountable to some degree. We need that accountability with AI agents.
AI will perform tirelessly and consistently at maximizing rejections. It will leave no stone unturned in search for justifications why a claim ought to be denied.
If they over-approve they will be unprofitable because their premiums aren't high enough. If they under-approve it'll be because their customers go elsewhere.
It’s just that A) I didn’t choose this insurer, my employer did and on balance the total package isn’t such that I want a new employer and B) I expect pretty much all my available insurance companies to be unreasonable.
Secondly, the reasonablity or unreasonability of payouts is linked to premiums.
In other words, one way that the parsimonious insurer would still have customers is that they offer low premiums compared to the liberal insurers.
Even people who know about the bad anecdotes from reading online reviews will brush that aside for the better deal. (Hey, reviews are biased toward negativity and miss the other side of the story; chances are that wouldn't happen to me.)
The free market doesn't optimize for quality. Firstly, it optimizes for the lowest price for a given level of quality. But the price optimization has a second-order effect of a downward pressure on quality.
If you're selling something and the margin is optimized: it's about as cheap as can be, what you can do is reduce quality by some epsilon, and make a corresponding decrease in price. It still looks like about the same quality to someone not using a magnifying glass and fine-toothed comb, and you have temporary price edge against competitors. That triggers a kind of "gradient descent" of declining quality which bottoms out at some minimum level of quality below which viability starts to get eroded past a point where the market still finds the thing acceptable.
If it's health insurance, it's not a free market. You don't have a choice. It's employer provided, so suck it up buttercup.
It's just socialized medicine but implemented in the private sector and, like, 100x more shit.
UHC, one of the largest insurers in the US, has a claim denial rate somewhere in the 30s if I remember correctly. Well... that sucks.
Quote from an IBM training manual from 1979
Seems just as true and even more relevant today than it was back then
The computer allows the humans a lot of leeway. For one thing, the computer represents a large diffusion of responsibility. Do you hold the programmers responsible? The managers who told them to build the software? How about the hardware manufacturers or the IT people who built or installed the system? Maybe the Executives?
What if the program was built with all good intentions and just has a critical exploit that someone abused?
It's just not so straightforward to hold the humans accountable when there are so many humans that touch any piece of commercial software
For example, if American Airlines uses a computer to help decide who gets refunds, they can't then blame the computer when it discriminates against group X, or steals money, because it was their own "business decision" that is responsible for that action (with the assist from a stupid computer they chose to use).
This is different from when their landing gear doesn't go down because of a software flaw in some component. They didn't produce the component and they didn't they didn't choose to delegate their "business decisions" to it, so as long as they used an approved vendor etc they should be ok. Choosing the vendor, the maintainence schedules, etc, etc: those are the "business decisions" they're responsible for.
If American Airlines uses a computer to automatically decline refunds, which human(s) do we hold accountable for these decisions?
The engineers who built the system?
The product people who designed the system, providing the business rules that the engineers followed?
The executives who oversaw the whole thing?
Sometimes there is one person who you can pin blame on, who was responsible for "going rogue" and building some discrimination into the system
Often time it is a failure of a large part of a business. Responsibility is diffused enough that no one is accountable, and essentially we do in fact "blame the computer"
Personally I'd be satisfied holding the company as a whole liable rather than a single person.
All that does is create a situation where decision makers at companies can make the company behave unethically or even illegally and suffer no repercussions for this. They might not even still be at the company when the consequences are finally felt
It means that the company is sued and is responsible for damages.
> decision makers at companies can make the company behave unethically or even illegally and suffer no repercussions for this
But now you've just argued yourself back to the "which human(s) do we hold accountable for these decisions?" question you raised that I was trying to get you out of.