Hacker News Reader

Comment by yanderekko | Hacker News Reader

>Letting a known faulty AI decide if someone will have money for food tomorrow is way across the line.

So is the implication here that humans are less faulty, or that there's something intrinsically wrong about an AI making this decision rather than a human?

If the former, then this is an issue that the technology will probably eventually solve. If the latter, well... that doesn't seem very justifiable, to say the least.

If Amazon has leaned too far in the direction of automation without oversight, then we'll see other firms fail to adopt these same practices. But it seems pretty inevitable to me that we'll see many more AI-driven hiring/firing decisions in 10 years or so than fewer. And if some sclerotic regulations try to slow this down, that will likely be unfortunate for consumers.

View on HN

No, the implication is that we believe that these algorithms are non-faulty and therefore throw away all the provisions for recourse and review that exist with human-driven systems.

There is no nuance, no leeway, and no way for you to reason your case with a thinking human on the other side of the decision. The algorithms & systems are also almost always completely opaque in their workings, with neither the person on the receiving end nor the company's staff having any insight into why they made decisions or what data was used.

Using algorithms for life-affecting decisions like this would not be nearly as bad if you had a reasonable ability to appeal and have the decision be taken as final.

It doesn't even have to be AI, or an algorithm. The British Post Office Horizon scandal is an example of lives ruined by an assumption that accounting software could never be wrong.

>No, the implication is that we believe that these algorithms are non-faulty

I think that falls under OC's argument and it seems eventually solvable. Still, what you are pointing is a human bias. Humans have been taking similar unfair decisions without possibility for recourse or review since forever, so it would be interesting to have a review mechanism in general.

>I think that falls under OC's argument and it seems eventually solvable.

Not solvable by purely technical means and the idea that the AI or algorithms will just keep getting better, which is what the OP seemed to be saying.

> Still, what you are pointing is a human bias. Humans have been taking similar unfair decisions without possibility for recourse or review since forever, so it would be interesting to have a review mechanism in general.

Sure. Having systems be human-driven doesn't inherently make them better. But over time the recognition of human fallibility has led legislators to enforce the creation of systems of review and recourse in many cases.

> So is the implication here that humans are less faulty

AI is just humans hiding behind a machine. And everything else being the same, those who have to hide their reasons or themselves are always intrinsically weaker and worth less than those who don't. That needs no explanation for people who own their shit, and no explanation will suffice for those who do.

> that will likely be unfortunate for consumers.

Consumers aren't just consumers. They have jobs, they have friends and family who have jobs.

> AI is just humans hiding behind a machine.

Just because people work on AI algorithms doesn't mean that this is true.

When an AI "decides" that someone should have no food tomorrow, there's always a human who set the AI up to do that, and there's always humans who honor this "decision", often while pretending that they have no choice, that they're mere messengers. No exceptions.

That's a lot of "always"s for a situation you're making up that hasn't happened. I wasn't aware there are AIs taking food out of people's kitchens!

Look at the context of this thread then and the comments upstream of mine. I was responding to someone who defended that, that ship sailed, you don't get to make this into something else now. It would be no use anyway, what I said doesn't just apply to that, it applies to any usage of AI by humans to make decisions on their behalf.

That may be the implication, but the actual problem is that humans will still be involved, still making the decisions and using AI to pretend they didn't.

I actually feel quite comfortable with the idea of AI making the hiring decisions, it seems like it would be as good as a human. But make no mistake, firing people is a miserable thing to do for both sides of the conversation. If managers can palm the responsibility off as "well, gee, the AI told me to so ... y'know, my hands are tied" then they will. Even if the AI algorithm is as blatant as "print out the name the manager entered in to the system".

There is some reasonableness to the argument that when an important decision is made that some human must be empowered as the decision maker. If nobody is responsible, suspicious things can happen.

>So is the implication here that humans are less faulty, or that there's something intrinsically wrong about an AI making this decision rather than a human?

Many people in the world have noticed the difficulty in getting people to change their minds once made up.

>And if some sclerotic regulations try to slow this down, that will likely be unfortunate for consumers.

Maybe not the consumers that have jobs though.

How unfortunate would it really be for customers? The worst I can think of is Amazon Basics cables being six eurocents pricier per foot.

> If Amazon has leaned too far in the direction of automation without oversight, then we'll see other firms fail to adopt these same practices.

By "leaned too far" I feel like you mean they relied on it in a way that made them less profitable than if they had not, in which case I agree. But I think most of us are worried that they leaned too far in a way that ruins peoples lives unfairly, we are more worried about that than Amazons bottom line. As a consumer I'm totally fine with paying 10% more for my coffee machine or whatever if ending such practices prevents even a single person from having their life upended.

Consumers aren’t the only people for whom laws and policies are optimised for. It’s fine to have laws which are unfortunate for consumers and great for the society.

I kinda agree. We also have the mounting pressure for getting unconscious bias out of the hiring process. It would seem if we think people are fired also for bias and unfair reasons that the ai would be the likely only way to save the situation.

Seems here with amazon we are quick to suggest humans are the best way but in nearly all other inclusion/diversity posts we are fighting against the humans being faulty.

Interesting problem for sure

> if we think people are fired also for bias and unfair reasons that the ai would be the likely only way to save the situation.

This assumes that bias and unfair reasons aren't codified or part of the training set, which we've seen previously, simply isn't true.

There's an example of a company using AI to do a first pass on CVs for recruiting throw away female CVs. Another example of a company using AI to identify people in photos identify certain skin tones as being animals instead.

AI isn't a magic bullet, simply because it's still coded by humans who make mistakes, conscious or otherwise.

Or to put it another way, garbage in, garbage out.

But a lot of fateful decisions are already made by computers. Decisions about granting a loan, an entry visa or insurance policy are largely automated today. If there are humans in the chain, they are mostly there to type data into the computer and don't have any decision power.

This comment is an excellent example of "technological manifest destiny"

https://www.youtube.com/watch?v=ajGX7odA87k&t=36m46s

> If Amazon has leaned too far in the direction of automation without oversight, then we'll see other firms fail to adopt these same practices.

What utter nonsense. What will happen in reality: business "thought leaders" of all types will say "Amazon does it, Amazon is successful, therefore we should do it" regardless of whether it's actually a good idea. And they'll be praised by their peers for doing so.

Just like Jack Welch's leadership philosophy (and stack ranking) in the 2000s and Google style interviews for every low-level tech job in the 2010s.

Reply on news.ycombinator.com