Hacker News Reader

Comment by fxtentacle | Hacker News Reader

Reading this, I'm so glad that in all of the EU it is illegal to fire someone purely based on a software decision. There's a minimum level of humanity that we should extend to everyone. Letting a known faulty AI decide if someone will have money for food tomorrow is way across the line.

I'm not surprised that treating humans like cattle on their way to slaughter is hugely profitable. But I'm not sure we want to operate large companies like WW2 prisoner camps. Even if you truly don't care about the poor souls who have to work there, the people who turn insane from this treatment will still live in your country and maybe their kids will meet your kids in school and "educate" them on the cruelty of life.

It's best not to push anyone down the mental abyss in the first place. And yet, here we (software developers) are, developing new monitoring and automation systems and deciding to delete someone's entire online life (Google account) without any oversight (AI) in the name of "efficiency" (higher profits for shareholders that we never met and never will).

View on HN

>Letting a known faulty AI decide if someone will have money for food tomorrow is way across the line.

So is the implication here that humans are less faulty, or that there's something intrinsically wrong about an AI making this decision rather than a human?

If the former, then this is an issue that the technology will probably eventually solve. If the latter, well... that doesn't seem very justifiable, to say the least.

If Amazon has leaned too far in the direction of automation without oversight, then we'll see other firms fail to adopt these same practices. But it seems pretty inevitable to me that we'll see many more AI-driven hiring/firing decisions in 10 years or so than fewer. And if some sclerotic regulations try to slow this down, that will likely be unfortunate for consumers.

No, the implication is that we believe that these algorithms are non-faulty and therefore throw away all the provisions for recourse and review that exist with human-driven systems.

There is no nuance, no leeway, and no way for you to reason your case with a thinking human on the other side of the decision. The algorithms & systems are also almost always completely opaque in their workings, with neither the person on the receiving end nor the company's staff having any insight into why they made decisions or what data was used.

Using algorithms for life-affecting decisions like this would not be nearly as bad if you had a reasonable ability to appeal and have the decision be taken as final.

It doesn't even have to be AI, or an algorithm. The British Post Office Horizon scandal is an example of lives ruined by an assumption that accounting software could never be wrong.

>No, the implication is that we believe that these algorithms are non-faulty

I think that falls under OC's argument and it seems eventually solvable. Still, what you are pointing is a human bias. Humans have been taking similar unfair decisions without possibility for recourse or review since forever, so it would be interesting to have a review mechanism in general.

>I think that falls under OC's argument and it seems eventually solvable.

Not solvable by purely technical means and the idea that the AI or algorithms will just keep getting better, which is what the OP seemed to be saying.

> Still, what you are pointing is a human bias. Humans have been taking similar unfair decisions without possibility for recourse or review since forever, so it would be interesting to have a review mechanism in general.

Sure. Having systems be human-driven doesn't inherently make them better. But over time the recognition of human fallibility has led legislators to enforce the creation of systems of review and recourse in many cases.

> So is the implication here that humans are less faulty

AI is just humans hiding behind a machine. And everything else being the same, those who have to hide their reasons or themselves are always intrinsically weaker and worth less than those who don't. That needs no explanation for people who own their shit, and no explanation will suffice for those who do.

> that will likely be unfortunate for consumers.

Consumers aren't just consumers. They have jobs, they have friends and family who have jobs.

> AI is just humans hiding behind a machine.

Just because people work on AI algorithms doesn't mean that this is true.

When an AI "decides" that someone should have no food tomorrow, there's always a human who set the AI up to do that, and there's always humans who honor this "decision", often while pretending that they have no choice, that they're mere messengers. No exceptions.

That's a lot of "always"s for a situation you're making up that hasn't happened. I wasn't aware there are AIs taking food out of people's kitchens!

Look at the context of this thread then and the comments upstream of mine. I was responding to someone who defended that, that ship sailed, you don't get to make this into something else now. It would be no use anyway, what I said doesn't just apply to that, it applies to any usage of AI by humans to make decisions on their behalf.

That may be the implication, but the actual problem is that humans will still be involved, still making the decisions and using AI to pretend they didn't.

I actually feel quite comfortable with the idea of AI making the hiring decisions, it seems like it would be as good as a human. But make no mistake, firing people is a miserable thing to do for both sides of the conversation. If managers can palm the responsibility off as "well, gee, the AI told me to so ... y'know, my hands are tied" then they will. Even if the AI algorithm is as blatant as "print out the name the manager entered in to the system".

There is some reasonableness to the argument that when an important decision is made that some human must be empowered as the decision maker. If nobody is responsible, suspicious things can happen.

>So is the implication here that humans are less faulty, or that there's something intrinsically wrong about an AI making this decision rather than a human?

Many people in the world have noticed the difficulty in getting people to change their minds once made up.

>And if some sclerotic regulations try to slow this down, that will likely be unfortunate for consumers.

Maybe not the consumers that have jobs though.

How unfortunate would it really be for customers? The worst I can think of is Amazon Basics cables being six eurocents pricier per foot.

> If Amazon has leaned too far in the direction of automation without oversight, then we'll see other firms fail to adopt these same practices.

By "leaned too far" I feel like you mean they relied on it in a way that made them less profitable than if they had not, in which case I agree. But I think most of us are worried that they leaned too far in a way that ruins peoples lives unfairly, we are more worried about that than Amazons bottom line. As a consumer I'm totally fine with paying 10% more for my coffee machine or whatever if ending such practices prevents even a single person from having their life upended.

Consumers aren’t the only people for whom laws and policies are optimised for. It’s fine to have laws which are unfortunate for consumers and great for the society.

I kinda agree. We also have the mounting pressure for getting unconscious bias out of the hiring process. It would seem if we think people are fired also for bias and unfair reasons that the ai would be the likely only way to save the situation.

Seems here with amazon we are quick to suggest humans are the best way but in nearly all other inclusion/diversity posts we are fighting against the humans being faulty.

Interesting problem for sure

> if we think people are fired also for bias and unfair reasons that the ai would be the likely only way to save the situation.

This assumes that bias and unfair reasons aren't codified or part of the training set, which we've seen previously, simply isn't true.

There's an example of a company using AI to do a first pass on CVs for recruiting throw away female CVs. Another example of a company using AI to identify people in photos identify certain skin tones as being animals instead.

AI isn't a magic bullet, simply because it's still coded by humans who make mistakes, conscious or otherwise.

Or to put it another way, garbage in, garbage out.

But a lot of fateful decisions are already made by computers. Decisions about granting a loan, an entry visa or insurance policy are largely automated today. If there are humans in the chain, they are mostly there to type data into the computer and don't have any decision power.

This comment is an excellent example of "technological manifest destiny"

https://www.youtube.com/watch?v=ajGX7odA87k&t=36m46s

> If Amazon has leaned too far in the direction of automation without oversight, then we'll see other firms fail to adopt these same practices.

What utter nonsense. What will happen in reality: business "thought leaders" of all types will say "Amazon does it, Amazon is successful, therefore we should do it" regardless of whether it's actually a good idea. And they'll be praised by their peers for doing so.

Just like Jack Welch's leadership philosophy (and stack ranking) in the 2000s and Google style interviews for every low-level tech job in the 2010s.

People have historically complained a lot about firings not being based on an objective formula. Now the complaint is an objective formula is being used.

> WW2 prisoner camps

For starters, those camps would shoot anyone who tried to leave. Not the same thing at all.

> People have historically complained a lot about firings not being based on an objective formula. Now the complaint is an objective formula is being used.

Cute line, but obviously people have always wanted criteria to be objective and fair.

If the rule was "take the ordinal value of each letter in employee's name, sum mod 365, and if the result matches the day of the year, they're fired!" -- that's completely objective, and also bonkers. Nobody ever thought all objective formulas were good, and it's sophistry to suggest otherwise.

>> objective and fair.

You do understand that the is a logical impossibility, as "fairness" is a subjective standard...

What is "fair" to me, likely will not be what you believe is "fair" thus something can not be objective and "fair"

> "take the ordinal value of each letter in employee's name, sum mod 365, and if the result matches the day of the year, they're fired!"

My Objection to this standard would not be "it is unfair" but rather it makes no logical sense to have employment choices made by that system and would likely cause many objectively valuable employees to be separated from the company.

Objective rules need to make sense in the context for which they are apply, "fairness" would not come into play here

The notion of fairness is contestable, but that does not mean we cannot progress past Perry’s second stage of development [1] (“it’s all subjective”). Fairness can be equated with due process or natural justice, which in a workplace relations context, commonly includes the right to be heard, have your personal circumstances taken into account and be given reasons for decision, and the right of appeal. Automated decision-making threatens these rights.

[1] https://www.jmu.edu/geology/_files/willperry21.pdf

When you introduce the notion of "logical sense", you're already deviating from an objective standard. For example, Amazon's warehouse bathroom policy is logical from a profit-maximizing perspective, but it's illogical from the perspective of the health and wellbeing of its workers.

I'd argue that if a process does not have room for you to explain yourself, and ensure that ALL data is taken into consideration by the other side then such process is authoritarian and not objective. An algorithm only looks at data it was programmed to look at, which is usually too narrow to be able to be without faults in the real world.

There are good reasons why judicial systems have higher instances in them to which you can appeal to if you don't agree with lower instance.

>An algorithm only looks at data it was programmed to look at

Programmed, or even worse trained, which means there’s no general intelligence around that really understands why it does what it does.

Just because there's an algorithm involved doesn't make it objective in any way.

There is no objective answer to "How productive is this employee", it's not something you can measure empirically.

I'd take it a step further and ask if we really want such an extreme in utilitarian ideals guiding such fundamental human needs like employment and its associated income that serves as a generic resource pool for food, housing, healthcare?

I don't know about most people but objectivity can fly out the window when it comes to my essential needs. I want another group of humans involved who have similar needs and may consider things an objective metric likely won't.

Some mixture of a utilitarian performance metric balanced with some empathy, morale, and ethical reasoning seems like a better blend to me. Either extreme leads to undesired outcomes in my opinion.

With pure human involvement you can have extremely biased no justifiable firings based on emotion, hatred, racism, sexism, ageism, whatever and in those cases, a metric is your friend. It can potentially show that undesired bias for what it is to others.

On the other hand, that bias may be a good bias. Perhaps your manager is a reasonable human being, they know data the algorithm doesn't like how many additional hours you put in unclocked or that you helped your coworker through a rough problem a few months back, perhaps that you saved the company at a critical junction. They also know your performance recently dropped because there's a pandemic and subequent emotional distress might be effecting you. Perhaps they also notice your performance dropped just like the metric identified, but they know the cause of this is because you just became a parent and have had to divert focus and time towards your family. Your manager has the ability here to step in and override the metric because they know better, have empathy, and understand the world isn't about constantly working at maximal efficiency. This is, to me, a desirable bias.

If we keep chasing and supporting utilitarianism pushed by corporations full-steam, we're going to create quite the hellscape that no one wants to live in.

Also, if you purely go by productivity, who knows, maybe the data says that people without kids and people who are sterile are just better according to the metric.

While I think you can measure 'how productive is this employee' according to some metrics, the problem is in the metrics. Someone who is mediocre at his job in the quantified sense can be a huge boon to the company due to soft skills or knowledge.

Someone not living for their job could be important to an entire community.

It is a philosophical problem, in addition to finding any kind of metric that actually kinda works.

> For starters, those camps would shoot anyone who tried to leave. Not the same thing at all.

It is an exaggeration, but with the health & social care systems that the US has it isn't a million miles away either.

No, it’s still a pretty stupid comparison. Like so stupid it’s borderline offensive.

Rather, the complaint is that the supposedly "objective" formula is completely obscured to both the victim and the executioner. A judge isn't called "objective" for deciding the outcome of a trial as a black-box call. If it were, we'd have replaced the justice system with the Vegas system long ago. A judge is called "objective" because they clearly articulate the inputs, scope, weightings and broader implications they considered to arrive at their outputs.

>>A judge is called "objective" because they clearly articulate the inputs, scope, weightings and broader implications they considered to arrive at their outputs.

Even a judge has limits on comprehensiveness and objectivity. No human judge has perfect knowledge of every law in the United States. When making law, judges choose to uphold stare decisis in most cases but overturn decisions in others because of a certain gravitation towards certain interpretations that largely remain unarticulated. The latter is not done out of some "special" knowledge unobtainable by AI, but by the very subjectivity an AI is meant to rule out in its own considerations.

In contrast, an AI judge could provide an exact numerical weighting and ranking to the sources it cited or present the conditions under which there would be contradictions between one law and another if one were to rule in a certain way. AI doesn't have to be a black box. If done right, it can be most transparent and revealing legal tool to have ever been created.

People have historically complained a lot about firings not being based on an objective formula. Now the complaint is an objective formula is being used.

If these same people got what they wanted, then realised it didn’t turn out the way they envisioned, and had worse outcomes, then used the new information to change their minds, then I don’t see what the problem is.

My main issue with it is that we don't have complete data to feed into the formula to make a good decision. Some aspects of your job can't be quantified and entered into an algorithm.

It's the same as when project managers track how many commits per week each employee makes and use that as a metric for how much work they're doing - an exceptionally dumb example but it literally happens at a lot of companies. And for an algorithm such as this, I doubt they have much better/more useful data to feed it.

Humans have their own set of issues as well though, as you've mentioned. I feel that the problem stems from the way companies are organized with a single entity having full control over resources and decision making, as opposed to responsibility, resources, and power being divided among the people doing the work, in a self-organized and likely democratic way.

>based on an objective formula Who knows if it's objective, you can easily put any bias into formula. Remember Amazon's application AI that downgraded applications of women just because they were women.

> Now the complaint is an objective formula is being used.

As programmers we really should know better than blindly trusting software.

I'm pretty sure not all HN readers are programmers...

I'm quite sure that Walter Bright is though ;)

I think the point is that to the AI, an employee is just a number. Interchangeable at any moment, with no consideration for the human person and how it affects them.

So if you have 2 Programmers, One is Single has FIRE plan to early retirement, and expense to income ratio of 25%..

Another programmer has 4 kids, lives pay check to paycheck, and has a wife with a medical condition...

The dept has to be downsized by 1 programmer, both programmers have the same income, the single programmer is slightly more productive..

Should the single programmer should be fired over the other because of there personal life situation?

Just be sure people complained about a formula being objective, it doesn’t mean any objective formula should we accepted.

"the software said so" is 2020's way to denying something to someone while also trying to deflect blame.

Not very different from "not financially viable", "your health insurance have expired", "your credit score does not meet the requirements".

The decision of implementing such rules and methods and use such phrasing is entirely done by humans.

I was a bit too ambitious/hard working, and a coworker made it her mission to get rid of me.

After 2 years she succeeded.

I admittedly didn't care because I got a higher paying job in a better field, but I'll take AI over office politics.

Yes - but this isn't an either/or situation...

Why the AI sidesteps responsibility? It's like stabbing someone with a screwdriver and saying screwdriver did it! Are regulators in the US that dumb or corrupt to allow it?

For not working and being their own self they are not hired.The lower level workers were not adopting the e-commerce discipline of promptness.We require JBezos like monitoring and action.

> like WW2 prisoner camps

FWIW I think your comment would've been more powerful without the Godwin.

> Reading this, I'm so glad that in all of the EU it is illegal to fire someone purely based on a software decision.

Why do people treat working for a company as something different from selling any other services or goods? If there was a law that limited your ability to cancel a gym membership or going to a different supermarket, you would rightfully object — it's not a government's business to get involved in your voluntary business transactions. Why treat a company that hires workers differently?

Because a company has tens, hundreds or thousands of employees.

Employees generally have 1 job (at a time) and 1 career.

It's really hard for an employee to screw over a company for good and trivial for companies to completely destroy an employee's career and life.

The asymmetry needs to be counterbalanced actively, especially since companies can have people on their payroll 24/7 to fight for more company rights. Employees can't afford that.

The way it is commonly seen is that workers deserve special protection through the law to make up for the inherent power imbalance between those who employ and those who are employed.

I heard this reasoning a lot. I understand logic and intent behind it — certainly a noble one. But I think that it is incorrect because it ignores unintended consequences, like the story with award for catching cobras in India.

You're shifting the power balance in an artificial way that doesn't fully change it. You're only shifting it in one particular market; this power balance still exists in other countries. Which creates a potential for arbitrage: instead of hiring people in a country which enacts those measures, hire them elsewhere.

You're also not exactly making the employee "more powerful"; you're actually limiting his freedom on the market. You're making his services more expensive to a buyer, and he can do nothing about it. He might have wanted to offer his services at a discount to get an edge above competitors who have more experience, for example; now he's unable to.

Do you really think that limiting someone's freedom is empowering?

That's how you get 2x salaries for engineers in US as compared to Europe, which leads to many brilliant engineers leaving Europe for US (or working remotely), which, with higher commitment requirement, contributes to the difference in startup scene between Europe and US. Of course, engineers are not the type of workers whom do you think about in such a conversation, but this happens in many other industries as well.

This is often repeated, but I don't understand it. What power imbalance? The ability to offer a job is only a power over a worker if the worker has no other options for jobs, which is only the case in a company town or something like that. Then the company gets to decide if the worker starves or not.

The labor market in the US is highly liquid, even for unskilled workers. This described power imbalance simply does not exist.

>The labor market in the US is highly liquid, even for unskilled workers. This described power imbalance simply does not exist.

Yes it does. See Amazon preventing workers from unionising.

I'm not familiar enough with the situation in the US to comment for what's happening on your side of the ocean.

In my country, there are laws to protect both sides. Unless an agreement is negotiated, the employee cannot leave without giving a ~2 months notice, the employer cannot fire/layoff withing giving a ~2 months notice.

In many cases, though, there is a strong power imbalance. If, for some reason, you decide to sue your (former) employer, or if your (former) employer decides to sue you, you most likely cannot afford the same budget for lawyering up as they can. Also, your main power against your employer (besides possibly unions) is, as you mention, the ability to change job, but that only works when changing job is possible. Both for highly skilled positions and unskilled positions, getting a new job is hard and may require you to move to a new place - something that you may not be able to do if you have children or a working spouse.

As anecdotal examples, I know former C*Os of influential companies who have been working make up jobs for several years because they cannot find anything matching their skills, even at much reduced salaries. I know burnt out teachers who do not leave their jobs because nothing matches their skills. I have several friends in IT, or in admin positions, or with lawyer degrees who just cannot find anything or could not find anything for years because they happen to live in the wrong place and cannot leave because that would cost their spouse a job or they would abandon elders, etc.

Having myself changed (HN-style) jobs a few times, I can confirm that it's long and certainly not easy, in particular when you also need, at the same time, to care for a young child, or a sick family member, or when you're burnt out, or depressed by a personal crisis, or by the rejections you encounter from potential employers, or by the institutional bullying on unemployed people. I imagine it's even harder as you get older - I'm not looking forward to be labelled too-old-for-HN-style-jobs.

All of that is in a G7 country whose unemployment rate is decreasing. Testimonies from friends in neighbouring countries make me feel that it's even worse in many places. Testimonies from friends in the US make me feel it's not as rosy as you describe, at least not in all fields/regions.

So... yeah, I believe that there is a power imbalance. You may be in a field and/or location in which you do not witness it, and if this is your case, I'm happy for you. But for many people, it exists.

This:

> the employer cannot fire/layoff withing giving a ~2 months notice.

Leads to this:

> Both for highly skilled positions and unskilled positions, getting a new job is hard

That is a theory I've heard. I see how it may be true but I'm not convinced.

In particular

> Testimonies from friends in the US make me feel it's not as rosy as you describe, at least not in all fields/regions.

I personally don't care, but for all the people living from paycheck to paycheck in a low wage job this is the bare minimum of security they would want from their job.

It's important to note that the company gets help from the state shouldn't they be able to pay their workers. So it's not only enforced on the employers but backed by the state

What a weird question. I could similarly ask, why do people treat employing workers as something different from animal husbandry? If there was a law that limited your ability to ship, cage or breed your workers like you do cattle, you would rightfully object -- it's not a government's business to get involved in your voluntary business transactions. Why treat a company that employs workers differently from a company that employs cattle?

Your question has as much relevance to the question at hand as mine.

To be fair, livestock do not consent to being livestock.

Because humans deserve the right to freedom to accept or deny any voluntary contract. Which legislation that we're talking about is directly infringing on. Just as people should be able to not accept having being treated like cattle, they should be able to accept any form of employment contract if they so choose: it is exactly the same issue.

In many EU jurisdictions being employed means you owe the employer the willingness to accomplish a result not the actual result.

Germany is known that once you are in an active work relationship its difficult to get fired.

So yes, there is a difference as it is generally frowned upon these days to call people a resource which implies an interchangeable unit of work force.

You can't compare selling goods to employing people in that way. Consider the fact that there are humans on both sides. Businesses can use their power to abuse employees in a way that can't be done to goods. If you are the main source of income for a person, you have massive influence over them. Government creates laws to balance the power. There are countless cases of abuse by employers including physical and sexual. Very different from selling goods or having a gym membership.

Your limited ability to cancel a gym membership is an important part of any gym's business model, at least where I live: it bets on people's impulse decision (say, at the beginning of a new year) to get in shape, a desire which often tapers off after a few weeks or months. But if your gym contract has a minimum duration of, say, two years, whether you go there or not, you pay.

A gym deciding to cancel someones membership is a minor nuisance for them, a company deciding to fire someone has potentially life-changing consequences.

Remember that for you as - I am assuming - an IT professional getting a new job is easy, for lots of professions not so much.

First of all, have an upvote, which is both the agree and disagree button, as long as a person contributes to the conversation.

Next, I think you are contributing, because companies really, truly operate as if people were flesh robots and your perspective, rather than being trolling, is actually dominant in corporations.

Thirdly, that you could potentially have accepted the narrative that we are fleshbots, ready for exploitation, no different than any other raw material and not needing any more care or consideration than cattle or cotton says something deeply disturbing things about the state of humanism.

> First of all, have an upvote, which is both the agree and disagree button, as long as a person contributes to the conversation.

It's a shame that HN makes no difference between "what you're writing has no place here" and "I disagree with you" I remember discussing with people who were experimenting with this kind of mechanism for comments on newspaper websites, but I believe that this hasn't had much success.

Humans are humans. Companies are not humans.

Probably because people's livelihoods depend on employment and they have much less power on the markets as individuals than companies.

Reply on news.ycombinator.com