Amazon's warehouse worker tracking system can automatically fire people | Hacker News Reader

Amazon's warehouse worker tracking system can automatically fire people

businessinsider.com

129 pointsvezycash133 comments

Worked in a warehouse for a german online retailer as a picker a few years back - tracking everything you do is totally normal - people worked through breaks to reach their limit, too and the break is a joke because it's a huge hall and more than 50% of the time is spend reaching the break-area and to go back.

Someone at Amazon basically throught why does a human have to look at the statistics after a few months and replaced it with some code - but besides that, that tracking and pressure is probably common in almost all online retailer warehouses. Maybe some cynic asshole thought that not letting the team-lead to do firing prevents keeping employees that perform bad but have good reasons for it - like beeing older or having medical problems.

Why ? What's the big deal if a package costs a few more cents to fulfill?

Is it a competitive disadvantage - or just standard greed ?

It is a competitive disadvantage. That's how Walmart got to where they are, they cared about a "few more cents" for each item, which adds up in high volume low margin businesses, and crushed everyone in their market.

Ever day low cost is the motto

To me, nothing. On a macro scale? Everything.

Exactly. One package that costs 5 cents more? 10? 100? Not a big deal.

50 millions packages that cost 5 cents more? Kind of big deal.

Amazon's sales are in the billions of items annually [1]

If you can save two cents on each of 5 billion packages, you save $100 million per year. Most employers will give raises and promotions to people who deliver operational expense savings that large.

[1] https://www.sec.gov/Archives/edgar/data/1018724/000119312518... Five billion items to prime users

The problem with this is the savings are usually negligible compared to spendings of higher management.

I think it utterly destroys the image of the company if it cannot act "humane".

It's fine to keep costs under control ..but..if you are then organizing super expensive meetings across the world it just looks bad.

> Why ? What's the big deal if a package costs a few more cents to fulfill?

When you shop online, do you choose the most expensive purchase option? Or perhaps the least?

>Someone at Amazon basically throught why does a human have to look at the statistics after a few months and replaced it with some code

I wonder too if its also someone who wanted to avoid the confrontation of having to make the decision. "We've noticed you've been under-performing and you haven't been meeting your quotas for the past x weeks" vs "The tracking system has determined that you have been under performing and so we have to let you go." It may seem close to the same, but its always easier to say it was someone else or something that decided instead of you.

> An Amazon spokesperson confirmed to Business Insider, "Approximately 300 employees turned over in Baltimore related to productivity in this timeframe.

Why do they talk like that? I think they're erasing doubt that this is a corporatist nightmare place to work.

Yet another instance of Newspeak. It's quite common nowadays in the real post-1984 world.

The good news is people can recognize it for what it is.

Because the sort of person who thinks it's okay to rely on an algorithm to fire people doesn't care about their employees, but cares a lot about not getting called out for it. They think they're clever but I'll be damned if I'm ever going to work for a company like that.

You should always be required to a look a person in the eye when you fire them. There are way too many managers who will do anything to avoid uncomfortable situations (for them).

So we end up with things like this...

"Computer says you're fired" is the new "Computer says no."

And the best is when the cowards send an SMS instead of just saying it face to face..

Or by robo-call as in The New Deal's "Exciting New Direction" - https://www.youtube.com/watch?v=Z0pmNDIXymc&t=2m37

It’s much easier as a company to introduce a system to automatically fire people than it is to teach your managers how to fire people without taking their own comfort into account (e.g. not firing because it’s a pain).

Also this is an easy way to claim that people were fired fairly and they weren't picked out because the manager didn't like them or was racist.

It's a warehouse. We life in the 21th century. In theory if we would sit together and structure the existing workforce without all the bullshit overhead of accounting, bullshit jobs and priorities life quality and sustainability, no one would need to work and we could achieve this in the next 5, 10, 50 years.

I get that amazon should pay there workforce properly and as long as we have this, it should be something people can and should do in a social way but Baltimore is a US City, they don't even have a working health care system.

A warehouse job is a warehouse job. No education needed, why wouldn't a computer system should not fire people on simple productivity numbers?

I guess it depends on how great you are at guessing what to optimize for. If you make a machine that just rewards completing tasks, maybe you'll get poor quality. If you add quality to the mix, maybe you'll fire everyone who has experience and you'll be left with only those yet to make a mistake.

> A warehouse job is a warehouse job. No education needed, why wouldn't a computer system should not fire people on simple productivity numbers?

Increasingly, this viewpoint bothers me. To be honest I don't know the ins and outs of working in an Amazon warehouse but I suspect it's not as brainless a job as you imply.

Further, why does that attribute make this ok? Why wouldn't it similarly be ok if the computer could fire people automatically if a threshold of unit tests failed or if a developer failed to meet LOC quotas?

Because there's more metrics to programming than LOC and if a company treats it like the only metric then they deserve to go under. Working in a warehouse does not have multiple dimensions to it. You are paid to pick up things, put them in boxes and then pick up more things. That's it. It's easy to measure.

For comparison sake, I know someone who runs a company sorting refuse. Each person in the warehouse needs to sort through 500kg of refuse every day. That's it. Literally nothing else matters. It's not a complex job, so the metric is not complex either.

Forget the LOC example. Let's just say failing tests. Pick any single thing you think is critical to software development. Regardless of what it is, would it be acceptable to you to have an automated process simply fire someone for not hitting this metric?

I feel like at some point we have to draw the line. We're going to have to coexist with automation and eventually it will get to software development, too. Some parameters about what we consider to be reasonable and unreasonable to automate make a lot of sense to me.

I also feel like we get very classist as software developers and fall into the I-am-special trap frequently.

We have drawn the line. Right now it’s at Amazon warehouse workers.

Also there is a critical thing to software development: Working for a business that makes money. If you don’t hit that metric, you’ll eventually be out of a job.

Plus, there are many more people who can pick up boxes than there are who can program.

We haven't draw any line at all, it's simply where it's been pushed so far.

How does that differ from any other situation? Something is acceptable until it isn't...

That's what I'm saying. We haven't drawn any line here, we're just acquiescing at every step.

Then nobody ever draws a line. We just acquiesce at every step.

I have drawn a personal line a long time ago. I do not use amazon at all. No PayPal, Hollywood, local restaurants without union contract for the employees and so on. The union have a search app to keep tab on what restaurants are ok. (Sweden only though). We don't have much laws for companies, the unions have to agree with the company organisations for a lot of the rules.

Yeah but a metric is a metric is a metric. That there are more metrics in one area than another is arbitrary.

You could just as easily say: all right LOC is the only thing that matters, we're not paying you to be right, we're paying you to write code so do it. And the warehouse could have thousands of metrics to measure the intricacies of working in a warehouse. Following that your argument easily applies to programmers as well. They deserve to be treated like crap.

That you view working in a warehouse as so simple,and that you talk about "deserving" so much reveals a lot. You're using metrics as an abstraction to hide behind so you can really say they "deserve" to be treated this way because they work in a warehouse.

But, really, whether someone deserves some treatment or not because of the job they work should be irrelevant.

My wife says programmers just press buttons and look at a glowing rectangle all day. I think you've just done the equivalent.

Having worked both in software development and at a warehouse, I can vouch that both have multiple dimensions.

Oddly enough, both jobs involve sorting algorithms, and both can be completely derailed by poor management.

Let me work in a warehouse. What do you think how long does my initial training will take to be a picker at an amazon warehouse?

I will tell you: between 2 days and 6 weeks.

"shrinking its warehouse training time from a conventional six weeks to as little as two days."

How long do you think i have to train your wife to setup a kubernetes cluster, make sure it is secure, backuped and works efficient? Then when she knows that, how to build images, and write code in java, go, bash, ansible etc.?

If the answer is longer than 6 weeks, you see the difference. And don't get me wrong, i have seen people who had a 3 year education and start as juniors and miss a lot for years until they get it all right.

It would be similarly ok, but not nearly as effective, I assume. Which is probably why nobody does it.

I worked in a warehouse and it is as mindless as you can imagine. You receive a slip, pick things on slip, place in box, tape close and repeat.

I actually would love to see real statistics about my job performance.

I'm doubting myself how good i am. I don't think its very objective how i or people around me earn money.

But yes i don't look down to people working in a warehouse but i don't think that it takes too much knowledge. That is the reason why so many more people can do it.

I have seen people struggling with my 'simple' tasks continuously and they are in the same field as i'm. There has to be a difference between software engineering and warehousing.

I would prefer to have a proper social system and a future were we don't work because of work. Therefore this is an discrepancy in my world view: You should not need them and they should have a good life. But it is much more efficient and logical to have a computer system to fire people.

> A warehouse job is a warehouse job. No education needed, why wouldn't a computer system should not fire people on simple productivity numbers?

Because they're human beings.

How'd you like to be treated in such a dehumanizing way?

Why are you picking on accounting here? There's nothing more vital to the enterprise than understanding how its money moves. It's not easy, and if it doesn't happen nobody gets paid.

To derail on the sentence: "Approximately" so later they can say that it was actually 200 or 400. "related to productivity" hints that they turned over some more but not related to productivity as to lower the precision of this number even more. "in this timeframe" put all this in a timeframe said by someone else.

Every part of the sentence works hard to make the number 300 more imprecise. I think he talks like this because he has no idea about the actual number.

Those are just datapoints, whats the big deal? It only becomes real when finally touching you directly, as some journalist learned recently #LearnToCode.

I don't think there is a huge ethical problem with automatically terminating employees based on metrics. Quite the opposite, it could make this process more fair.

The problem is in how the metrics are used and there is no real difference between an ill-meaning profit-hungry manager and an ill-meaning profit-hungry algorithm.

Laying off employees for bathroom breaks or short (days, few weeks) fluctuations in productivity is wrong. But it's wrong if an algorithm does it or a human does it. Algorithms could be used to prevent firing an employee in such a situation.

The psychology of it is fucked. It is so impersonal. Are you even notified that you've been terminated or you show up and your badge doesn't work. No explanation provided, just algo129 determined poor metrics. Sucks to work there not knowing what the blackbox is watching or when your time will come.

    > ... Algorithms could be used to prevent firing an employee in such a situation.

I think you're being _very_ optimistic and a bit naive about the motivations of a business that would resort to such algorithms.

They want to maximize the throughput rate of picking boxes and minimize their cost. Sure, it sounds "objective" to say that everyone who doesn't meet the pre-agreed work criteria would be fired (and not-fired if they meet it).

But who gets to set that threshold? Who determines what is reasonable? What if that performance threshold can only be met by fit young people and only until their bodies can't take the stress anymore? How will workers even assess whether or not they can do the job until they try?

This is an ugly practice. If it survives, it will be coming to a cubicle near you someday. Hope I am retired by then.

>But who gets to set that threshold? Who determines what is reasonable? What if that performance threshold can only be met by fit young people and only until their bodies can't take the stress anymore? How will workers even assess whether or not they can do the job until they try?

If it's harmful to the worker, then it should fall under the purview of OSHA in the US.

Otherwise, that threshold is set by consumers who choose to patronize businesses that treat their employees in a certain way. Almost all of the time, for goods that can't be distinguished from one another, consumers will choose to patronize a business offering the lowest price available to them, hence the businesses that squeeze the most out of the workers survive and the others do not.

   > ...then it should fall under the purview of OSHA in the US.

Will OSHA end up being an algorithm as well?

Sorry, but contrary to techno-libertarian fundamentalist beliefs, institutions and courts just can't keep up when things are so insanely lopsided in favor of the rich and powerful.

That said, nothing is going to stop amazon from instituting whatever crazy "expert system" they want to abuse warehouse workers until they're all robots.

All they've really done here is automate their management. It's the next logical step. If you can automate the pickers with robots, and the managers' jobs can be replicated by an algorithm, why not automate the managers too?

If Bezos could find a way to automate himself, I'm pretty sure he would. And why not? He still owns the stock. Robot Bezos would do all the work and meaty Bezos would make all the money.

Is the path to these awful decisions forged by human malice or is this just an organic byproduct of corporatism? Is it that nobody cares and are just hoping they don't get called out in the press, or is this stuff generally happening under the radar under the blanket of "efficiency?"

Also, what a pleasant mobile reading experience! https://imgur.com/GXiVkvE

Nitpick: I'm not sure corporatism means what you think it means.

https://en.m.wikipedia.org/wiki/Corporatism

I didn't read the article because by the time I'd clicked 4 different popovers of one sort or another I figured it wasn't worth it. Then I follow your link to Imgur, and get another quantcast modal. Aargh!

How do you use anything on the web without adblock? :S

On the bright side, this software replaces a human that used to fire people. That human has now been fired. :)

Only three days ago the NYT published "I Used to Work for Google. I Am a Conscientious Objector." and already we see that someone has coded a automated employee firing system. I assume the same culture that developed the idea persuaded the programmer to implement it.

Software is eating the world, but often not in the way we believe it to do.

https://www.nytimes.com/2019/04/23/opinion/google-privacy-ch...

It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.

As long as the check clears what is the problem with implementing stuff?

Well at some point that engineer should ask themselves why they're writing that procedure for a company called DestroyBaghdad Inc.

Money. It's useful to have more of it. Or just fun. Solving hard problems is rewarding.

There us subset of us that will play Ender's game even if we know beforehand it is not a game just for the thrill.

The dystopia is coming, you can only influence which kind and whether you will be near the top.

Because DestroyBaghdad Inc. pays well and offers lovely working conditions. Judging the legality of its business actions is a job for the state, not for me.

That’s how I almost talked myself into working for Palintir.

Just the name indicates perfectly well what you are going to be working on.

But really, it seems fairly benign compared to some of the other options.

It's also a trailing indicator.

Godwin point incoming

Are you being sarcastic or is this a genuine belief that you hold? I honestly can't tell.

As distasteful as it may sound, it is. Maybe you can accuse of not having morals but if a company's doing something so evil that I may object to it, the odds are that what they're doing is illegal in the first place so all I'd have to do is whistleblow confidentially and the problem sorts itself out. On the other hand if they're doing something very evil but legal, then the problem lies with the lack of regulation of it, as it's virtually guaranteed that some other company is also doing that so quitting a job over it would be ineffective, in that scenario I object by voting towards whichever party proposes the regulation I deem necessary.

Any behavior which is not illegal is tacitly acceptable. That's pretty much the basic tenant of a liberal society.

It's likely the same that happened at Amazon. The programmer probably didn't create a procedure to fire people, instead it forwards matching data sets to another system. The programmer of that other system implements procedures for terminating employments given a list (which could have enough ethical use cases). Putting both together results in an unethical system but I doubt everyone involved knew about that.

Why is it ethical to terminate bad employees when a human determines said employee is bad, but unethical if software makes the determination?

Ethnic bias would be eliminated in the latter case.

    If break[employee] > 1 minute
     employees[employee] = "bad employee"

Depending on what information you’re using, you can easily accidentally code ethnic or other biases into the system.

Amazon corporate offices have inter-faith prayer rooms and mothers rooms. I have a feeling they don't offer these basic amenities to hourly employees.

Given the fact that warehouse employees are pissing in gatorade bottles because they don't have time to use a bathroom I would suspect that participating in a religion that requires stopping work to prayer mid-shift or stopping to go breast feed would cause their quotas to drop.

So it's already pretty clear there's a huge bias towards only certain types of people who can even operate successfully in their warehouse as opposed to their corporate offices.

Although I guess it's all by design, reading through comments here or anywhere on HN about worker mistreatment outside of high earning tech salaries and you can see how little empathy exists among many people here. These are the people Amazon wants and needs working for them.

I'm not sure how you could have bias in a system that literally just measures packages moved per second.

Because a human would at least consider if a termination is warranted. Humans make mistakes but by my experience, software tends to produce wrong results more often than humans. Therefore, for all important decisions it makes sense to always have human making the ultimate decision and consider if there is no prospect of performance improving.

And even if the human doesn't do a better job, it's still the nicer way to be told by a person rather than a machine that you lost your job.

Its probably some brought in HR system

This type of monitoring always leads to employees missing breaks, bathroom visits and working off the clock.

Algorithms have no understanding of human limits and will push and push. Employees are scared to miss quotas since they will get fired and employers feel that they can push employees more since quotas are being met. At some point, it becomes a sweatshop where workers are strained to their limits.

We have seen this type of high-pressure work previously and it does not end well for both employers and employees. Employees revolt and employers have to concede to demands or face employee dissatisfaction which leads to societal disruption.

Companies think that they are improving their bottom line but in the long run, they disrupt society to the point of chaos for all.

This time may seem different because it's a high tech world but as long as humans are part of the equation we will see a similar result to previous situations. It's bad for everyone even if you are not the employer or employee.

So, Manna is finally becoming a reality: http://marshallbrain.com/manna1.htm

This reminds me of 'Manna', by Marshall Brain, a sci-fi short story in which an automated, AI-driven management system (Manna) gradually takes on more and more responsibility and authority over the world of humans in the workplace, leading to dystopian consequences. There's a lot more to it than that (I'm bad at blurbs) but it's highly worth a read.

http://marshallbrain.com/manna1.htm

Once again, fiction predicts reality..

or someone read it and thought "I like that idea"

Computers Don't Argue - Gordon R. Dickson

https://www.atariarchives.org/bcc2/showpage.php?page=133

I don’t see anything wrong with this.

Most of the warehouse operations will get automated soon. So it’s better (from the laborers’ perspective) to keep manual labor as efficient/profitable as possible, otherwise they’ll make it attractive to automate all manual labor.

i.e., those who are considered good warehouse workers, are at risk of losing their jobs too to automation if the not-so-good workers are not fired.

Just because robots exist does not imply humans should accept being treated like one in order to be maximally competitive. Societies get to collectively decide what the floor is for human treatment, not just market forces. (Hint- the market has no floor in this regard.)

(I upvoted your comment for making a salient point that I think has merit though I disagree with it.)

That would have nothing to do with Amazon choosing to automate a process. Society, i.e. government, has the power to decide what the floor is for human treatment. The people lucky to live in democracies can choose to vote for representatives that provide minimum amounts of paid leave, maximum amount of hours worked, etc.

A society could reasonably elect that firings must be reviewed and signed off by a human. In real life, there are instances of hard working people who through some bad luck end up having to do manual work into their 70’s. They haul ass and have a high work ethic, but can’t keep up with workers who are 50 years younger. They don’t meet quotas in terms of productivity, but usually management is happy to keep them on. Sure, you could automate compassion by formulating a metric that keeps these people, but let’s be honest, Amazon is not doing that. So a reasonable society might say that a human must be part of the process in order to offer “humane” relief for extenuating circumstances, or at least present an opportunity for that. It’s kind of fucked if Amazon realized they can save money by removing human judgement from the loop because humans are not universally in dispossession of a heart or compassionate facility.

Unfortunately governments plan in 5 year cycles, companies plans of Amazon scale are typically longer than that. I fear we will, and in some ways already have, reached a point where corporation actions influence government decision through public perception. PR is corporate propaganda, although it's currently inconsequential save for consumers being ripped off, in a democracy if you can convince your company is doing good and politicians are bad, then the role of government in regulating those companies is subject to people whose careers are being decided by the electorate that believe the words of a private entity more than a public one. Suddenly reform is off the cards.

In more recent events people keep saying these huge companies need to pay their fair share of taxes, yet politicians have made no headway with closing the mechanisms that allow them to avoid it, and if people lose their jobs because companies are suddenly forced to pay taxes they would have otherwise look to avoid paying, then the government will most likely be given the blame.

I kind of agree with your point.

The way I foresee it is, if extracting every bit of value from manual labor becomes costly, for a business, that means comparatively the automation option is becoming cheaper. Subsidies / tax breaks etc for the businesses can ensure that manual labor in warehouses is not so easily replaced by automation. But that can only work until a certain point.

Problem is not with existing set of automation solutions. It is with making the future solutions for automation more attractive.

in future, if a fully automated warehouse is possible, then no one (from the society) get's to decide how a warehouse worker should be treated, because they don't exist.

It's like the elevator operators in the past (they dont exist in most countries if not any). That's how I see the current set of warehouse workers (they can get automated out even before truck/car drivers).

Many of the comments here make me think of the interview I heard yesterday with David Graeber (anthropologist, author of Bullshit jobs).

He made the point that it seems always to be the people who do something useful who get the worst pay and conditions.

I have been a longterm dreamer to work for amazon AWS. But honestly speaking as much money as this company does, I am not sure I want to support this... as this is a tech forum with big AWS fans here, don’t you think that without separation of the blue-collar-cruching part and AWS there is a serious risk? Would you support this organization?

How is this any different to one of the many gig economy workforce companies like Uber, Lyft, Postmates, etc? How much human interaction do these employees have before they are fired? Do humans tell these people if they are performing badly and offer training to improve?

Its not like data aka algorithms has not been used for decades to decide this sort of thing.

I suspect IBM punch card systems have been used for this predating the First IBM computers.

Having the computer decide is new and could well cause more problems than having a low level manger do it..

For at least a couple of decades beforehand: https://en.wikipedia.org/wiki/Punched_card#IBM_80-column_pun...

Not sure if you’re aware of https://en.wikipedia.org/wiki/IBM_and_the_Holocaust

Surprised I've never heard this book or IBM's specific involvement with the Nazi regime. Strange how of the many times IBM has been mentioned this isn't one of the glaring facts that atleast one person brings up.

Its interesting that in WW2 NCR got the contract to build the 4 rotor Bommbes and Not IBM

Their subsidary - Watson Business Machines - was optimizing the logistics of transporting people to concentration, labour, and death camps with punchcard workshop straight next to the Warsaw Ghetto. The IBM's original "Watson project".

https://en.wikipedia.org/wiki/Dehomag#Holocaust

They terminate customer accounts in the same way for no reason so this certainly shouldn't be a surprise to anyone. Amazon is simply too big to care about people whether they be customers let alone employees.

Every month, Amazon’s warehouse system gets closer and closer to the one depicted in Manna [1].

1: https://marshallbrain.com/manna1.htm

It kind of reminds me of https://idiallo.com/blog/when-a-machine-fired-me

The real treat will be when they turn it on the managers.

The manger that decided to fire people was let go first.

This happened at Netflix. I don't know if it was automated or not but the person that came up with a firing method was fired by said method

https://www.fastcompany.com/3056662/she-created-netflixs-cul...

that's why i don't want to buy on amazon...

What a type of algorithm like this will never account for (and what it should really be used for) by design is nepotism.

Hah. Having crohn's disease I would be royally screwed. =/

Amazon is basically building "Manna" for their warehouses: http://marshallbrain.com/manna1.htm

At least there won't be any gender/race bias.

Not necessarily. If it fired people who listen to rap music while they worked, would it be racist? Some might argue— no— because it also fired white people who listen to rap. But it would be a rather obvious discriminatory practice. Now we must use our imaginations to consider less direct examples of the same phenomenon. For what it’s worth, I have no reason to believe that amazon is not an equal opportunity human soul extraction and disposal system.

It fired by the worker's output. Any racial bias in there is likely due to a deeper racial inequality (e.g. differences in health care access which can lead to worse performance) rather than the algorithm itself.

Algos tend to inherit the biases of their developers.

Let's say you're a woman on your period and that the bleeding, cramps and pain can get severe. You might need to visit the rest room or maybe take a few more pauses than the modeled average, that happens to be based on men, during certain days. The algorithm that is about to fire you doesn't factor in these issues when it decides to fire you for not meeting its performance quota.

The example you describe inherits the bias of the sample group, not the developer. A woman comparing individual performance with averages might notice monthly fluctuations rather than a man, but it doesn't mean they'd always recognize it or even look for it.

The algorithm should have average figures specific to all relevant factors: age, sex, race, nationality, immigration status, not a one-fits all approach. But it's a horrible idea in the first place.

Hahaha, wow -- full circle, this inadvertently arrives back at a racist system it seeks to counteract! For example, if it averages that hispanics are 1.6 times faster than the average worker at sorting boxes, does that mean it should fire a hispanic who is outperforming a white co-worker because adjusting for averages, that individual is 'under-performing'?

I get your point, but I’m going to do a “well actually” anyway....

The majority of people who buy rap are White.

https://genius.com/discussions/281920-The-majority-of-hip-ho...

Interesting as that may be, I'll raise you: discrimination is best measured as a deviation between a selected ratio and a population ratio. If you don't account for that, then silly things happen, like all heat maps turn into population maps, etc.

How does that say anything about people who listen to rap, as opposed about people who purchase music?

People are buying music and not listening to it? Right now, the top steaming artist is Drake.

The software will track only the work done. Sometimes I would really want to use a computer system rather to deal with people.

How about workers who need more toilet breaks one week in four?

We can always add more features.

Depends on who inputs to the system - a few years ago I overheard a UK amazon worker discussing with a mate how a lot of the supervisory posts had been captured by a particular group and they played favourites.

Similar to the problem Ford had at Dagenham which was BAME workers excluded from some higher paying jobs.

Until it turns out this is incredibly sexist/racist by the current climate and "reverse" bias gets added to the software.

Can a manager edit the statistics to account for tracking errors, special cases, etc?

If so, can a manager edit the statistics to account for employees of the wrong religion, sexual status, etc?

this sort of thing has to stop

tips on avoiding Amazon ?

This sounds like it would be illegal in Europe because it falls foul of "automated decision making".

Article 22 of the GDPR:

> The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.

Does the GDPR have anything to do with employment law? Seems unlikely.

Of course the GDPR applies to a company's employees. They don't stop being people just because they are also employed.

I am guessing that technically, since a human can override the system, a human chose not to. So the decision is not made solely on automated processing. This is purely semantic of course, but might get the off the hook (which tells a lot about the usefulness of those laws)

Is this legal? Can a computer have standing to terminate a contract? Can a computer enter in to a legal contract?

Edit: High frequency trading would seem to suggest it is? Is there nominally supposed to be a human in the loop?

Computers don't need standing, they act on their controllers' behalf.

There's a limit to that somewhere though? There's some minimum input that the controller needs to put in. If Outlook took it upon itself to send "you're hired" emails to all applicants for a job, that wouldn't be legally binding right?

I suspect the line isn't where I would like it to be in this situation though. Thinking about a generalised rule, requiring the counterparty be human if the original party is human? That seems reasonable and workable?

There doesn't need to be any rule, because computers simply don't make decisions that have legal relevance. Nothing a computer does is legally binding; things humans do using computers may be. Computers only facilitate things that humans do. If Outlook sends "you're hired" emails, it is legally binding if its controller intended to enter into a contract, and not if it was an error or accident. You don't have to consider the behavior of the computer at all when deciding legal ramifications.

"because computers simply don't make decisions that have legal relevance"

The computer is automatically firing people, that seems like legal relevance to me. It is making the decision (on some level) without human input.

High frequency traders buy and sell without human interaction. Is that not another example?

If there were an error, as with the Outlook example, there was absolutely no human input, so obviously there wasn't intent, therefore it isn't legally binding.

There is a continuum between a human doing everything, and a human doing nothing. At some point a legal line is crossed. On one side the computer is following a humans instructions, on the other it is working independently. Where is that legal line, what happens if a computer enters into a contract when working independently?

No computer runs without human input. My computer isn't firing people right now. Why? Because I didn't set it up to do so.

Utterly horrifying. No better than terrible slavery - while Bezos travels by private jet and enjoys his luxury super-yacht

http://superyachtmag.com/2017/08/06/456-foot-lurssen-superya...

I mean it’s certainly dehumanizing but... they’re not being whipped to death after working endlessly in the hot sun. I’m not saying it’s not awful working conditions, but the ability to leave alone makes it better than slavery.

You are right. My apologies - I should have said: "just a bit better than slavery".

No, you could have said "at least it's not slavery".

I empathize with your outrage, but you're being hyperbolic. I think you should avoid this because it's a good way of discrediting your position.

Reply on news.ycombinator.com