One of the big complaints about modern society is all the isms. Sexism, racism, homophobia (ism). Computers don’t know your skin color, gender, religion, sexual preference, etc. Not saying it will solve all problems, but wouldn’t algorithmic decisions be preferable to biased human judgement?
e.g. getting fired because you couldn't fulfill your "quota" because some neighborhoods have their gates locked over the weekend.
Imagine getting fired for something you have no control over, or didn't even realize was getting penalized for.
This system is cruelty, pure and simple. Is it any wonder that Amazon execs are worried about running out of people to hire?
Because the algorithm just bases its decisions on performance, without respect to context, you don't hear any meaningful accusations of bias.
When you give supervisors agency to make decisions you get accusations (real or imagined) about various biased actions, or get accusations that the non-binary pregnant Pacific Islanders are under-represented, or do not represent some other constituency. It's bullshit 80% of the time and generates alot of wasted time, cost and bad PR.
It's a policy decision made because it is better for the company and isn't unlike how supervisors in industrial settings operate. If you've ever seen a very physical factory workplace (where the employees are at-will), they tend to eject broken people with chronic injuries as they age out. When I worked at a mall food vendor in high school in the 90s, it was routine for employees to be terminated for being 5 minutes late or too slow. The bigger lesson is go to school and avoid working in some shitty job where your welfare is a lower priority than the machines you serve.
That is life, that could always happen, the right thing is to accept the fact and plan accordingly.
If i took any kind of job I acknowledge that there always possibility that I'm going to be fired for something I have no control.
Also, even in the US, you can sue for wrongful termination depending on circumstances.
The reason of being fired is beside the point.
Employer should be able to fire anyone for any reason.
Likewise employee should be able to quit for any reason.
At this moment? Big NO. Algorithms
- are programmed by humans
- trained on past data (which by definition includes all the "isms")
- have a gajillion corner cases in which they perform hilariously bad
- don't understand the data they are processing so they can't reason something must be garbage in
How's that Amazon recommendation system working out for you compared to a human recommendation? That's the current state of the art algorithm on a very simple problem.
Is that necessarily so?
Can’t they measure more objective data points like deliveries per hour, missdeliveries, mishandled/broken items (controlled for traffic, pop density, data from deliveries to similar areas, etc) rather than other things like “doesn’t get along with boss:peers, has bad breath, does cat-calls, etc?
Now, I admit this has too much Taylorism in it. To much of squeezing the last drop out of people, but I don’t think it has more soft bias.
In principle this sounds great. In practice, there are so many other edge case variables that an algorithm cannot control for (but a human manager can). Some amount of discretion is required when enforcing rules, which an algorithm is incapable of.
This is the problem, the software is just trained by human decisions. So in many cases they are set up to become even worse. Generally it is bias cast into software.
The worst part is that they manage to give the false impression of neutrality.
And that doesn't even include all the other problems (cases being ignored/overlooked, false analysis and errors etc.).
Also who guarantees that neutral/fair treatment is ever the actual goal?
Sometimes I think though, maybe we should admit in the longer run that some of these biases are actually the truth and all talk of "fair" and "unbiased" is some kind of wishful thinking against all the facts that belongs to the land of teletubbies.
For human managers, this can be hard to prove, especially if they do not promote people very often. But for algorithms, it is much easier -- multiple people can inspect the source code for suspicious logic. And even if this is some sort of impenetrable AI model, you can still run it with different inputs, and check if the output changes if you change the input race.
When implicit biases find their way into algorithms, it is usually a specific edge case that’s a function of the training data (e.g. facial recognition preforming poorly on darker skinned people due to training set imbalance). The biggest case of implicit bias in algorithms I’m aware of is when the training set is the behavior of society as a whole, as with search engines: for example, women search less for C-level jobs and so are less likely to be shown postings for C-level jobs when doing job searches (thereby perpetuating a vicious cycle), or websites depicting Black teens are more likely to show them in a criminal context than White teens, so searches for “black teenager” show pictures of criminals whereas searches for “white teenager” do not.
These implicit biases are not encoded by the algorithm developer but rather by the dataset the algorithm is applied to.
In the case of automated firing, I don’t see how implicit bias can creep in if the metrics are strictly work-related (e.g. fraction of on-time deliveries). For the record, I do not agree with automating performance metrics, since they cannot account for nuance (e.g. delivery drivers assigned to gated communities have issues opening the gate and thus deliver fewer on-time packages, which is not accounted for in the algorithm.) However, this is not a form of demographic bias, explicit or implicit.
And implicit biais goes way beyond their training dataset. Did they put a clause "if pregnant, then...", likely not. However, what conditions did they use for testing this algorithm and ensure if it works? Testing is really much part of the process as well.
I agree in the case of things like facial recognition, but in the case of search results, Google can’t exactly choose whether websites it indexes present “Black teens” in a poor light and “White teens” in a good light. Perfectly unbiased algorithms trained on data produced by society will always exactly reflect the unfortunate biases of society. (Ironically, attempting to correct for society’s biases in the algorithm would by definition bias the algorithm itself!)
This is (likely) not the case with Amazon drivers, since the training data are (presumably) strictly job performance-related, e.g. fraction of on-time deliveries, or number of times a worker showed up late. And even if these happen to correlate to demographics (e.g. if Black workers lived further from the warehouse than White workers [or vice-versa] and were thus more likely to show up to work late), tracking employees based on their on-time performance is not bias on Amazon’s part, since a White worker who showed up late would be just as likely to be fired as a Black worker who showed up late. The fact that one demographic (by no fault of its own) happens to show up late more often is not bias on the part of Amazon.
al•go•rism /ˈælɡəɹɪzəm/
n.
1. Obsolete spelling of algorithm.
2. An irrational belief in the infallibility of algorithms.Now combine that fact with the demographics of Silicon Valley companies and you get a recipe for products and algorithms that are "fair" only if you match a set of implicit criteria.
None of this is intentionally malevolent, just a product of hubris and "move fast and break things".
[1] https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-te...
[1] https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...
Besides if the goal is "make an algorithm that only base its judgment on population studies" all human decision making algorithms will be the same, and you can endlessly cry wolf.
TL;DR: One of the factors for referring people to treatment was doctor visits over the last year. Makes sense; unwell people go to the doctor, so those going to the doctor more often are more likely to be unwell. But, of course, due to historical segregation, generational wealth, etc, black people on average tended to go to the doctor less. So even if they exhibited the same risks, symptoms, etc, as white people, because they saw a doctor about it less often the algorithm was less likely to refer them for treatment.
Think of the stories of an overly aggressive algorithm banning someone for unclear reasons. This has happened to people’s Google accounts. Since Google has limited customer support, people often have a hard time contending the decision. They are locked out of an important slice of their digital lives (email, photos, etc). Can you imagine this happening except it’s your employer who fired you?
No, it's not better. Algorithms aren't some magical unbiased utopia. They're created by people, with their own biases. They just add a level of separation between the human bias and the person getting fired.
Humans are, at least, accountable to other humans.
An algorithm also can't apply correctional factors (e.g. age, physical strength, disabilities).