A computer or “algorithm” doesn’t decide goals and values on its own.
The computer executes an algorithm that efficiently implements the values of decisions of the people in charge of developing and deploying the software (and their bosses).
Businesses have done a great job of laundering controversial and sometimes illegal decisions through “algorithms” as though they were some kind of independent entity.
Its true that somewhere behind the scenes is a set of people making the algorithm and a set of people implementing it, but for the end user/worker there is no human, no realistic arbitration, no negotiation, no favors, no coworkers, no smiles, no understanding, and no recourse. It reminds me of yesterday’s “Did I just lose $500k?” post. When S got fired it may as well have been mandated by god.
ADDENDUM: I don't want to toot our horn too much, but also remember that as a member of this forum you are probably an order of magnitude better at navigating complex institutions, understanding business objectives, understanding the “intent” if a piece of algorithmic technology, and advocating for yourself than the average person who picks up gig work. To many people it is literally magic, and the further your boss is toward being magical the closer you are to being a slave to them. And I don't think I am being melodramatic with my choice of words.
As weird as this probably sounds, I gave many of the scripts silly names and a couple of them pretend "personalities" (ie error messages/prompts will be phrased as if there's an imaginary character inside the machine).
To call software a scapegoat or "not their boss" seems a bit disingenuous. Each algorithm or script has its own goals, its own quirks and its own strengths and weaknesses, just like a human manager. Sometimes these quirks or misaligned goals lead to good people getting fired, just like with a human manager. These algorithms also replaceable and retrainable by the top-level brass, again just like a human manager.
To me, what seems wrong is not that machines are making decisions, but that there is no appeals process for the people who've had their lives ruined by such a decision.
So true. Not everybody is going to see through the smoke and mirrors.
There are other domains where analogous conversations should be had. Many people agree with their laws in the general and even specific senses, but almost nobody would want to live in a society where all the laws are enforced perfectly 100% of the time. Occasional lawbreaking is necessary for society to function. Anybody who objects to this should really think through what they're wishing for.