An Anatomy of Algorithm Aversion
papers.ssrn.com
papers.ssrn.com
In particular, algorithsm do not offer advice. Advice is a case where your own goals, ambitions, preferences, desires have been understood -- and moreso, what ones you arent aware of, what needs you might have that arent met... and these are lined up with plausible things you can do that are in your interest.
There is no algorithmic 'advice'
Re: advice, well, there could be, but the people who put these things in place aren't necessarily thinking in those terms. They're thinking about a statistical edge and acceptable negative outcomes on their end and no one else's. They're not maximizing what's good and helpful for you unless it helps them also, they're probably maximizing short to medium term profit seeking. Computers are an amplifier of human bad behavior.
See also, "computer says no": https://en.wikipedia.org/wiki/Computer_says_no
Perhaps unsurprisingly author's i've read on this topic are all from the humanities, or adjacent (here law) and seem to not really to have a grasp of technology as a social and political phenomenon the way modern (aware) tech workers do.
Here I think software engineers feel intuitively and immediately that replacing any of their decision making with "algorithms" is a high risk. I suspect papers like this would benefit a lot from assessing the "algorithm aversion" amongst "algorithm experts".
In my own experience, human forecasters and decision-makers tend to be much easier to hold accountable for bad forecasts and decisions. At a minimum, they stake their reputations, just by putting their names to their actions. With algorithms, by contrast, there's often no visible sign of who created them or decided to use them. There's often no effective process for review, correction, or redress at all.
The fact that high-volume, low-risk decisions tend to get automated more often may partly explain this. But it may also partly explain general attitudes toward algorithms, as a consequence.
Au contraire. It is the correct understanding, born out of deep expertise, that algorithms, outside very structured artificial environements, often do not work well at all.
Even provably correct algorithms fail if there is even the slightest mistmatch between the assumptions and reality, imprefect data, noisy sensors, or a myriad other problems. Not to mention that the implementations of these provably correct algorithms are often buggy.
When of algorithms are based on user input, users learn very quickly how to manipulate the algorithm to produce the results they actually want.
Is there something special about the algorithms people are averse to? Maybe not actually working?
They act as if the only way to get algorithmic knowledge is to grind specific examples, when really if they would read through Knuth they'd be fine.
These are people who see algorithmic knowledge as the bane of their careers rather than seeing algorithms as:
1) a great way to add value to resource constrained projects
2) a trivially simple and easy way to signal programming abilities, letting you easily breeze through the interview
I seriously would hate having to work with one of those who take pride in how little computer science they know, and because I run a math and computational geometry heavy organization, I will never hire them. But I would estimate that algorithmically and mathematically averse coders form the majority of coders.
First of all, nobody "read[s] through Knuth." [0] (I couldn't find the reference, but I recall a story about Bill Gates telling Knuth he had "read his books," to which Knuth replied that he believed Gates was lying.)
Second, the way the "algorithmic portion of the technical interview" is generally constituted currently is beyond flawed. Depending on your perspective, you can either pass it by memorizing 5 or 6 algorithms and re-using them over and over; or, it's a completely unrealistic test of anyone's ability to think about and work with algorithms and data, because there is no such thing in the real world as a 45 minute deadline. Of course, you can certainly argue that it's not intended to be a test of one's ability to work with algorithms and data, but, rather, an IQ test of sorts. But, then, we have companies that are literally giving candidates IQ tests now, so, why not just drop the pretense?
---
[0]: https://www.businessinsider.com/bill-gates-loves-donald-knut...
Related:
The legal rule that computers are presumed to be operating correctly https://news.ycombinator.com/item?id=40052611
> In England and Wales, courts consider computers, as a matter of law, to have been working correctly unless there is evidence to the contrary. Therefore, evidence produced by computers is treated as reliable unless other evidence suggests otherwise.
DNA evidence is also trusted by default and yet:
https://www.science.org/content/article/forensics-gone-wrong...
https://www.nbcnews.com/news/us-news/investigation-finds-col...
With software, generally the only people with the means to demonstrate the software is flawed are the people in control of the software and associated data.
It's a sort of lazy argument that one can imagine a homo economicus which might might better decisions on a proxy variable, less lazily, bemoaning that they don't optimize the authors' preferred measurables.
It shows self-awareness at times
> It is worth noting, however, that the algorithm in the study was designed to optimize system-wide utilization rather than individual driver income. > The algorithm’s design weakens any conclusion about algorithm aversion, for individual drivers may have been better off optimizing > for themselves rather than the system.
It has the air of a future cudgel. The title works as a punchline, and as for the strength of the argument, well it's published (posted at all) online, isn't it.
There are all sorts of cofounders to algorithms in the real world and an expert human is better at dealing with unexpected cofounders than an algorithm. Given the number of confounded possible, in real world use, it is likely that there will be at least 1 confounder.
> algorithms even though (2) algorithms generally outperform people (in forecasting accuracy and/or optimal decision-making in furtherance of a specified goal).
Bullshit. Algorithm means any mechanical method, and while there are some of those that outperform humans, we are nowhere near the point where this is true generally, even if we steelman this by restricting this to the class of algorithms that institutions have deployed to replace human decision-makers
If you want an explanation for "algorithm aversion", I have a really simple one: Most proposed and implemented algorithms are bad. I get it. The few good ones are basically the fucking holy grail of statistics and computer science, and have changed the world. Institutions are really eager to deploy algorithms because they make decisions easier even if they are being made poorly. Also, as other commentators point out, the act of putting some decision in the hands of an algorithm is usually making it so no one can question, change, be held accountable for, or sometimes even understand the decision. Most forms of algorithmic decision-making that have been deployed in places that are visible to the average person have been designed explicitly to do bigoted shit.
> Algorithm aversion also has "softer" forms, as when people prefer human forecasters or decision-makers to algorithms in the abstract, without having clear evidence about comparative performance.
Every performance metric is an oversimplification made for the convenience of researchers. Worse, it's not a matter of law or policy that's publicly accountable, even when the algorithm it results in is deployed in that context (and certainly not when deployed by a corporate institution). At best, to the person downstream of the decision, it's an esoteric detail in a whitepaper written by someone who is thinking of them as a spherical cow in their fancy equations. Performance metrics are even more gameable and unaccountable than the algorithms they produce
> Algorithm aversion is a product of diverse mechanisms, including (1) a desire for agency; (2) a negative moral or emotional reaction to judgment by algorithms;
In other words, because they are rational adults
>(3) a belief that certain human experts have unique knowledge, unlikely to be held or used by algorithms;
You have to believe this to believe the algorithms should work in the first place. Algorithms are tools built and used by human experts. Automation is just hiding that expert behind at least two layers of abstraction (usually a machine and an institution)
> (4) ignorance about why algorithms perform well; and
Again, this ignorance is a feature, not a bug, of automated decisionmaking in practice with essentially no exceptions
> (5) asymmetrical forgiveness, or a larger negative reaction to algorithmic error than to human error.
You should never "forgive" an algorithm for making an error. Forgiveness is a mechanism that is part of negotiation, which only works on things you can negotiate with. If a human makes a mistake and I can talk to them about it, I can at least try to fix the problem. If you want me to forgive an algorithm, give me the ability to reprogram it, or fuck off with this anthropomorphizing nonsense
> An understanding of the various mechanisms provides some clues about how to overcome algorithm aversion, and also of its boundary conditions.
I don't want to solve this problem. Laypeople should be, on balance, more skeptical of the outputs of computer algorithms than they currently are. "Algorithm aversion" is a sane behavior in any context where you can't audit the algorithm. Like, the institutions deploy these tools are the ones we should hold accountable for their results, and zero institutions doing so have earned the trust in their methodology that this paper seems to want