The Hidden Costs of Automated Thinking
newyorker.com
newyorker.com
Modafinil works by at least 3 mechanisms. Part of the complexity is because we don’t yet fully understand how and what causes sleep; or why it is so necessary that without it you will surely die. So balancing what does what it (modafinil) works by is difficult, however possibly the greatest contribution to our understanding of the mechanism governing the drive towards sleep has been modafinils insights into this:
As you go about your day, adenosine (yes, the dna base) accumulates in the extracellular space in the brain. These levels reach a peak before you sleep; sleeping allows re-uptake/cleaning of intracellular adenosine. Theories have proposed that levels of intracellular adenosine contribute to the drive to sleep/fatigue.
Caffeine binds as an antagonist to the adenosine receptor; which is where the stimulant effect is derived from.
Modafinil causes re-uptake of adenosine; so there is less intracellular adenosine bonding adenosine receptors (and presumably causing the reverse of stimulation), a function normally carried out at night.
There are at least two other known functions since I last did an enormous deep dive in modafinil and read every published paper during Med school
My honest opinion is that you should try to balance your life, sleep well and exercise, because everything else is just a crutch, and crutches shouldn't be used long term.
“A world of knowledge without understanding becomes a world without discernible cause and effect...”
To put that point another way: It's commonly known that shark attacks are rare. But that's because we know where sharks are. We don't swim in beaches where sharks can easily swim, and rarely far out enough for them to be prowling. When swimming in deep water, shark attacks are comparable to risks of sports like cycling [1]. Sharks aren't going to kill you, but if we didn't have institutional knowledge of where they attacked (and continued watersports at the rate we currently have), you'd likely know someone who died of one.
Most problems reach equalibriums. Some problems have a very strong S-Curve effects. "How deep is the water" and "probability of dying of a shark attack" has a sigmoid curve, where as soon as you go into water deep enough for sharks to actually swim, the danger goes from "Unheard of if not impossible" to "Potentially Life Threatening but within acceptable risk territory". Once out there, it keeps going up.
People don't swim in deep water because deep water is not generally accessible. One of the reasons that deep water is not accessible is because deep water also contain sharks. These have reached an equilibrium.
Second example of this fallacy:
"Helicopter parents" is a commonly cited problem [1]. Parents are overprotective of children. The rise of helicopter parenting has coincided with a reduction in child mortality rate from non-natural causes. Wouldn't that indicate that helicopter parenting is working? The answer ranges from "No" to "Maybe". Children killed by car accidents correlates with a decline on overall accident rate - But much of that is driven by initiatives like MADD, which has worked to improve general auto safety as the most effective way of protecting children. These factors can be compared to stoichiometric math - They are trending towards equilibriums, but probably have not yet reached them, and as other reactions happen around them the setpoints they are trending to change.
[1] https://www.washingtonpost.com/news/wonk/wp/2015/04/14/there...
I can't help but think even if there were no sharks at all the amount of people swimming in deep water would not increase that much, because I find it likely that the other causes of people not swimming in deep water have a much greater effect on how many people actually swim in deeper water.
Also there are people now who swim in deep water specifically to swim with Sharks.
Its a parlor trick perhaps, but its silly to ignore the results.
As for building a biased system, again its just about the data you use and you could easily build an "intelligent" system that had a racial bias. We should worry about automation that simply pushes the status quo but I don't think that's a problem specific to statistical based ML.
> It’s possible to discover what works without knowing why it works, and then to put that insight to use immediately, assuming that the underlying mechanism will be figured out later.
> Our accounting could reflect the fact that not all intellectual debt is equally problematic.
We often talk about the technical debts we are accumulating when creating something new. But what sometimes can be worse is when one forges ahead without the proper understanding, extrapolating from familiar but possibly not applicable precedence.
The complexity of modern tech makes it all but impossible for even most of us here to fully understand how our computers work - a very different situation from the time Woz was able to hold all moving parts of the Apple design in his head.
We have to rely on stuff we don't understand and it can be an emotional challenge with different individuals reacting very differently. Avoidance leads to a crawling speed and attempts to reinvent the wheel when the simplest sufficient solution calls for a four wheel drive. Deliberately closing eyes is also not uncommon - believing in the magic of the solution with no regard to the laws of nature. Overly broad deep diving, studies and investigations is another anti-pattern. As is being overly conservative - my current project technical debt is way too much code as we did not fully understand the framework we build upon and feared to make the required small innovations.
Finding the right balance here as an individual or finding common ground as a team is challenging.
Nassim Taleb covers this idea in his Antifragility.
Asbestos was a remarkable flame-retardant material. However, putting it to use as widely as we have was probably a mistake.
The asbestos sort of danger is much more insidious and omnipresent. You have to not only know that your use is safe, but consider everything that could happen to it on the molecular level and that all the consequences of all those things are safe.
But I'm even more glad that I learned to navigate LA via map and memory well before I came to rely on GPS.
Lately I've been trying to keep that fresh by taking a moment to mentally walk through the navigation instructions before I start traveling somewhere. Having the overview in your head -- even if you forget some of it -- makes a world of difference between building a geographic model and just following instructions.
Somewhere in here there's probably a lesson or two about the difference between augmentation and automation, but it's not one I've teased out well enough to articulate better than this.
I recently went to Amsterdam. Before I left, I got Google Maps directions for various places I was going and then used Google Earth to "walk" the routes. After 10 or 15 minutes I built up enough of a sense of landmarks and general layout that I felt pretty well oriented when I arrived.
That was cumbersome to do, but I'd love to see GPS tools move in that direction of supporting not just my current goal, but my long-term independence from needing turn-by-turn directions for everything.
https://www.youtube.com/watch?v=DOW_kPzY_JY&feature=youtu.be...
You really only need to know how to get to and a major thoroughfare, how those thoroughfares connect to vicinity of your destination, and then the steps from the thoroughfare to your destination.
I think it isn't truly artificial intelligence but fundamental theory vs practice in a long standing dance over eras.
It should be obvious that we have and are continuing to push a lot of half baked AI into industry and government processes and that it's frustrating users no end.
Yes we're closer, but we are still a million miles away from AGI. Or unregulated self driving cars. Or flying ones.
The difference is the front line personnel can sense if you're agitated and they need to escalate your case quickly. I've heard saying "operator" repeatedly down the line may help in this particular case. A human operator will normally react. AI/ML/Expert Systems etc don't care.
Plenty of "average" people have complained to me about this use of their time, and that it is not good value for money, especially given the extravagant fees or taxes they may be paying these particular organizations.
This always did happen before, people who knew people or were much higher net worth obviously got better service, but when it can be automated and subjected to everyone in a granular fashion, it really brings home where you are in the hierarchy and removes any veneer of being equal in society. This is also already implemented at various types of businesses via their “rewards” programs where you’re tiered into different support based on how much you spend or can spend, and get routed to different support agents accordingly.
But of course, that won't happen for much longer, because we love our monopolies (yay FAANG or utility companies etc.)
Consumers can and have benefited from economies of scale greatly that the big companies can provide in the short term, but in the long term it can be costly as they exercise their monopoly power, but who can accurately put a price on this so that people choose wisely today.
I've been left waiting at the counter at an auto parts store while the store personnel left me to address the needs of the local auto mechanic's shop (who undoubtedly buys 1000x the parts I buy in a given year). The parts store should prioritize the mechanic over me, because they're a much more valuable customer to the parts store. It doesn't do me any good to have the parts store fold because they served everyone exactly in sequence.
The USPS undoubtedly has specific support lines and discounted rates for high volume mailers while I as a retail customer pay retail prices and get crappy retail support. Amazon and UPS should get much better service from USPS than I do.
Comcast Business customers probably (hopefully) get better service that $99/mo “TriplePlay” residential customers.
People with startups here probably prioritize engaged, paying customers over free users.
Airline Elite or Diamond or Private customers spending $25K or more every year should get better service than the buyer of a once-a-year $250 trans-continental ticket. When an airline experiences a disruption, they're going to re-accommodate their First Class and Elite frequent fliers first.
That's just good business, every bit as much as when I walk into the local diner and the server brings me a coffee prepared as I normally order it before taking someone else's order.
It's also psychologically different when you know an organization is tracking every single person and how much future potential revenue they can bring versus how much they cost to support and price discriminating accordingly. It's obviously the optimal thing to do as a business, but as a society, the idea that we're equal and should be treated as such is also an important feeling. I can see possible discord in society from having that socioeconomic tiering be so blatant and in your face, but perhaps it's inevitable in a world where the gap between the haves and have nots keeps getting bigger and bigger.
I don't go to theme parks much, but I did a couple years ago as an adult for the first time since I was a kid, and I have to say it felt weird to see all of the different tiers for the queues for the rides. You pay $x, you wait in longer line, you pay $x+$20, you get to skip to halfway through the line, you pay $x+$50, you get to skip to front of line. It really brings to the forefront how un-valuable your time may be compared to someone who can afford to spend more, and I know the world has always worked like that, but I wonder how it feels to the kid who can see it happening in front of them.
You say “ideological”, but it is also sound science. Machine learning models of this kind are based on observational data and thus find correlations without knowing anything about causation, and so you need to put in a lot of effort to find and avoid problematic correlations. Of course, then the magic disappears, because an ML algorithm can't actually tell you if someone is a criminal.
I think it's quite common that the quantity you're interested in isn't observable, so you need to proxy it with something. The GP's example is "Person X is likely to commit a crime". If that could be estimated reliably, it would be extremely useful for allocating governmental resources like policing and education.
The problem is "Person X is likely to commit a crime" isn't observable, so a careless researcher might proxy it with "Person X is likely to be convicted of a crime". The latter is actually very different for the former, since it includes factors like a defendent's ability to hire a good lawyer, existing police presence around Person X's neighbourhood, and government priorities on which crimes to prosecute (think crack vs. cocaine in the 80's).
Any good social scientist or economist will be aware of all of this. But once you bake it into a model that doesn't explain itself, you have a mess on your hands. Especially if the model gets more credence than it deserves by people who don't understand it.
The incarceration patterns therefore reflect this political bias.
And that's ignoring more complicated feedback loops. Since colonial times, American whites have often used their dominance to keep black people impoverished. [1] Poverty and crime are correlated. Wealth is correlated with getting away with crime. So if a system looks at crime statistics without considering at the history, it would be easy to perpetuate the ugly parts of that history.
[1] See Kendi's "Stamped from the Beginning" for the colonial-era laws and practices, and Loewen's "Sundown Towns" for the Nadir up through suburbanization.
Yes. Statistics are for populations, not for individuals.
Using a statistical correlation as the basis for an individual decision is inherently biased, but is something that is seen all too often unfortunately.
If the algorithm is going to insist on treating all black people like criminals because their crime rate is higher, than it's a bad algorithm and needs to be fixed before shipping, or scrapped altogether.
The author raises a concern about automated thinking. But the article is displayed in a scenario that relies on a plethora of automated thinking. Let me guess he's only using good automated thinking while the other automated thinking that affects him is the bad kind?
Ultimately, humans use tools and we push really hard to develop better and better tools. Maybe the tools other people are using concern you (and maybe that's a good thing to be concerned ... after all shoe stores used to x-ray feet), but progress continues and those that don't adapt are left behind.
Maybe he's got a point, but I also don't see the new yorker abandoning its website and reverting to paper. How bad can the automated thinking really be.
Honestly, I'm not sure I have any major refutations or solutions concerning the article. It's just that I keep bumping into people making odd statements about how they're really worried about the computers or the internet and they're making these statements on facebook. Seeing someone concerned about AI on an ad supported website makes me wonder if they appreciate how much of their existence is already dependent on AI.
Not something that had occurred to me before
At some point, if a decision process is widely adopted yet cannot explain itself, law becomes arbitrary. Ends justify means - ipso de facto. Power becomes unaccountable.
That's the best reason I know to demand accountability (and explicibility) from any system or authority, AI or not.