The Impact of Generative AI on Critical Thinking [pdf]

On your last point I tend to think it will. Tools replaced our ancestor's ability to make things by hand. Transportation / elevators reduced the average fitness level to walk long distances or climb stairs. Pocket calculators made the general population less able to do complex math. Spelling/grammar checks have reduced knowing how to spell or form complete proper sentences. Keyboards and email are making handwriting a passing skill. Video is reducing our need / desire to read or absorb long form content.

The highest percentage of humans will take the easiest path provided. And while most of the above we just consider improvements to daily life, efficiencies, it has also fundamentally changed on average what we are capable of and what skills we learn (especially during formative years). If I dropped most of us here into a pre-technology wilderness we'd be dead in short order.

However, most of the above, it can be argued, are just tools that don't impact our actual thought processes; thinking remained our skill. Now the tools are starting to "think", or at least appear like they do on a level indistinguishable to the average person. If the box in my hand can tell me what 4367 x 2231 is and the capital of Guam, why then wouldn't I rely on it when it starts writing up full content for me? Because the average human adapts to the lowest required skill set I do worry that providing a device in our hands that "thinks" is going to reduce our learned ability to rationally process and check what it puts out, just like I've lost the ability to check if my calculator is lying to me. And not to get all dystopian here... but what if then, what that tool is telling me is true, is, for whatever reason, not.

(and yes, I ran this through a spell checker because I'm a part of the problem above... and it found words I thought I could still spell, and I'm 55)

> good enough result (…) sounds plausible

It’s paramount to not conflate the two. LLM answers are almost always the latter with no guarantee of being the former. That is a tremendous flaw with real consequences.

> it’s a general weakness in human cognition, and has little to do with AI in and of itself.

A distinction without merit. Like blaming people for addictive behaviour while simultaneously exploiting human psychology to sell them the very same addiction.

https://www.youtube.com/watch?v=EJT0NMYHeGw

This “technically correct” approach to arguing where the fault lies precludes productive discussion on fixing the problem. You cannot in good faith shout from the rooftops LLMs are going to solve all our problems and then excuse its numerous failures with “but we added a note you should always verify the answer. It’s down in the cellar, with no lights and stairs, in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’”.

It has become norm for people to “settle” disputes by quoting or showing a screenshot of an LLM answer. Often wrong. It is irrelevant to argue people should do better; they don’t, and that’s the reality you have to address.

The study is basically just... a bunch of self-reports on how people feel about AI usage in their daily job. That's it. The "critical thinking" part is completely self-reported.

It's not "how good a person is at critical thinking." It's "how much a person feels they engages in critical thinking." Keyword: feels.

It's like all the nutrition studies where they ask people to self-report what they eat.

It really has been a sight watching the loudest anti-AI people flog this around, then turn to rage when you clarify the range of the actual conclusion.

Generative AI got a lot more useful when I started seeing it abstractly. I put data in, it finds correlations in its neural network, and produces different data. I have to be intentional in interpreting that data to figure out what to do with it.

Once I started thinking in terms of "this word will kick up a hornet's nest in that branch of the network and make the output useless. Let's find another word that gets closer to what I'm aiming for," the outputs got so much better. And getting to the good outputs took fewer iterations.

I beg to differ. AI made it possible for human to pursue critical thinking. Overwhelmed by the basic facts and routine works, limited by bandwidth and 8 hours a day, we hardly have the luxury to think above and beyond. That's when you hire consulting firms to stuff the content, the ocean of information, the type of work now potentially suitable for AI.

It is time for human to move up the game and focus on the critical thinking, even only the critical thinking while the AI is still unable to perform the critical thinking. Eventually there is the hope that AI would be able to handle the critical thinking, but it remains a hope at the current state of art.

Thanks for clarifying, based on the title I totally thought the study was about critical thinking in general.

For the record, I’m not going to read the article to verify your statement.

> Analysing 936 real-world GenAI tool use examples our participants shared, we find that knowledge workers engage in critical thinking primarily to ensure the quality of their work, e.g. by verifying outputs against external sources.

The researchers appear to be saying that a lot of people perceive LLM outputs to be superior to whatever they could generate themselves, so users accept the LLM's logical premises uncritically and move on. Sort of a "drive by" use of chatbots. The researchers are not wrong, but intuitively I don't think this is a fair critique of most LLM tasks, or even most users.

One of the most compelling LLM use cases is individualized learning or tutoring, and LLM's can support a much deeper dive into arcane scientific or cutting-edge topics than traditional methods. I don't see anything here that suggests the researchers balanced these scenarios against one another.

The key thing about this paper is the invalidity of the measures. They are essentially looking at the correlation between measures of frequency of AI usage and measures of critical thinking skills.

Yet, in the questions themselves, questions like “I always trust the output of AI” (paraphrasing) are in the measures of frequency—and questions like “I question the intentions of the information provided by AI” (which is not a reasonable question, if you use AI regularly) is in the measures of critical thinking.

Sorry I don’t have time now to share the actual text. Take a look yourself though, at the end of the paper.

Climate change is already a pretty good precedent on this

If a study comes out, no. But I do not think the trajectory of LLMs is as determined as you may think

I mean, that question is really a question for yourself. You cannot control others, so assume others will do whatever is quickest, easiest and/or least resistance, outcomes be damned.

Of course nothing would immediately do a 180, but if there would be research put out that your brain slowly stops working to more you use LLMs, then you can at least make your own choice from there on out.

I haven't used LLMs a lot and have just experimented with them in terms of coding.

But about a year ago, I had a job to clean up a bunch of, let's call them, reference architectures. I mostly didn't mess with the actual architecture part or went directly to the original authors.

But there wasn't a lot of context setting and background for a lot of them. None of it was rocket science; I could have written myself. But the LLM I used, Bard at the time, gave me a pretty good v 0.9 for some introductory paragraphs. Nothing revelatory but probably saved me an hour or two per architecture even including my time to fixup the results. Generally, nothing was absolutely wrong but some I felt was less relevant and other stuff I felt was missing.

> in my experience LLMs hallucinate so much on stuffs I know about that I instinctively challenge most of their assumptions and outputs

In my experience most people don’t do that. Therein lies the problem.

It probably also depends on whether you trust the AI. I don't, at all. So, I'm often critically thinking about whatever nonsense it spews out.

This would be even better:

https://www.microsoft.com/en-us/research/publication/the-imp...

This is already how society works, you just aren't wearing the glasses.

Childhood education, entertainment, social networks are all constantly trying to guide our actions. Agendas are everywhere.

AI might actually be more objective or transparent, depending on how it's implemented. The key will be choice - you can either choose not to use it, or you can choose which models you prefer.

Unfortunately, most people have little choice in their society.

Many people have already abandoned their mental autonomy to whatever delusions they receive from whatever social network they are on.

I suppose the LLM could be worse since it will more directly promote someone's agenda. I just question how much worse it could possibly be.

From my experience it does. A lot of VPs and higher have no direct experience with the things they are responsible for so can be easily manipulated by smooth talkers. I just attended a strategy meeting where a lot of higher ranks asked questions that clearly showed they have no idea.

The notion of a correct spelling is a relatively new innovation. I can't come up with a defense even for a conventional spelling that doesn't lean on "people will try to shame you for it," and that doesn't seem like a great reason to do things that add cognitive load for no real benefit.

That's a great point about how using the models may actually just be revealing our own incompetence, not causing it.

Maybe it's a quantity over quality issue. What if we're actually doing more of the thing - or at least we're guiding the process - than we would have if LLMs weren't available, which enables us to become better at recognizing how LLMs are actually beneficial for the task.

I remember the story of the two groups of pottery students, one tasked with making as many pots as possible, and the other with making one perfect pot. At the end of the day, the quantity group's pots were also higher quality than the quality group's single pot, simply because they had far more practice.

I know I code more with LLMs than I did before.

“Just like when pocket calculators became popular and we all got worse at mental arithmetic but much better at applied mathematics overall. High level programming languages have made software engineers worse at comp-sci, but much better at applying it; commodification of cars made the average driver much less capable of understanding car mechanics, but much better at driving; and so on.”

I don’t agree with any of these conclusions. I guess it’s debatable of course.

I can either spent 20 minutes arguing with an LLM to get something that maybe does most of what I want, or spend those 25 minutes to get something n that does exactly what I want.

I wouldn’t call those 5 minutes times wasted

200 IQ: Waste time rewriting AI slop.

Why? It wouldn’t surprise me at all if critical thinking is overapplied in most roles or poorly executed.

There are many use cases where people not following a process and thinking they know better is an issue.

After watching [Oxford Researchers Discover How to Use AI to Learn Like a Genius](https://youtu.be/TPLPpz6dD3A?si=FJJ-S6wz0PPrJuSn) a few days ago, I've been using ChatGPT in "reverse mode" a lot. I give it a excerpt of a text I'm reading and ask it to ask me questions from it at different levels of detail.

I have to say it feels like a superpower! The answers to questions that you needed to supply really stick on your memory as do the links that spontaneously form to bodies of knowledge you already know when answering deeper level questions.

I'm thinking that LLMs might actually address some of Plato's complaints against reading and writing:

> You know, Phaedrus, that is the strange thing about writing, which makes it truly correspond to painting. The painter’s products stand before us as though they were alive. But if you question them, they maintain a most majestic silence. It is the same with written words. They seem to talk to you as though they were intelligent, but if you ask them anything about what they say from a desire to be instructed they go on telling just the same thing forever.

See [here](https://fs.blog/an-old-argument-against-writing/).

This assumes you always have reliable GPS:

- areas that lack map details and obstructions that preclude straight line paths, e.g. deep forest

- wartime GPS blocking

- device failure incl battery

- errors in mapping data

- maps that contain other information not found in your GPS-enabled map

Knowing which direction you are facing is an obsolete skill?

I don’t think you’ve been paying attention.

You’ve been watching too many Ted talks or something

AI can program, but not engineer. Even then, you eventually reach a point in the project where it is too complex for AI to even do snippets; especially if you are pioneering something new that has never been done before.

A sprinter is unlikely to win a marathon, and that is what using AI to program is like. By the time you have to take over, you have a huge learning curve ahead of you as you can lean on the AI less and less.

If you're doing something boring/boiler-plate, yeah, AI is helpful I guess.

> People are worried AI is making us dumber.

I'm far from an "LLM Defender" but I've heard plenty of people say "well Google said.." for at least a decade.

I think LLMs accelerate this, but we're not in totally unfamiliar territory here.

For GPS:

https://www.npr.org/2011/07/26/137646147/the-gps-a-fatally-m...

This is dead on. I'm not even a big AI fan, but this is a key idea about technology in general. I don't want to have to bring to mind the laws of physics every time I drive to work. The whole point is that a group of engineers encoded them into the machine for me, and now I enhance my capabilities without needing to know how. It's what the classic Alfred North Whitehead quote is talking about. I understand the impulse towards mastery and ever-expanding knowledge—who doesn't love the idea that they should be able to "plan an invasion, change a diaper, butcher a hog"—but the truth is we have finite capabilities we are capable of mastering in a lifetime. This is why even literal geniuses often fail when they step outside their field of expertise. It's a valid concern that as a society some skill will be lost, or concentrated in the hands off too few, but losing skills and knowledge (or I would simply call it "being permitted to forget") is in general fine. Now if AI literally killed people's ability to think, that would be one thing. But what I suspect is that like parent is saying, it allows you to turn off your brain for certain tasks, like every technology. Then the question is what more complex tasks can we do on top of the automatic and thoughtless ones.

EDIT: I see some good replies to parent about stability/reliability, alienation etc. There are definitely tradeoffs to the power you get from technology, and it's worth acknowledging those. But that's exactly the framework we should be thinking in. What are the tradeoffs involved? Often these kinds of stories are one-sided arguments that imply losing skills is straightforwardly bad, when in truth it's more complicated than that.

> so we can focus on what humans are actually good at

You know what humans are good at? Deluding ourselves. Because that's what you're doing. Using vague, feel-good words, based on vague analogies, no proof, to keep the inconvenient truth at bay. Not being able to navigate with a paper map is a big thing: people get lost inside buildings without a map. Next time the power fails, half of the GenZ will be lost. Not being able to write with pen and paper is a big thing. Not being able to add a few number is not as big a thing, but it certainly can be a problem. And what for? So you don't have to feel bad about using AI tools?

You know what comes next? Everything based on audio and video. Are you then going to argue: reading is an obsolete skill?

> It’s the same old story. New tool comes along, people freak out about what we’re “losing.” But they’re missing the point. It’s never about losing skills, it’s about shifting them. And usually, the shift is upwards.

Except for that widespread feeling of hopelessness, alienation, powerlessness, lack of motivation, and lack of ambition. Almost as if not learning any human skills, and relying solely on technology for everything might have some second-order effects.

I've thought the same way you did, how people are resistant to change, but eventually it's better for everybody.

I do believe that GPS made people worse drivers. It made it so people lost sense of direction and distance. It has removed all critical thinking on the road. Plenty of stories where people drive over stairs because the GPS told them so.

From a driver's ability to navigate, I don't think you can do more now with GPS than you can before with a map. It surely has made it easier, but at a significant cost.

Now, of course, there are plenty of benefits, such as reducing time when getting somewhere unknown (e.g. ambulances), planes not flying over hostile territory (mostly), ability to be able to tell someone where you are when there no landmarks around etc.

But the reality is that overall, a mistake of a GPS is usually rather localized, and the cost of the mistake is rather low.

Books are interesting, instead of memorizing details we now memorize where to find information, little bits that help us get to the solution of the problem we're solving. But books themselves haven't replaced memory, otherwise no-one would read them anymore ahead of time.

When we search something on the internet we are thought to apply critical thinking. What are the sources? [0]. But GPS? Just go with it.

And AI is more like GPS than it is like books. We are being taught take it at face value, and to abandon critical thinking for the sake of speed. Worse yet, because of the enormous financial investments of companies, there is an incentive to lie about how useable it is.

I'm not even talking about context windows. I'm talking about the endless minutia of languages, frameworks, and changes related to specific versions that you only learn by doing. Just the same way a resident does not become a doctor until they finish residency. They have to have done the work, and applied critical thinking.

Software Engineering does not have such legal requirements, but we all learn on the job. AI, and the companies pushing for it basically tell potential clients that this is no longer needed. Would you want a gallbladder surgery done by someone who just read a Wikipedia page about it?

Now, a seasoned developer who writes a crystal clear prompt will probably pick up on bugs, and tell the AI that they want edge cases A, B and C considered. But how did they learn that those exist? Right. By hitting the issues.

Something that happens a lot in Software Engineer, due to the massive amount of things out there and no fixed specs/docs/etcs, is that your approach changes when you're developing a solution for a problem. But the need for those changes only become apparent when you're writing and testing code.

You literally cannot front-load that into your prompt. Yet, reading the news here, we see that our future is writing prompts for a much lower wage. This is orthogonal as to why I went into Software Engineering. Prompts rob me of the ability to express something in an extremely well defined language. Clarity of rules. A syntax where you can express something without ambiguity [1].

You don't know what you don't know, meaning you can't prompt for what you don't know. Hence why they brought back a whole bunch of people out of retirement to build new manpads.

[0] Interestingly when I was growing up a book quote was ok, but Wikipedia was not, even though it came from a book. That now definitely has changed.

[1] A wife sends her programmer husband to the grocery store for a loaf of bread... On his way out she says "and if they have eggs, get a dozen". The programmer husband returns home with 12 loaves of bread....

I doubt they mind if their customers lack critical thinking skills