Noam Chomsky on ChatGPT: High-Tech Plagiarism, Way of Avoiding Learning

That ChatGPT is good at truthfully answering one kind of prompt (analytic, where the facts are contained in the prompt, so the task is translation) and bad at truthfully answering another kind of prompt (synthetic, where the facts are not contained in the prompt, so the task is synthesis), tells me something about the nature of language.

It turns out that setting up solutions to math problems is mainly a language translation task, followed by computation task as long as you have a way to eval() the resulting thunk:

  Do not perform calculations. Do not compute the answer. Use Javascript Math and Date to perform calculations. Wait for further questions. Show your work in the DESCRIPTION. The number of steps in the %%%THUNK%%% should match the number of steps in the %%%DESCRIPTION%%%. Always answer with Javascript compatible code in the %%%THUNK%%%, including numerical form without commas (eg, 238572348723). Always answer with this JSON compatible object form, eg, {"key":%%%VALUE%%%}: { "question": "4 days a week, Laura practices martial arts for 1.5 hours. Considering a week is 7 days, what is her average practice time per day each week?", "description":["Multiplying 1.5 and 4", "Dividing the step 1 answer by 7"], "thunk": "(function() { const step1 = 1.5 * 4; const answer = step1 / 7; return answer; })()" }

Analytic augmentation is the procedure by which a question is augmented in a manner that promotes translation over synthesis. That this can be tested empirically is definitely something novel with regards to the philosophy of language!

Chomsky's universal grammar vs. the probabilistic model behind tools like ChatGPT or Google Translate is a pretty old debate.

> It's not obvious what value ChatGPT has. Kind of controversial, but I think it's not actually obvious that ChatGPT is currently most used for funny Tweets, SEO optimization, and marketing tools. So I think he's not really saying much to get angry about here.

Or in other words, it is great at being a confident sophist and its use-case(s) are limited to almost nothing of serious use requiring explainability or trustworthiness; especially for legal, medical and financial applications or even search engines. If you cannot trust the output of an AI and it keeps hallucinating its answers, then what is the point?

If OpenAI was being 'responsible' or following 'responsible AI' guidelines, maybe they should be writing the detectors first before releasing ChatGPT, GPT-3 into the wild?

This AI hype cycle is no different to what happened with GPT-3 and it fizzled out just like how Clubhouse did. The only way is open source AI models.

ChatGPT makes it easier to cheat, but cheating in that way was always an option, as long as you had enough money to pay someone else to do it for you. So in that sense, all that ChatGPT does is expose the flaws in the system by making this form of cheating available to everybody. I have to wonder why only now this is considered a serious problem that undermines education, but it was mostly ignored when the option was only available to the rich kids.

I think this can only solved by having a clear separation between learning and obtaining the credentials that certify that you possess the knowledge that it is expected of you to do some job. Universities should be centers of learning and students should be writing their essays because they recognize the value that doing so provides to them, not just because they need a certificate issued by the university to apply to the jobs that they desire. The certificate part should come from elsewhere, probably through means that make cheating much harder, such as exams and individual interviews. This would also open the door to other avenues of learning that may be more suited for some people, ie. self-taught people may have more success learning from a book or an online resource than from sitting through several hours of lectures every day.

> It's not obvious what value ChatGPT has.

This is a blatantly incorrect take. Anyone who can’t see that ChatGPT has real value has blinders on.

It has severe limitations and many quirks, but millions of people are getting value from it.

I use it regularly while developing software. If you learn how to leverage its strengths and are aware of the pitfalls, it isn’t too hard to get value out of.

It is also a good investment to learn how to use these tools, as they will become much more capable and useful over time.

This seems mostly right. But ChatGPT also can make it a lot easier to learn if one wants to learn. But I think that you need to be able to critically evaluate it's output for that. And to formulate your ideas on some level. If one never obtains those skills, perhaps learning with chatGPT won't be of much use.

I like this term. It even extends to concepts like hyper-palatable foods, with the equivalent being hyper-palatable entertainment (like TikTok, or Youtube Shorts) that similarly short-circuits the brains reward loop.

On the other hand you might also see an increased interest in doing things with your brain (e.g. writing poetry or jokes) because it's rewarding, much as you see now with interest in endurance sports, climbing, etc. Or the interest in hand making things now that we produce most things with highly scaled abd automated processes.

Look at long term stats on mathematics ability and literacy in the rich western countries. It does not paint a good picture. Two things I know off the top of my head is that literacy scores in the SATs have trended slightly down over the past 40 years, for both low-achievers and high-achievers.[1]

Australia, one of the richest countries in the world, is experiencing a multi-decade decline in mathematical standing[2]. Australia is fabulously rich, but is doing a worse and worse job of producing citizens that are numerate and understand mathematics. I haven't looked at literacy performance in the country, but I'd guess it's declining like it is in the USA, as Australia is subject to broadly the same pressures of political-economy and technology.

1. Cultural Literacy by E.D Hirsch Jr

2. https://www.sciencedaily.com/releases/2022/05/220517094842.h...

> "For years there have been programs that have helped professors detect plagiarized essays,” Chomsky says. “Now it’s going to be more difficult, because it’s easier to plagiarize. But that’s about the only contribution to education that I can think of.” He does admit that ChatGPT-style systems “may have some value for something,” but “it’s not obvious what.”

The headline makes it seem like Chomsky hates ChatGPT when he really just seems indecisive.

Chomsky kind of hints at this in the interview. He says he doesn't have a problem at MIT with not having engaged students, and though he doesn't say it I think it's obvious that his MIT students are not subjected to typical economics-driven educational pressures.

While other students are treating school as a means to a good salary and just want the highest marks for the least effort, Chomsky's graduate MIT students will be ignoring all that stuff and just having classic educational debate in essays and in conversation.

With Zizek it's the same. He doesn't have to bother with disengaged students, students who are desperate for college attendance to grant them a ticket into the comfortable upper middle classes and care not for the substance of learning. You take Zizek classes because you actually like philosophy and want to engage with it.

The problem with using ChatGPT for "filling in details" is how often it is confidently, utterly wrong.

My friend tried to use it to write a legal letter and it completely invented a state law and cited it very specifically.

For him, it was annoying and amusing. For other people, mistakes like that could be very serious.

Is the high level structure of writing any more important than the low level structure?

Maybe it will tend towards everyone sounding the same. Which then gets amplified as LLMs train on writing that is essentially mutations of itself.

Me no like.

It sounds like you're advocating a system of education, thinking, and work that is exemplified by 'Software Architecture Astronauts', people who have not recently or ever contended with the details, the nuts and bolts, of actual software development and maintenance.

To be honest, the idea that you can produce valuable intellectual work by outsourcing the details is absurd. This would have seen me absolutely fail my challenging philosophy and engineering coursework, and end up a fundamentally incompetent dilettante in work and life.

The model for mathematics is something like a “good book” of once-entered, unchanging posable questions and theorems. More may be added, but once added there is a sense of permanent definitiveness and relevance.

For your example and beyond-mathematics calculator analogy, maybe the dates of historical events are equally definitive, but I don’t see much beyond that being so. Science goes through transformations. Either the questions remain the same (what is there, what exists), but the answers to them change as science develops. Or, the questions change, and the answers don’t (what did plogiston, aether, or general relativity predict).

Your expanded “calculator” paradigm seems unfounded. It’s not analogous to math.

> after your primary school years, you get a calculator to work out the arithmetic

Do we? I do a lot of mental arithmetic as a software developer. When I was an exchange student my host-dad was an engineer and he was incredibly good at it. As he was thinking through a problem he'd do all sorts of ballpark math to get a feel for things. Eventually, yes, the numbers would all get crunched properly. But even then he'd also be doing the math mentally to make sure that something hadn't gotten screwed up. A person who uses a calculator for everything starting from a young age just doesn't develop that skill.

And I think there will be a similar problem with ChatGPT. Can an excellent writer get a performance boost starting with ChatGPT? The jury's out, but it's not impossible. But can one become an excellent writer by tarting up LLM-generated prose? I doubt it. Learning to write is a process of sweating the details. Especially so when the output is an essay. Real higher-level creativity requires foundational work. I think at best you'll get the sort of content that comes when an uninspiring politician hires a mediocre ghost writer.

That said, a lot of writing is bullshit. And I mean that both in the Frankfurt sense ("speech intended to persuade without regard for truth") [1] and in the Graeber sense ("completely pointless, unnecessary, or pernicious") [2]. And that's where I think ChatGPT will shine. Marketing copy to fill in the "lorem ispum" space on the product page? ChatGPT to the rescue. Long sections of TPS reports that nobody will read? ChatGPT again. And once they expand ChatGPT to being able to make PowerPoint decks, a zillion ineffective middle managers will be able to return their focus to Candy Crush, harassing the interns, and creating pointless meetings.

I think that's also bad, of course. Lowering the cost of bullshit production looks like a net societal negative to me. But it would be successful in the traditional tech industry metrics of "gains users" and "produces revenue".

[1] https://en.wikipedia.org/wiki/On_Bullshit

[2] https://en.wikipedia.org/wiki/Bullshit_Jobs

> Did adoption of calculators lead to less learning of arithmetic. Certainly. Was that a bad thing? I honestly don't know.

IMO it was bad for individuals, good for society. People are innumerate and that is exploited by people selling products and services.

> Did adoption of calculators lead to less learning of arithmetic.

There's discussion of this in the book The Shallows, which is a book that gives a good overview of the relationship between technology that augment our intellect and contemporary society.

The gist is that calculators where a boon to mathematical education, because they freed up students to spend more time working through the challenging abstract aspects of mathematics, and less time with the mechanics of arithmetic.

I don't see there being any analogy between using ChatGPT _as an essay writer_ and using a calculator. The essay writing itself is the substantive educational process, and so it cannot be outsourced. What is not the substantive part of the educational process is the actual production of the typed word, and so the use of a pen, typewriter, or computer keyboard is less material, though still absolutely relevant. The affect of the writing/typing tool on the essayist is also discussed in the book.

generalizations dont apply to individuals, but its correct people , in general, like to jump on any opportunity not to learn but just to use something. that being said id expect most ppl lurking around NH not to feel that generalization applies to them... but they do use intellisense, ides, stack overflow, good etc. rather than taking the long road. up to a certain point that is ok, aslong as u are aware which corners you are cutting and what that means to ur personal development.

Even better would be one that generates its sources and backfills to an arbitrary depth.

It is exactly what www.perplexity.ai does (and yes, it has replaced google for some of my usecases).

For certain things I think so. For me the fascinating and really useful utility of LLMs is their ability to synthesize answers to questions. This is something search generally speaking can't do. I find myself using ChatGPT to answer question like:

> Explain tar -xzvf

> Answer: The command tar -xzvf is used to extract a tar archive. Here is what each option does:

-x: This option tells tar to extract the contents of the archive.

-z: This option tells tar to decompress the archive using gzip compression.

-v: This option tells tar to run in verbose mode, which means it will display the names of the files being extracted as they are extracted.

-f: This option specifies the file name of the archive that you want to extract.

So, the full command tar -xzvf is used to extract a tar archive that has been compressed with gzip and display the names of the files being extracted as they are extracted.

However for other types of information I much more care about being taken to a trustworthy source of information rather seeing a summarized view or worse, a synthesized view of various sources that might not agree with each other. References are useful, but it's risky to rely on what a bot says without validating the references at which point the utility of the bot for that type of query is questionable.

The Bing AI chat does. Also you.com.

So you don’t need it in life because you don’t do it, but it’s an important basic skill. Explain this belief.

> I don't think the view that just because we can automate the creation of a simple essay makes writing essays for students obsolete. It's not what Chomsky said but I've heard the argument a lot in the context. We are not requiring it because everyone is writing essays all the time but because it's an important, basic skill.

I think there's something to be said for the view that ChatGPT makes the sort of simplistic, tedious essays typically assigned to K-12 students obsolete. Essay writing is fun, but school essays take the fun out of it, much as is the case with school math. School is corrosive to enjoyment of intellectual activity. This is by design: the primary purpose of school in the USA is to break the students' individuality so they learn to obey commands and submit to authority. Vital skills in every industry from construction and manufacturing to high tech.

ChatGPT puts the lie to the idea that essays in standard formats on standard topics with standard argument structure should be how we teach kids to write, much like TI-83s put the lie to the idea that school-age math is mostly rote memorization and application with little conceptual thought. We will have to show them how to find their own personal voice and have it come through in their work.

The ChatGPT era will require adjustment from educators. Again, remember Danny Dunn and the Homework Machine, a 1950s kids' novel whose McGuffin was a supercomputer that functioned much like GPT: you asked it natural-language questions and it would type out answers. The computer was sabotaged such that it would answer in a way that was coherent and authoritative sounding, but wrong, challenging the young heroes to actually study to understand when and how the machine was wrong. ChatGPT poses similar challenges to today's kids, and if we want to teach them we will have to acknowledge and accommodate those challenges in the curriculum.

> but I do not understand the reason.

Some people get mad if you talk down on their favorite new toy.

It's like having wood sawing as a woodworking exam, when power saws have just been invented.

This is easy to say once you already understand how to write essays.

Meh... if a dumb robot can do <X> for you, maybe we shouldn't have <X>. <-- This logic doesn't get us far. Robots do many things for people in industrialized societies.

I for one, think training students on how to write is very important. It trains them how to think critically, how to present solid arguments, how to persuade, etc. I certainly don't want my society to have fewer educated people.

You’re right. We should cancel all math classes immediately. We’re way past due since we should have done it with calculators.

This completely misapprehends the point of writing essays.

I don’t get this, that dumb robot got trained and so is doing the job. There are some fundamental building blocks that every human needs to be trained in. We can question what these building blocks are but never apply a blanket statement that “if a robot can do it we don’t need to learn it”. We will end up with severely challenged humans and that would be the manifestation of robots taking over not because the AI became sentient but because we humans gave up learning.

Unless we plan to make the movie Idiocracy a documentary, we need to continue to learn how to learn.

This is like saying that phones can play songs, so maybe we should stop learning how to play musical instruments.

The robots can write shitty essays because they were trained on billions of other people's sentences. They can regurgitate human effort, not replace it entirely.

And we don't write essays so that the world is full of essays. We write them so that we understand and consider new concepts.

Computers can dictate words now, maybe we shouldn't waste time teaching toddlers to read.

What should we do if a dumb robot can keep us suspended in a nutrient rich gel and create a detailed set of mental experiences for us?

One might say his colorless green ideas are no longer hegemonic in academia

We have been waiting for AIs and neural networks to explain themselves transparently for decades. Given that they still cannot do this, then it means you cannot trust its output and its decisions.

They have a long way to go, but ChatGPT's and LLMs use-cases are limited outside of anything serious or trustworthy. There is nothing revolutionary about a AI SaaS.

What is 'revolutionary' or game changing is an open source equivalent of ChatGPT; like what Stable Diffusion did to DALLE-2.