"AI", students, and epistemic crisis
miniver.blogspot.com
miniver.blogspot.com
I use both the internet and GenAI extensively now. But I feel that having gone through the "knowledge work" activities without the crutches puts me in a better position to assess the correctness and plausibility of internet sources and AI in a way that kids who grow up using them constantly don't have.
I feel quite privileged to be in that position, that I wouldn't be in had I been born 10 or 20 years later. I also feel sorry for kids these days for not having the opportunity to learn things "the hard way" like I had to. And I feel extremely snobbish and old for thinking that way.
With generative AI we are now shortcutting directly to the destination while eliding all the knowledge and understanding we are supposed to be gaining on the way. It's deemed sufficient to merely produce and exchange the symbols of understanding without needing to possess any real underlying wisdom (the first view above), because it's through the exchange of these abstract symbols, irrespective of whether or not there is anything behind them, that we can play and win certain social games.
This is merely the natural continuation of trends that began during the dawn of the Internet era, when we started to consider pixels on a screen an accurate map of reality.
They’re going to use it. Give them the training and guidance to use it correctly.
Terse prose was a "lost art" even in my generation (millenial-ish) and I'm not surprised that it has gotten worse.
On the other hand I cannot not help thinking that this is similar to the arguments brought forward when the internet was new. How could correctness and plausibility be established if you don't have established trustworthy institutions, authors and editors behind everything? And yet it turned out mostly fine. Wikipedia is alive, despite its deficiencies, Encyclopedia Britannica not so much.
So, is it only old people's fear?
Not long ago I had the same viewpoint as you! But thinking back now — it dates me but I definitely lived a childhood without Internet access — probably the optimistic belief before our age of "misinformation" is that, in the marketplace of ideas, the truth usually wins. Goes along with "information wants to be free" — remember that slogan?
For us that grew up learning things "the hard way" so to speak, that made perfect sense: each of us, as should have the capability to discern what is good or bad as, individual independent thinkers. Therefore, for any piece of information, there should be a high probability, in the aggregate, that it is classified correctly as to its truth and utility.
I think, that was to some extent, even a mainstream view. Here's what Bill Clinton's said in 2000 advocating to admit China to the WTO: (https://archive.nytimes.com/www.nytimes.com/library/world/as...)
Now there's no question China has been trying to crack down on the Internet. Good luck! That's sort of like trying to nail jello to the wall. (Laughter.) But I would argue to you that their effort to do that just proves how real these changes are and how much they threaten the status quo. It's not an argument for slowing down the effort to bring China into the world, it's an argument for accelerating that effort. In the knowledge economy, economic innovation and political empowerment, whether anyone likes it or not, will inevitably go hand in hand.
I would say, what we have since learned after some 20 years, is that in the marketplace of ideas, the most charitable thing we can say that the memes with the "best value" win. "Best value" does not necessarily mean the highest quality, but rather there can be a trade-off between its cost and the product quality. Clearly ChatGPT produces informational content at a pretty low cost. The same can be said for junk food, compared to fresh food: the overall cost of the former is low. Junk food does not actively, directly harm you, but you are certainly better off not eating too much of it. It is low quality but has been deemed acceptable.
There are examples where we can be less charitable of course. We all complain about dangerous, poorly manufactured items (e.g. electronics with inadequate shielding etc.) listed Amazon, but clearly people still buy them anyway. And then, in the realm of politics, needless to say, there are many actors bent on pushing memes they want you to have regardless of their veracity. Some people on the marketplace of ideas "buy" them owing to network effects (e.g. whether they are acceptable according to political identity, etc.) in the same way that corporations continue to use Microsoft Windows because of network effects. We also probably say nowadays Clinton has been ultimately proven wrong by the government of China.
Survival of the "fittest" memes if you like: evolution does not make value judgements.
If you ask me, maybe our assumption of de-centralized truth-seeking was itself, not an absolute truth, to begin with. But it took years to unravel, as humans, collectively speaking, atrophy from disuse of the research and critical thinking skills before technology dropped the barriers of entry to producing and consuming information.
Remember that the new generation doesn't just have different tools; they're also much less experienced & mature, just like we were.You can only really compare yourself to them in the future where they're at the place you're at now.
And when chatbots don't know, they lie.
Kids will learn the hard way. Possibly harder then we did.
And even this scenario assumes a working postal system, availability of paper and pen. But if you didn't have those, you may not have heard of Greek to even ask the question.
Few (if any) "laypeople" did their own research, as you and I understand it today.
Old people usually make correct assessments given their knowledge and experience. They know how to maximize their expected gains and play safe. The problem is, real life often rewards those who take risks, and make seemingly wrong decisions, that later turn out to be good.
For example, as a kid I loved playing video games, while my grandma yelled at me for not wanting to help her with work at the farm. She had absolutely no way of predicting that playing video games, which were essentially just toys, would teach me the right skills at the right time, allowing me to move up the social ladder. At the time, forcing the god damn lazy kid to milk the god damn cow was the sensible thing to do.
https://news.rice.edu/news/2019/quantum-dot-tattoos-hold-vac...
Or perhaps people who have been confused by conspiracy sites like npr.org and make the mental leap that this would be used for other purposes:
https://www.npr.org/2018/10/22/658808705/thousands-of-swedes...
There are Microsoft patents for microchips to track body activity to reward in cryptocurrency and subsidiaries who wanted to microchip vaccine passports into the hands of immigrants.
Mention the word "elites" and you are a Nazi by association. Works every time.
Growing up we heard, ad nauseam, that "wikipedia is not a reliable source". People just need to state the same thing about LLMs- they aren't reliable, but can potentially point you to primary sources that -are- reliable. Once the shininess of the new toy wears off, people will adjust.
Wikipedia is as good as anything else.
As an example, I frequently use Wikipedia to read about history, computer science topics (e.g. how to implement an algorithm), or scientific topics.
The exception is current events, but even then, I suspect that Wikipedia is not any more biased than the news.
I'm open to having my mind changed, though.
Encyclopedias – including Wikipedia – are not acceptable sources for college-level work certainly. They are tertiary literature, which can provide an overview to someone trying to get a toehold into a subject, and which can hopefully point them toward primary and secondary sources. But tertiary sources are not typically allowable citations for college research.
But, this thread has discussed the reliability of Wikipedia.
I will say that wikipedia, as things stand, is way more accurate than chatgpt. So much so that comparing them doesn't even make sense to me.
But don’t get me wrong, I also view ChatGPT conversations as being on par with pub chats as far as “confident facts” go.
I think our difference may be in how we view Wikipedia.
Last I read about this, the error rate in Wikipedia was actually lower than in the Encyclopedia Britannica, by a measurable amount.
This was a while back, and admittedly it only counted articles where there was an equivalent article in both (which probably gives a better picture of Wikipedia, as those kinds of “boring” articles have less vandalism…) but it’s not immediately a given that Wikipedia is just objectively bad at being accurate.
Now of course like any tool this can be used without falling into that trap, but people are lazy and the truth is that if you don't absolutely have to do it yourself most people won't.
(Actually, didn't ChatGPT have a disclaimer right next to the prompt box that warns against incorrect answers?)
So I'm more surprised (and scared) that students don't just use LLMs for sourcing but are also convinced they are authoritative.
Maybe being in the tech bubble gave a false impression here, but weren't hallucinations one of the core points of the whole AI discurse for the last 1.5 years? How do you learn about and regularly use ChatGPT, but miss all of that?
I am a product of the German school system, I'd say I wasn't indoctrinated too much, so its not entirely broken, but with these new """tools""" maybe we need a reform anyways.
What exactly do you mean by the word “axiom” here?
Think of it as one layer of abstraction above the model under discussion. Like a hyperparameter. In later years, students get taught the same topics again, with the hyperparameter tuned to be more realistic.
eg, All bachelors are unmarried.
eg, If X is a triangle, then X has three sides.
eg, the world is round or it is not the case that the world is round.
And does “universally recognized” just mean that most people believe the proposition is true?
It's AI in the older sense of Machine Learning, not in the currently widespread sense of an LLM, which is the source of the problem that the author is discussing.
We will be able to negotiate excellent trade deals outside the EU effortlessly
The EU need our trade so badly we will be able to negotiate a seamless trade deal with them
… and many others.
Another faction of the pro-Brexiteers would think of the system in terms of a uniparty. The fundamentalists like Farage have no chance of getting elected as PM in the British system, though he did get an MP seat now.
Would things change if Farage were PM? I'm cynical, probably not.
And this can be seen from the raw numbers too. Before brexit, the annual net migration to UK was about 300k. After brexit, the annual net migration to UK is ~700k.[0] (745k in 2022, 680k in 2023.) And quite a bit of that from the fresh trade partners.
The best way I have heard anyone describe the disaster that's Brexit was as "nostalgic self-immolation for the population who still dream of the Britain where faces were white, passports were blue, and the map was Imperial Pink."
0: https://commonslibrary.parliament.uk/research-briefings/sn06...
If you ask it about something it doesn't know, right then and there, it will concoct a fiction for you. It won't say "I don't know," or "I can't help with that."
If you coach it to respond to something in a certain way, it'll respond that way for you as its top priority.
If you ask it to review a text, it'll usually find a way to give you at least a 7 or 8 out of 10. (Though, interestingly, rarely a 10/10 score. You can upload excerpts from some of the great works of literature and philosophy and see ChatGPT give them an 8/10, just as it gives an 8/10 to your essay or blog post.) Practically the only way to get a halfway critical response is to add the words "be critical" to your prompt.
A more circumspect and less obsequious ChatGPT would solve a lot of problems.
It doesn’t know that it doesn’t know. Texts it was trained on rarely go “On the History of the Greek Language, by Andros Dimos; ahem what is the history of the Greek language? I don’t know. The end.”
GPT-4o responded: "The Palace at Midnight" by Robert Silverberg is a science fiction short story that was first published in 1980. It is part of Silverberg’s "Majipoor" series, but stands alone in terms of its setting and characters, distinct from the expansive world-building typical of the series. The story is set in a dystopian future in India and revolves around themes of memory, loss, and the haunting allure of the past.
The narrative follows the protagonist, Edward, and his group of friends who are survivors of a nuclear war that has devastated much of the world. . .
All of which was complete nonsense. "The Palace at Midnight" is a short story of a few pages that has nothing to do with Majipoor, was published in 1981, and was set in San Francisco.
It doesn't know -- it knows that it doesn't know -- so it just makes something up that seems superficially credible. Because it aims to please and be helpful; altogether too helpful.
No it doesn’t; it isn’t an intelligence, it’s a token generator based on patterns in the inputs it was trained on (and those are explanations and answers, not millions of people writing “I don’t know”).
For instance, a cursory check of isfdb.org would have given ChatGPT better information than I was provided with.
The only smooth conversations with Copilot are the ones where you allow it to regurgitate "facts" and act in a submissive manner.
Relying on kids to do cross referencing and deeper fact checks into everything they ask an LLM is just not going to happen at scale.
It would take seconds to figure out of the ChatGPT citation is real or made up.
The AI does not circumscribe all of reality; not all of reality is captured on the internet.
That it would just come up with random false facts about something as common and "basic" as the history of the Greek language sounds like a made-up issue.
Chatbots are great systetising in areas where there is a strong open source like culture - a lot of people writing tutorials and on stack overflow. The moment you move into area that is not popular to write about, they start making stuff up.
And it sux massively. I would much rather seen "I don't know" then what ChatGPT produces currently.
Since there's no actual knowledge on the part of the AI, it doesn't know that it's doing it, so it's not malicious (the designers are a separate question). But it is still entirely possible, and even somewhat likely.
It’s kind of terrible.
But yeah ever since I could start adding “and include an explanation with citations” to my prompts I’ve not had an issue with hallucinations more than once or twice.
If I’m using it to make purchase decisions or scheduling my day then I use it purely to find me source pages.
It’s really terrible at calculating the cost of services.
It is very much a black box in terms of letting you track its logic.
Seems like future versions should be much more transparent in terms of letting you track the logic of why it’s telling you what it’s telling you.
Many facts and answers are gathered/aggregated from many different (sometimes conflicting) sources. It won't link the internet page where it found the information, because it didn't find the information on a single internet page.
That's why they "cant" show sources.
This is a little hyperbolic and instead maybe the kid can learn that chatgpt.com is not a reliable source. It even says at the bottom of the page "chat gpt can make mistakes". Lots of things are not reliable sources. Wikipedia is not a reliable source. Teachers are supposed to teach this, and teach cross referencing at the very least, not freak out.
Otherwise, you could do a mixture of very strict exams without multiple choice and large individual projects (no group projects).
If you only do exams, people who don't do well thinking in a room crammed full of people at 8am will be disadvantaged.
Not as wrong as the author thinks. From Britannica.com:
"Greek language, Indo-European language spoken mostly in Greece. Its history can be divided into four phases: Ancient Greek, Koine, Byzantine Greek, and Modern Greek."
From wikipedia: “[Homeric Greek] is a literary dialect of Ancient Greek consisting mainly of an archaic form of Ionic, with some Aeolic forms, a few from Arcadocypriot, and a written form influenced by Attic.”
I’m not sure if this is a plausible explanation as I don’t have much experience using LLMs.
I do believe the students keep coming up with weird (correct) facts, and that this can be scary for a teacher who is stuck at a search bar.
isn’t the same as
Four distinct languages had an influence on the development of this fifth language.
I think young people growing with LLMs will know all the ins and outs of their limitations. But it's a rapidly evolving field, maybe in 5 years we won't doubt LLMs very often.
Of course they changed it multiple times, it used to say that you shouldnt take what it says at face value, now it says it may sometimes be inaccurate, soon it'll say "learn more" and link to a page about how their model is super accurate but potentially, one in a million, makes mistakes.
Im worried about these students because they will be in power in a few decades. Its already a shitshow with people who didnt grow up as AI iPad toddlers.
I feel like parents are failing here, more than anything else. You can't stop these companies from doing this if it drives up the stock price, equally you cant vote against it effectively because a majority of the voting population either doesn't care, doesn't know, or asks ChatGPT what to vote for.
Of course Plato was right and the only way we can fix it is to have philosophers in charge, not demagogues. Good luck with that, maybe in another 2000 years we'll be smart enough to make that happen!
LLMs are often wrong and you cannot check their sources. Also since it is generated you can trick it into spitting out falsehoods intentionally.
They are not even close.
"Generally right" is not the same as "reliably right", and therefore if you really need to rely on a fact for something important, I would trust neither Wikipedia nor LLMs.
Depends on which LLM you are using, perplexity and copilot both cite their sources.
I do feel sorry for professors like this that cannot adapt to technology changes at the rate they are happening
Considering this is a transcription of a deleted twitter post, my internet radar of "yeah, that happened" gives a lower chance this is true than your average ChatGPT answer.
And somehow this is an improvement over the previous status quo. ChatGPT is pretty well read and on important issues rarely says stupid things. Better than believing some influencer.
No student is ever hurt by the introduction of a more advanced knowledge system. We heard similar laments decades ago, with: Students just believe the first 10 search results of Google. Those students are now the teachers of today, starting at the search bar.
I'd go so far as saying (if version other than 3.5 was used) that ChatGPT was correct and has far more linguistics knowledge than this teacher ever will. "Greek is actually a combination of four other languages" is not an answer that ChatGPT will ever give, but something a teacher makes up to claim Ch*tGPT is a Nonsense Machine.
ChatGPT: Greek has evolved in stages from Mycenaean Greek (Linear B script) through Classical Greek, Hellenistic (Koine) Greek, Byzantine Greek, and Modern Greek. It has been influenced by ancient Near Eastern languages, Latin, Turkish, Italian, and French.
If there really is an epistemic crisis, then it already existed and ChatGPT merely reflects it, not caused or contributed to it.
This would be the equivalent of my saying that a basketball coach is obsolete because NBA 2K is available.
ChatGPT output some garbage, and the student doesn't understand why or how it can be wrong. Presumably this is where a professor would attempt to help the student develop some critical thinking skills.