[flagged]
"When disagreeing, please reply to the argument instead of calling names. 'That is idiotic; 1 + 1 is 2, not 3' can be shortened to '1 + 1 is 2, not 3."
"Edit out swipes."
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
However, the education system, as formulated in many countries, isn't solely focused on broadening understanding and instead on students getting a degree with a given grade to aid them in getting a job in their preferred career.
With that goal in mind, their incentives may not be solely focused on learning and more on passing essay questions by creating texts. In that case, it's possible to see how their incentives might line up with the use of LLMs to partially or wholly create their essays.
This is, of course, nothing new. There's a reason why universities have plagerism detection software and systems. Students often do not write essay purely themselves, and LLMs will be just one technique to allow them to do so more easily...
Both of these true at the same time, integrated via finance. Fanciful ideas of pure education with its motives and methods are almost like fairy tales in this environment.
Anyone reading news media recently will have noticed the vast amount of badly written/researched articles, often with click-bait headlines. Recently I've even noticed that sources considered "high quality" like the Times in the UK are heavily using click-bait techniques.
I think the USP of a free press is to shine a light on the issues missed by the formal government — be that large-scale corruption, or hyper-local potholes, or systemic racism or sexism in some (public or private) institution.
"Quis custodiet ipsos custodes?" Ideally, if not in reality, it's the press.
Wrong. LLM plays a significant role in building understanding in this day and age. Combining Google (Internet) Search and LLM to do one's research and knowledge accumulation is the bread and butter.
With LLMs you could get even some background or context to the stories with perhaps even a hint of critical "thinking".
Although more probably the press releases will be more and more just passed through LLMs to match the paper's "style".
Most publishing is not in the business of increasing understanding. It is difficult to get a business to understand, when its revenue depends on it not understanding.
Indeed. What I thought most interesting about this exercise were the ways in which this author's process pushed him towards learning and understanding - even to the extent that he found himself tempted to read the original sources for himself!
I'm a former humanities lecturer, in both English and History, so it's a thought exercise for me to consider how LLMs would change my pedagogy. It'd be naive to assume that students won't want use this technology, and foolish to think that they can be stopped. The only sensible approach, it seems to me, is to guide students towards productive use - ie, methods which will expand their understanding.
This author's most productive use, it seemed to me, was his literature review, which guided him towards useful sources, and an understanding of the contours of debate around the topic. Those, in traditional pedagogy, are the primary purposes of the instructor. It's notable that to do this he had to provide his own corpus of texts (of dubious provenance!), and self-train the model. Both of those functions require skills which are beyond most students and nearly all humanities professors.
The most obvious use was the structuring and writing of the essay. He pointed out one hallucination he caught, but there were probably more that he did not. In the humanities we have traditionally used a well-structured and elegantly-written essay as the gauge of learning and understanding. Will that have to change? Based on this example, and others I've seen, I'd want to review students' working process, and help them refine their use of the LLM. If the prose it produces can be expected to be of reasonable quality - and it's already far better than most undergraduates (unfortunately) are able to produce - then the truer test of their understanding is the guidance they have given it.
Or maybe essays are now the wrong "proof of work"? If so, then what replaces them?
I have no firm conclusions, besides a depressing expectation that educators in the humanities will remain entirely naive to these technologies, to their and their subjects' detriments. I appreciate the author's project, and welcome further discussion.
No! If the student produces a poorly written essay they should receive a bad grade.
This "engaging with the LLM" is nothing more than Googling the answers. It's incredibly detrimental to the student's understanding. One way to prevent this kind of cheating would be to require essays to be hand-written, or typed on a mechanical typewriter.
That's useless, I'm afraid. Students will generate an LLM text, and then copy it out. If you lock down their machines, or university networks, then they'll access the LLMs another way. It's cat and mouse games all the way down, and we'll never win.
I mean, I get where you're coming from. I'm a humanities guy, through and through. I love writing essays - or, well, really dig having written a good one; the writing process is invariably a slog - and I'm good at it. The writing process catalyzes my learning and crystallizes my thoughts, etc etc. I believe all that stuff, preached it without irony, and spent countless hours in tutorials coaching students.
The trouble is, it doesn't really work. Writing a lot of essays makes only marginal improvements to students' writing, no matter how many tutorials and "writing labs" they go to. Reading complex texts, and learning and imitating good writing, teaches people how to write. And, you know, that only works when they want to a) read complex texts, and b) learn how to write well.
For the students who intrinsically neither a), nor b) - which is the vast, vast, vast majority - we could force them a bit, with grades. Now, however, there are LLMs, which break both halves of that method. Our whole approach to a system of study will have to change. I would rather find something new and useful than cling to a useless paradigm because I was once comfortable and successful within it.
I think this is more a product of low standards than anything else. Let's bring the mean down to 2.0GPA. Not everyone should pass. Not everyone should graduate. Failure delivers valuable lessons, too. But I'm sorry, throwing up our hands and saying "oh well, I guess cheating is the norm now" is fatalist BS.
> Reading complex texts, and learning and imitating good writing, teaches people how to write. And, you know, that only works when they want to a) read complex texts, and b) learn how to write well.
I agree with this. Inspiration is super important, and inculcating a love of learning and the life of the mind is the whole ballgame. If the vast majority of students aren't in this boat, and instead they're just trying to check a box to graduate, the whole thing is super broken. But that doesn't mean it can't be fixed, and it doesn't mean we should lower standards in the face of a threat like LLMs. Instead we should raise them.
> inculcating a love of learning and the life of the mind is the whole ballgame. If the vast majority of students aren't in this boat, and instead they're just trying to check a box to graduate, the whole thing is super broken.
Which... Yup. The whole thing is super broken. It has been for a generation. (It's maybe always been at least a little broken? Complaints about students not caring about the life of the mind and only craving the credential were common in the middle ages, too!)
Here's a counterpoint for discussion, which I'm not sure I fully believe in: LLMs can support life-of-the-mind learning. (Even if you think they aren't there yet, their trajectory is clear.) Even apart from that, they will be used everywhere, outside of the classroom. Don't educators have a responsibility to train students in their responsible use?
In the end I dropped out because the point of college wasn’t “learning” (which I was) but rather “how good are you at playing college” (which I wasn’t).
I creeped your profile real quick (I'd up-voted you a time or too, lol), and I'd have loved to have had you as a student! Your writing style is admirably concise, which mainly means you have your thoughts in order. (Avoiding "terse", which is sometimes accurate, is a matter of learning a few rhetorical tricks to better engage a reader. That's, like, the easiest writing "problem" to solve.) Clear thinking is 90% of what we (should) want undergraduates to demonstrate, which would put you orders of magnitude ahead of the typical student who hasn't any ideas of their own, and desperately tries to eke out 2k words of pure waffle. From me, 1k words of closely-written reasoning would have been a 'B' from the jump, and provoked a conversation about where else you could take your argument, should you have the time and interest to pursue the subject further.
I'm sorry you had such bad experiences. They're sadly not rare.
I do think we're heading for a pedagogical crisis, and I don't see much beyond hand-wringing coming from people in education. This is mostly because their technology skills are (by and large) very nearly nil - "cliometrics" in History, and "digital humanities" in English, are regarded as niche - so nearly everyone with influence within the profession has been blind-sided. I have one former colleague who retired last year, a few years ahead of her plan, rather than deal with LLMs.
Yours is, frankly, the first "practical" investigation of what's possible that I've seen, anywhere - I've passed it along to several people I know. Thank you for doing it, and please post anything else you may do in this space. There might even be a business opportunity in it? Education consultants can make good money, which is usually regrettable, but this specific topic is essential to address right now.
So seeing that essays aren't a good proof of work anymore, lecturers would need to come up with a better one. Or, heaven forbid, allow students to actually do something productive with their time.
Or do you think I enjoy losing my time reading stuff that you could not even be arsed to write yourself?
Of course, the irony is that your readers might take the full text and use an LLM to summarize it back into bullet points.