In my experience, my students simply deny deny deny, even when I have them dead-to-rights.
I mean, I can understand a professor have a strong suspicion of plagiarism, but proving that a student used ChatGPT seems difficult.
You read a good article so you wonder which bot made this.
You've heard it here first.
https://www.microsoft.com/en-us/security/blog/2021/07/27/com...
The world, society, is not prepared for a scenario where computers generate human quality copy. This is hugely disruptive and will be used by workers and students to cheat their classes and do their work for them for a while.
It's a disruptive technology.
Same goes for anything that has to do with selling written words. Not a small market precisely.
Rather than prove the student used Chat GPT (hard for reasons you point out), isn't this better handled as a suspected cheating scenario (collusion, candidate impersonation etc.)
Many institutions have a regulation on the books to allow for oral examination to be carried out for any course at the discretion of the course leader, for the purpose of evaluating a student for award of a grade.
If you suspected a student had used significant outside help, you'd presumably do the same - have an oral examination, and discuss the work, ask them to discuss the subject matter, and test their understanding of it.
At that point, either the student crumbles and doesn't have any level of understanding beyond "paper thin" (like if they paid an essay mill to write the paper), or they have a good understanding. If the latter, then they meet the learning outcomes of the class and have demonstrated this satisfactorily. And if they have learned the content and show an understanding of it, it's effectively acted as a study aid and actually helped them to learn. They now have shown they know the material, so they have passed the course legitimately.
I suspect in the longer term we'll see attempts to ask questions which can't be so easily answered by generative waffle, which actually should be an improvement on many types of examination. How this works in less scientific disciplines will be more challenging - I imagine they'll struggle to create "un-GPT'able" questions.
But colleges adapted (or at least good professors did) to remote examinations during the pandemic - declare the exam "open book", and ask questions that require robust understanding of topics, such that sitting and reading the book will run you out of time. Give students questions with generated parameters so each student gets a slightly different variant of numerical questions, etc.
The solution to language models generating believable or compelling waffle doesn't need to be technical - I suspect non technical solutions may win out. Whether they scale enough (or a professor cares enough to actually sit down with students for a few minutes) is another question, but maybe they should be doing that in the first place?
This, in my opinion, is the only approach that will work. When I ask cheaters to come in to explain the work, they get cold feet.
In my country, that would require the whole class to be subjected to an oral exam; as our examination regulations require strict equivalency of evaluation methods across all students that take the course during the term (contents of the exam may differ across students but the main method of evaluation, and outcomes to be measured, may not).
I wonder if there is a transparent proxy in place logging access to ChatGPT when on a campus network. My employer does this, and makes no effort to hide it. Everything is logged and specific things are captured for evaluation later.
All you have to do is fail every paper of ChatGPT quality.
However devils advocate here - I took an essay question verbatim from a humanities assessed exercise, and very quickly (think seconds of effort) created a prompt and got Chat GPT to write text each time it was prompted, until told to stop.
I copied the output (which I'd requested references to be included in), and gave it to the professor who set the question.
They very quickly confirmed the language model essay was considerably better than their human students' essays. It was more coherent and readable, better answered the question set, was more "on topic", and even better structured - it followed a structure that introduced a position, then substantiated the position, gave some mention to alternative viewpoints, and compared and justified the position in relation to them.
The topic wasn't hard, you could easily Google copious essays on the topic, but the output wasn't pulled from the web verbatim (even in parts), and it was better than the actual students' work!
The biggest failing was the references - language models generate plausible and readable text... So it invented lots of very interesting and plausible sounding references, none of which existed. The titles sounded relevant. The authors were appropriate to the subject matter, the journals or publication venues were also appropriate and relevant. It's just that none of them actually existed!
ChatGPT has a very verbose, uncanney valley style to it that likely sounded like nothing the student had organically submitted before, it's not that hard to tell.
I've had 2 students threaten me and my university with lawsuits over accusations of cheating (not plagiarism, fwiw). And I had far more compelling evidence than an uncanny valley of verbosity.
I think it's more likely that this story is bullshit (as someone else in this forum has also speculated) than the likelihood that a professor would be willing to go to the mattresses with only a stylistic argument for plagiarism.
> it's not that hard to tell.
It might not be that difficult to tell, but proof is another matter.
edit: The students were bluffing with their threats of lawsuits, but it still made for two unpleasant semesters.