She Was Falsely Accused of Cheating with AI – and She Won’t Be the Last | Hacker News Reader

She Was Falsely Accused of Cheating with AI – and She Won’t Be the Last

rollingstone.com

48 pointsminimaxir37 comments

What a shitshow.

> Stivers received an email from her professor indicating that a portion of it had been flagged by Turnitin as AI-written

> Last month, a Texas professor incorrectly used ChatGPT in trying to assess whether students had completed an assignment using that software. It claimed to have written every essay he fed in — so he temporarily withheld a whole class’ final grades.

> William Quarterman, a senior and history major at the college. A professor had run his exam answers through an AI detection called GPTZero, which returned a positive result. The professor gave Quarterman a failing grade and referred him to the same student affairs office

Let's let professors indict students based on untested (and amateur) AI algorithms that not even the vendors completely comprehend and subject the accused to a process in which they are presumed guilty until they prove their innocence-- while they mortgage their future paying 5 to 6 figures for the privilege.

These are the real AI "safety" issues that need to be addressed. The only "danger" posed by AI is in its total lack of accountability. This is outright abusive behavior on the part of colleges, who wasted no time in weaponizing it to conduct virtual arbitration. The irony of an AI system flagging students for copying what someone else wrote is apparently lost on everyone.

Any sensible schools should have plagiarism allegations reviewed by a committee where due process is followed. The professor should not allowed to just give a failing grade on personal belief.

I'm kinda appalled that many professors have jumped on the GPT bandwagon so quickly, claiming it should be part of the standard curriculum and how we should have it as the foundation of academic programs, going forward. I don't really understand their lack of critical thinking on the use of this unproven tech, at all. Makes me really doubt their competence as an educator and an academic.

> Makes me really doubt their competence as an educator and an academic.

Lol, happens everywhere, power of memes and all that. Only difference is you know more about ChatGPT than them.

I’m not sure I know more, but they like to and are incentivized to dream perhaps quite a bit more than needed.

Go Aggies! Good to see my alma-mater getting known for something other than pepper-spraying sitting protesters in the face.

Turnitin advertises the feature as "98% accurate". That's a pretty terrible accuracy rate for potentially destroying someone's life.

The utter hypocrisy of having to write "ethics / impact" blurbs for class projects, when the school can seemingly implement this sort of life-changing technology without a long trial period and carefully designed guard rails around how to interpret the results is galling.

The best case scenario is that there are so many false positives that they get recognized for the garbage they are, but I think in reality there may be a small population of students who are unlucky enough to write in a way that gets flagged significantly more often.

How do you defend yourself against that sort of situation? "Well, your classmates didn't get flagged. Why did you?"

The problem with this technology is that there is no concrete evidence that you can use to prove / disprove the accusation. There is no single piece of plagiarized content that you can dig up and point to as the smoking gun.

The ethics of letting this technology go live is breathtakingly stupid. I'm concerned this sort of laziness is a harbinger of things to come, and it fills me with dread.

>Turnitin advertises the feature as "98% accurate". That's a pretty terrible accuracy rate for potentially destroying someone's life.

Assuming the 98% applies to both false negatives and positives:

For a given 20k person school, assuming they each write only one paper a year that's run through this, an administrator gets to expel 400 students a year!

What fun!

The problem is, that's actually sort of the best-case scenario. Obviously, the protests and insanity of that situation would cause it to be quickly scrapped.

The more insidious issues is you have a sub-population of students that write in the "wrong" way. That means the average student might never get a strike, but this unlucky sub-population might get multiple strikes. That would seem to point to the results being "correct" -- after all, you'd expect a cheater to cheat multiple times.

If this population skew is significant enough -- say, 1% of the population writes in the "wrong" way, and the multiplier effect is 10x as strong for them -- you might never get a flag for a normal student. But what you're measuring is not cheating, but deviation from the norm in writing style.

I'll just point out that college students papers are exactly the sort of thing these AI chat things also do. Take a body of preexisting material and distill it into an essay. Really seems to me this is moving the bar as to what counts as cheating rather dramatically.

Memory that pops up is Barton Fink being told to write a script for a wrestling film.

Also sort of topical in high school had a quiz on a short story that I had not read. But I had read some of the authors other short stories. So I started out with 'While I failed to read 'story title'. Based on the author and the title I think the story is about bla bla. And the teacher took off 2 pts for not reading the story first.

98% accuracy?

Haven't these people all taken statistics course? Or at least enough of them. Just to read that one number and do some rough estimations of what it means in any meaning...

You can detect it. You interview the student about their paper. If they don't know or understand what they wrote, then either they cheated or they may as well have cheated because they didn't learn anything. If you have enough money (they do), make the interview part of the assessment for all students.

Crazy that educators think 98% accuracy is good. My back of the napkin math suggests that it's statistically improbable to NOT be flagged at some point in a 4 year program. This kind of gross negligence should be dealt with. It sounds like there are no responsible parties somehow. This poor woman's life is altered at a critical time in her life. Someone should be held responsible.

so in a class of 100, 2 people will be incorrectly flagged?

very low chance, unless you are one of the two people this week.

Exactly. 10 essays a quarter,…

Even if the number was correct, which it ain’t, this is insanely bad

If the number was correct, openAI would kill to obtain the algorithm to employ GAN techniques. As always, detecting anything AI-made just means the AI is now capable of passing this test too.

Students who cheat get an 'Academic Dishonesty' statement added to their transcripts. Professors who accuse students of cheating based on flimsy evidence or broken tools they selected should have 'False Accuser' put next to their name in all university schedules and catalogs, along with their employment record.

The burden of proof on the professor is so high there that the result would be nobody ever getting accused of cheating, which already happens because the graduate students that do most of the grading are not paid enough to do detective work on short papers. In my uni it works like this: the student is given a 0 for the assignment and if they wish they can go to Academic Affairs to contest it and get a formal review process. Most do not escalate it because this is a rare event for someone who is cheating their way through undergrad and can be seen as a cost of doing business.

Evidently it is not.

key word being "there", in your scenario the process is entirely in the favor of cheaters.

Yes. There's a power imbalance without accountability. Of course they're going to abuse it. This professor and his university are both clearly incompetent. Students shouldn't even have to defend themselves against a cheating accusation. Either prove it or assume it didn't happen.

I used to be a teacher and even when I strongly suspected a student of cheating, if I couldn't prove it, I wouldn't even mention it to them or anyone else. Weak evidence doesn't mean partial guilt.

I'd be interested to know the argument for calling ai-assisted writing cheating in the first place. Nobody would call using grammarly cheating, and if the text is style over substance in the first place it is not really right in academia. Somewhere between grammarly and chatGPT it becomes cheating, but universities haven't yet explained where that border really is.

universities haven't yet explained where the border between black and white is either. That does not make them identical.

I taught a 102 course last semester with 13 weekly very short writing assignments. The level of cheating was pretty demoralizing. In one week, 15 of 75 students all used ChatGPT. Luckily when you are grading that many at a time it becomes very clear who is cheating based on the overlap between them and structural similarity, though they do try quite hard to muss it up to look original.

In general I found those tools mostly useless and very easy to fool by making minor changes. I had better luck putting the original writing prompts into chatGPT, saving them all to a text file, and cross referencing when dealing with student work that didn't pass a sniff test.

Repeat after me - detectors are snakeoil. They sell you the idea you can avoid the disruption you cannot. Not only is it trivial to bypass detection, including more sophisticated not yet implemented watermarks, the technology also biases against non native speakers.

I am half tempted to show students how easy it is to fool the detection. Turn something into a parenthetical, remove red flag phrases, cut out fluff. Or better yet, just use it as a skeleton draft and rewrite it in your own words. Then I would show them how easy it is for me to see through their fudging, which I do by just noting down the similarities between papers and the information inside the GPT response. In some cases there is 100% overlap between the two. They always repeat certain words, too, so I use ripgrep to see who else used them and do cross sectional comparisons. But I also think for now it's a good idea to take the magician's code on that part of the process, at least when communicating with the students directly.

All of this is the kind of detective work that most graduate students are not qualified to perform easily.

Unfortunately this will also catch students who write clear and well structured essays. My friend was taking a summer course on her break from college. Her professor accused her of cheating because her work was a clear outlier. It was a clear outlier because she was a significantly better writer.

A better check is that there’ll be a mismatch between content and citations, or the citations will be entirely hallucinated. As far as I can tell ChatGPT is incapable of generating actual citations.

The outliers consistently write better prose but they never include nearly as much distinct points of information within their responses. They also have way less fluff. They include personal anecdotes. That professor probably didn't have enough materials of hers to work with.

As for checking hallucinations I definitely do not have the time to go that deep on 75 submissions a week. If that could be automated it'd be helpful. The students are correct that I can afford to spend only about 5 minutes per essay on grading (they're short, 200-250 words).

next up: CiteGPT

plug in your finished work, making up whatever stats and arguments and assertions suits your taste or agenda, and CiteGPT will back-solve matching snippets that support your writing, inserting all in line citations and building a reference list automatically. Maybe even suggest minor tweaks to make your arguments match the available supporting material.

This problem has such a simple solution: have the kids write some in-class essays throughout the term. Then, if cheating is suspected on a homework assignment, the teacher cross-references against the known GPT-free work to compare its quality.

It does mean that good writers may be able to get away with cheating, but then they really didn’t need the practice to begin with.

The students just use GPT within class because they are completely shameless. I'd have to sit behind them and watch their screens like a prison guard. I haven't resigned myself to that so far.

I think the more reasonable solution is to just wait for the university to take active measures against it, and accept that this is just a new development in how students cheat, which they have always done. The fact that the administrators are dumping this all on teachers and TAs is totally unfair to the actual workers.

Why can’t this be done like in-class pencil-and-paper exams?

I sympathize with victim in this story. The AI detecting anti-cheating tools really seem like quite the racket.

At a certain point, telling if someone "cheated" on a research paper by evaluating only the text of the paper because ineffective because a student can always pay someone else to do the assignment for them. Instead of devising better assessment techniques, they do the same old things.

I've always learned a thing or two after completing a research paper, but ultimately they always felt like busy work. The goal was always the paper, not an actual objective that requires research.

AI seems easy to identify because it writes like a middle schooler doing their homework. Which is kind of a problem when you're a middle schooler doing your homework.

I wonder if a tiny upside could be that we stop teaching students to write like that. The formulaic, vapid mechanisms we teach them aren't merely unpleasant to read. They're opposite to the goal of writing, which is to get a point across. When we teach them to write like Mad Libs, they're not really learning anything about communication.

Per TFA, "98% accuracy rate" is advertised by Turnitin. They don't say what they mean by that, but if that's the specificity, then that means a class of 300 with no cheaters would have 6 false-positives on each assignment, making it not particularly useful for catching individual cheaters. OTOH if it really is 98% accurate and you get 100 positives on an assignment, you can probably figure there is systemic cheating going on.

The only answer is to do the writing in person with a pen or pencil. Or with an armored word processor like Exam4. Or to display knowledge verbally.

There’s no reliable way to identify LLM generated content. Even if OpenAI could watermark someone could download Alpaca and run it locally.

Students falsely accused by the software ought to file small claims defamation suits in their local courthouses.

That kind of decentralized legal action would get costly for TurnItIn very quickly.

Rolling Stone would know a thing or two about false accusations.

Reply on news.ycombinator.com