AI Detection Tools Falsely Accuse International Students of Cheating
themarkup.org
themarkup.org
When you ask ChatGPT to write an essay on a topic, it produces a series of well-formed paragraphs. Very often, many of the paragraphs begin with a transitional or signposting expression, such as “However,” “On the other hand,” “In addition,” “Conversely,” “Furthermore,” and, for the last paragraph, “In conclusion.”
Books and websites that teach academic writing to second-language learners of English often explicitly tell the students to use such signposting. Some examples aimed at Japanese learners are at [1], [2], and [3].
While such paragraph-initial signposting is sometimes used in real-life English writing, it isn’t very common. You won’t find many newspaper op-ed essays or HN comments, for example, with final paragraphs beginning “In conclusion....”
Since last December, whenever I see writing with a lot of paragraph-initial signposts I immediately suspect that the author has used ChatGPT. The detection software is probably looking for them, too. It’s important to remember that many learners of English have been taught to write that way.
[1] https://etsjapan.jp/criterion/pdf/essaywordsphrases.pdf
No wonder, these tools just gather word statistics from some large online publications repository.
These conversations should not be graded, just flag for problems (of which cheating is just one).
Combining this with verifying the student's ID is about as good as it gets. I think a mandatory 1-on-1 meeting with a member of staff at least once per term is essential, and serves many purposes.
EDIT: I think oftentimes it's known which students are cheating, but it's not easy to do anything about it.
We're all (understandably) upset if we pay for something much more modest and when it has an issue we're unable to talk to a real human to try to resolve the issue. Imagine paying _many thousands per year_, working on projects and papers, being taught increasingly by adjuncts, struggling to register for all your required classes as administrators admit more students every year (without hiring more faculty), and then when you're accused by a bot of cheating, the response to a suggestion that you should talk with a staff person at least once per term is "oh that's too expensive".
Lecture sizes between European public universities and American universities are very different. While in the US few lectures were "MOOC-style" with most class sizes allowing for 1:1 interaction, in Austria you really generally are "just a number". (There is even some funny German comedy based around students abusing that the professor does not know the name or face to each student ID, many such lectures happen in one of our many so-called Audimax halls)
This might be confusing to an American audience but works due to most classes in Europe having no required attendance and the grade being almost wholly determined by one exam at the end.
One exception is the small Austrian / German private college (University of Applied Sciences) which limits access much more to allow for small class sizes. These usually have much stricter entrance criteria by necessity but once you're in you generally get a much more personalized experience like I had in the USA.
As with everything, ymmv.
Cheating was basically unthinkable.
It seems insane to me to return to something we know is bad just because a new calculator was invented. At least before we figure out how to fix it's known issues first.
That seems like leaping to a conclusion. Could it also be that this is not exactly an exact science and it's very easy to get it wrong?
> but it's not easy to do anything about it.
This is actually prolific and a problem. I caught two students cheating once and their excuse was "I didn't know GitHub was on the internet." This was a junior level course... and the professor's focus was on networking... (he put in the syllabus that you can't use the internet for help. Let's put personal opinions aside (I wasn't checking them against SO fwiw)). The other student couldn't speak a lick on English. Yet, our department didn't want us to do anything about it. Despite being about as clear cut of a case as you can get, they even admitted it. In my other post I mention Goodhart's Hell. The reason the department didn't want to do anything is because the university evaluates a department's level of cheating by the number of reports that they make. So obviously, no one fucking reports it.
But besides this, I think we have to ask why students cheat in the first place. It's because despite GPA not having meaningful outcome [0], they are critical for students so get into grad school or get their first job. No one knows that you took the hard professor, learned twice as much, but barely got a C which would easily be an A in the other course. So of course they cheat. I design my class around this actually. I make it easy to get a good grade on purpose, but get students to work. To do projects, explore, and be creative. If I don't, they just cheat and that undoes any benefit from a hard and challenging homework. I let them know too. But I also am purposefully late to return grades, which makes them sweat. It might sound dumb, but you got to adapt to the environment. My goal is to make students learn. I can't make someone choose to learn, and I don't bother with students that are checked out. But I do make my class engaging and highly encourage creativity, and I've found that a lot of students that previously were checked out end up checking in (I give often give them opportunities to make up their slack, but require more from them). And the students that do work hard, they have good projects in their portfolios which I've seen them use to get hired. Its dumb, but it is the only way I've found to re-align the a class with the intended goals: to teach students.
[0] https://web.archive.org/web/20230107050342/https://www.nytim...
Biases are hard, and you have to be careful to avoid perpetuating them.
*: honest meaning in this specific context I think otherwise, without any accusation or assumption of maliciousness, you're probably right
ChatGPT if not promted specifically has a very particular tone, which you can surely detect. But writing in that careful, somewhat formulaic english can not be taken as evidence of anything.
I can absolutely believe that ESL students tend towards that tone. But that only shows how absolutely useless the detector is.
I keep proclaiming that we live in Goodhart's Hell, and this is yet another great example.
First, you have to remember that metrics are never a complete measure, but they are guides. This doesn't matter if you're cutting lumber or training a ML model. Metrics are __always__ proxies. For the vast majority of things you are intending to evaluate, you don't actually have a measurable function to express that. Every fucking measurement has error/noise, and this is why you measure twice and cut once. It is absolutely insane that in the age of advanced algorithms we've just tossed caution to the wind and we're instead doing a half assed measurement and then getting upset when our cuts are bad, blaming the ruler or blaming the saw.
Second, you have to ask how closely your metric aligns with the thing you're actually trying to measure. Even cutting lumber you have to check that your measurement aligns. Accounting for angles and grain. We should be doing the same in algorithms, but with __much__ more care, but we don't for some reason. All metrics must be continually scrutinized. This is why we measure twice (or more), why we double check our calculations, and we do frequent sanity checks and build in redundancy, because you can't fucking trust your metrics.
For some reason we seem to have this belief that empirical measurements are absolute and generalized. We seem to have this belief that noise is a thing that can be decoupled from the system and precisely removed. We seem to have this belief that measurements are unhackable. What the fuck is going on. This isn't just ML, it is academia, it is bureaucracy, it is the workplace, it is governments, it is economics. So I gotta ask, what the fuck is going on? In the past we treated things as hard, so there was an inherent uncertainty built into mental models but now we treat everything like they are simple to understand and a quick reading or video can make you an expert. We may have never had nuance from non-experts, but now everyone acts like an expert and that math is unbiased and objective. Why do we keep shooting ourselves in the foot and deciding the next best action is to reload the gun?
I also think this is just one my symptom of a much bigger problem: big US institutions trying to automate human things.
I was lucky to be part of a class of 60 (Exeter, UK, Physics). Core modules were a max of 60 people in the class and options ranged from 10 to 30. I knew basically all the professors. The concept of anticheat was anathema to us because people each other. If a student gave in top grade work but couldn't hold a conversation or pass an exam it would have been pretty obvious. This also likely discouraged people from trying.
Now that's not possible in a class of 600...
If even OpenAI thinks such a thing is too low accuracy, how can a third party service like TurnItIn think they can do a better job at it?