Hacker News Reader

Comment by a_bonobo | Hacker News Reader

TurnitIn's cheating detection tools have always been garbage - and I say that as an ex-academic who was 'forced' to use those tools extensively (the PDFs submitted by students are only visible in the Turnitin portal with the 'flags' to the right).

Even before the 'AI detection' tool (IMHO snake-oil) the features around copy-pasting/plagiarism were so bad I always ignored them. The tool would flag things like commonly used coverpage forms. That universities pay a fortune for those tools is an indictment of the industry.

View on HN

The one student who got caught for plagiarism when I was a TA was busted by turnitin. Maybe the tool isn't perfect but it caught this student who had painstakingly replaced every single word in a copied paper with a synonym. I'm talking "perhaps the utility is not flawless, but the use of this utility allowed for the apprehension of a pupil who..." etc.

I was surprised at the level of effort

So it either only caught a single cheater among some unknown number, or only a single student cheated? Neither case seems to justify the expense and hassle of subjecting everybody to this process, not to mention everything else wrong with TurnItIn.

I'm always shocked at how much students will put in effort simply not to do an assignment given. I guess this is just part of what you will get when you attach cultural and economic worth to a set of hoops to jump through [even if the intent is to have institutions of knowledge, sadly I think many students just see it as a way to get economic stability].

Unrelated to the cheating part, if you don't see American colleges primarily as a path to economic stability, I think you are probably either rich that it doesn't matter or haven't seen the bottom 25% outcomes of college graduates who didn't treat it as a financial opportunity/investment and are struggling.

More likely that they used something like quillbot or grammarly for automatic paraphrasing.

This was about 15 years ago

grammarly is pretty old, Not sure when it became available, but it was already pretty well established as a high quality writing assistant by ~2012.

It had various tools and heuristicsv to suggest different phrasings and better words that were pretty incredible if compared to anything else back then

I never once submitted an assignment on a course with TurnitIn turned on, it makes you agree to their terms of service, which I disagree with and I'd just sed my work to the teacher through email, explaining what was my issue with TurnitIn, and most times they'd just turn it off, which I think speaks loads about that "tool".

They're 92% accurate. Sounds good until you realize that's a false plagiarism accusation in every 20 person class.

As a current academic who uses Turnitin regularly, I find it works fairly well. There are a lot of false positives, but it also regularly identifies actual plagiarism for me.

It is a tool like anything else. From student perspective I perfectly understood reasoning for it and how useful it would be when person doing review actually cares. Being able to easily find most obvious things and then cross-reference is very useful.

Problem comes when users do not care and blindly just believe in percentage value and some threshold

"lots of false positives"

Maybe this is okay when we're talking about plagiarism and you can review manually, but humans can't reliably identify LLM written text

Totally agreed. I'm personally only relying on it to identify cases of material which was directly (with minimal changes) copied from a third party source without attribution.

For what’s worth (for you) the results would be the same if it identified all of them as plagiarism

No, it would definitely not. There are a number of false positives, but significantly less than all submissions. It also specifically highlights areas of concerns along with a cross-reference for me to review. I would not be able to do all this manually.

Reply on news.ycombinator.com