If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?
FWIW if I were a student I would definitely be using Track Changes or version control, etc etc, to make clear my work was human-written. Which sucks.
I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested.
And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process.
You need good tools and good oversight.
Plagiarism and cheating sucks for everyone. Worth solving.
Agreed, that's a fair and reasonable stance.
The reason I asked is that I have a hard time understanding the point of these tools. When it comes to education, it can be a matter of learning objectives. But outside that, what's the point?
The prediction from the tool is pointless for deciding on copyright or contract issues, and other text should be judged on its correctness or applicability to the task.
If all the tool is good for is "maybe this student cheated, but only an in-depth investigation would maybe prove it", it isn't a very useful tool, because it's more straightforward to just mandate that evidence is submitted regardless of what the tool says. On top of that, even the lack of evidence of manual work isn't good proof of using LLMs.
But yeah, in general I think you’re right, the actual utility is pretty niche.
>> Not true at all. Pangram is highly effective and has a very low false positive rate.
> So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work?
What point are you arguing? Something having a high success rate does not necessarily translate to treating it as a 100% success rate.