Teacher failed kid for essay because an AI-detection tool flagged his work | Hacker News Reader

Teacher failed kid for essay because an AI-detection tool flagged his work

twitter.com

33 pointsnotRobot32 comments

Worth noting that OpenAI's own tool for detection tool only correctly identifies AI written text 26% of the time and also incorrectly identifies human-written text as AI-written 9% of the time[1].

Is there any reason to think any other tool (eg TurnItIn has one claiming 98% accuracy!) is actually any better?

[1] https://openai.com/blog/new-ai-classifier-for-indicating-ai-...

Any serious student runs their own work through detection tools before submission. A student who’s work I was reviewing was telling me that the university actually encouraged this.

That is busy work that should neither be necessary nor encouraged.

I’m suppose the depends on who wrote it, the student or an AI.

This is actually common advice given at major universities in the UK. This was standard advice even seven years ago. Students may inadvertently plagiarize, and likewise, they may also inadvertently self-plagiarize. There's a lot of nuance that you may not have accounted for.

If this were available for free, it sounds fine. If not, then I can see this as a problem.

Personally I never found a free tool that did that and would be weary of using any that did for fear of my work being stolen and resold.

Also fuck Chegg. I will never give them a dime after they tried to take over every citation tool on the whole god damn web.

It should also be worth mentioning that AI development is specifically optimizing to reduce the difference between human and AI generated texts. In the end it is simply a rat race that the tools for detection won't win as there won't be a conclusive difference.

Maybe better to just quit essay and writing practices altogether, it will end up in a pointless cat-and-mouse exercise between deception and detection.

Edit: I see this was not a popular take, but I'm open to hear a practical solution, where teachers don't have to waste time grading the work of an AI instead of actual student work.

Teachers are very good at spotting when a paper was not written by a student. Any student they've worked with and graded works from they can identify. When someone changes tone, pattern, word choice, and grammer all at once? It's usually not because of a flash of academic inspiration.

The problem is that accusing a student of cheating (especially in the US) comes with a risk of lawsuit by parents and being forced to "prove it". If it's not clear-cut plagerism but a new novel work, that claim doesn't hold up.

I think the future of this is milti-fold. Don't try and identify AI papers (as you say, a cat and mouse game), just try to identify if the author is the same author as previous works. Students over their academic careers produce many works, enough to identify them. Secondarily, emphasize more short form writing in class, where research is done ahead of time but papers are composed in real time. Last, especially for younger students, emphasize (and grade) the process, not the outcome as much. Show your sources, outline, etc long before you turn in a paper. The value of having ai write it drops if you have to show your work.

Neither party can prove that inconsistency with prior styles means cheating or original authorship. And asking someone to undergo the burden of writing an extra essay just to prove that they can repeat prior output is unfair and capricious.

I can only talk about myself, and I haven't had to do this kind of essay since school roughly 20yrs ago.

I remember pretty vividly that my ability to formulate good essays varied massively between days. (And it's the same now, though it's code instead of essays nowadays.)

There are days on which I struggle being coherent and others on which things just flow out - sentences chain together into a fantastic essay.

The proposed idea would absolutely fail people like me as cheating.

Honestly, this whole debacle was obviously going to happen and anyone with half a brain predicted it when these services (the fraud detection ones) started to pop up.

I've been accused of cheating several times over the years. It's because I don't perform well in a classroom environment. I don't think on my feet very well, so I tend to sound a bit dumb. But if I take a problem away and study it intently, I can produce very high quality work.

I think there's a danger of teachers rating their cheat-detection ability too highly and catching people like myself in the crossfire.

Quit homework essays. Students can still write essays on paper by hand in class. I did that in school during exams.

Homework is inherently classist (and by extension racist) in the first place, and (this i'm way less certain) isn't it largely shown to be ineffective?

This is disappointing rhetoric that I would like to see improved. Homework is not inherently classist or, even by extension, racist. The practice of assigning and completing homework is affected by the conditions that are often associated with (but not always caused by) class and race. The labels of ‘classist’ and ‘racist’ are absolutes, indicating a 1-to-1 causal relationship that is not present here. It would be better to state that homework, as employed by many teachers today, may be (or arguably is) less effective at promoting learning across all economic and racial categories.

The idea behind homework is sound: it gives the student additional study time than is available during school. If additional time studying has positive outcomes, then homework is a positive concept. If some students do not have the time, environment, or support for this added study, it is reasonable to look for ways to support and enable their success instead of effectively penalizing them for circumstances outside of their control.

Using terms like ‘classist’ and ‘racist’ discards much needed nuance.

Why do teachers trust these tools so much when the false positive rate is known to be high?

It's hypocritical because AI detection tools are infact themselves AI tools.

Either way, false negatives are quite high too, all it takes is some prompt magic. Most AI detectors assume AI written text maintains a consistent format and style. Unless there's some probabilistic signature tied to one specific model, there's no reliable way to check that doesn't make the error rate too high for it to be reliable.

Simply put it is because in the US the quality of teachers is fairly low.

For those of you who haven't down voted already I can explain why this happens:

Its a low paying job, the only reason to go into teaching is passion. Its great that folks are motivated, but that doesn't make them GOOD. Plenty of people pursue careers in art and music and fail, few people who pursue teaching get pushed out by "better" teachers.

Give someone a tool that lets them take the easy way out and not have to exercise critical thinking skills and this is the outcome you get.

Ontario has teachers that make 100k+

Quality was still horrible when I was in highschool.

The average is probably more important than the extremes. In my field (radiography) the range is large, and also fairly meaningless. In the small group I do pay sheets for and with the same qualification, the lowest paid is on approximately 1/6th that of the highest paid.

100k is the average

https://www.blogto.com/city/2020/09/average-ontario-teacher-...

Oh wow.

This might be surprising to some. But many people are simply not highly motivated by money. Sure, some amazing teachers simply can't stick with teaching because it doesn't pay enough, but a lot make it work.

10 years ago, at my university, I remember there was an opening for an English associate professor. They had over 300 applicants. I guarantee you there were lots of amazing teachers in that group because they certainly weren't doing it because of the $38k starting salary.

'Those who can, do; those who can't, teach.'

An absolutely horrible, horrible message. The worst part is that this sentiment creates a self-fulfilling prophecy.

"Those who can't teach, teach gym"

I like gpt4 more and more. I'm still skeptical of the AI future technologists want to sell, as hyperbole tends to rebut me (I know it's mostly cultural, but still).

My sister used it to resume scientific papers (she's in her second year, ) and to follow up on the most interesting ones. She tried to use it to write the report too, but it wasn't good enough to do it properly, although she used the same formatting.

She got a lot of praise from the professor, and added a thanks to open-AI, which is funny.

Also it was the first presentation she didn't feel like an impostor, because of the preparation gpt4 did (I'm a bit sad she replaced me, but I'm shit at biology past the 1rst year). Which is great for an ex-chef who barely finished high-school and worked 6 years in restaurants.

Recently I wrote an essay that came back to me as around 20% AI written by turnitin. My teacher failed me, I got 30/100. I already got in an argument with my teacher, and he was adamant on my AI use. I have never used ai, I don t have an account and I don't even know how to work it. I already get called an "NPC" by fellow students but now its proven haha. What should I do though? just willingly fail the class? Obviously not. How do I prove I wrote it? this is a very shitty situation for me, all because of over-reliance on technology.

>i sat there with him to support him through the work as he did it but i guess people are so eager to demonize children they can't finish reading a tweet

Sounds like the teacher correctly identified that the essay submitted wasn't written by the student; they were only wrong about who actually wrote it.

i suspect the only real solution to this problem is to have proctored in-person essays written by hand or on locked down computers - similar to certification/professional testing. i further suspect this will only be offered at elite educational institutions due to cost constraints.

You can also just take computer written documents out of the equation. Write the reports in pen and paper in the classroom.

Although I certainly would've hated writing essays by hand back in the day, there's plenty of scientific evidence that says that handwritten text is better for learning things than typed text. The extra time and consideration it takes to write things out clearly helps memorize the information.

I think we're either heading for a future where schools accept chatgpt like they more or less accepted copy pasting from Wikipedia like back on my days, teaching kids even less actual knowledge, or a future where homework disappears and is replaced by spending more time in a supervised classroom.

Neither are things I'd like to see, and with the way society seems to treat teachers, I'm expecting the education system to give in and go the lazy route.

Reply on news.ycombinator.com