Hacker News Reader

Comment by caconym_ | Hacker News Reader

Are students really comparing their work against the scores generated by these tools to game the scores lower because they're worried about being punished for false positives?

That is, pardon my French, fucked up.

Beyond that it's egregious that any academic institution would allow the success of students/researchers/whatever to hinge on scores generated by a black box algorithm that hasn't been rigorously proven as very close to 100% accurate. There should always be a human in the loop, and if we don't have enough humans for that then I think we have a problem that goes beyond automated plagiarism checkers.

View on HN

In my experience, that's not happening.

First of all, if your work is submitted through these detection systems, you almost never have the opportunity to see your 'score' and then resubmit. That's just asking for people to cheat the system.

These plagarism checkers are also not really smart enough to differentiate proper citation from actual plagarism. Thier function is to alert the person grading the papers to sections that might be problematic. It will highlight "plagarized" sections, but the professor will ignore it if it's properly quoted and attributed or if it's just a short phrase, etc.

What stops me from renting the software and running my own paper through it to see what it says?

They usually only grant licenses to certain organizations. Same reason you can't simply buy the solutions manual to any given textbook.

Well hey, there's a business opportunity!

Quite often you can buy the solutions manual to a given textbook. They are on half.com or available in pdf format, a lot of the time.

They won't show you the work, so copying the answer is not sufficient to get credit in practically any course.

"Academic integrity" has ridiculous rules (eg self-plagiarism is not really a thing that anyone should care about) and I'm sure some people would be pissed about that, but if you're doing the work I personally don't see anything wrong with getting immediate feedback on whether you've done it correctly or not. Particularly given that many textbooks do like one or two simple examples and then jump to complex problems in the assignments... being able to check a couple problems to be sure you understand the process is extremely helpful as a learning aid.)

If you want a real-world assessment of whether someone has mastered a concept, schedule an in-class quiz or a test.

> self-plagiarism is not really a thing that anyone should care about

Sure it is. The entire methodology of schooling is to learn through exercise; if you are asked to write a paper, it isn't because the teacher wants the information in the paper. They are wanting you to learn through the act of writing the paper.

If you just turn in a paper you have written for another class, you are not doing that exercise.

Turning in a paper that you wrote for a different class and getting upset that the school doesn't approve is like thinking that sending a picture of you lifting weights 3 years ago is going to make you stronger.

All of this nonsense comes one root evil: giving publicly viewable grades for fundamentally in original work. Grades and homework should be for the student's private benefit. The only public marks should be proctored exams for demonstrating skills, and whatever original work someone creates that has value that stands on its own outside he artificial school environment.

That all depends on what you think school is for. I subscribe to the idea that school is for preparing people for society and provides a "great filter".

Doing a task your superior asks of you and doing it well is arguably the most important thing school teaches you, with time and stress management being a close second. This is what grades at school measures.

People love to talk about the virtue of being "well rounded" but the truth is that knowing bits of Shakespeare or knowing different types of Greek collumns doesn't benefit you as a person. All that information leaves your head pretty much immediately anyway. This is all especially true of high school. "Exams demonstrating skill" are not what matters to the people who look at your grades.

A proctored exam can only really measure a few types of skills, and don't work very well for demonstrating skills learned in many classes.

Take, for example, a class that is about learning to write effectively, which is clearly a valuable skill that adults need no matter what they do in life.

Part of that skill is being able to construct, from scratch, an effective piece of writing in a length of time that is longer than a proctored exam session. You can only measure that skill by having them do that, and if they turn in work they have written prior, you aren't learning anything about their ability to write something effectively on demand.

Research classes are similar; you can't demonstrate that you know how to do research well in a 3 hour exam. Programming classes, too; you can't measure the ability to create a large project in a 3 hour exam.

I think a defense (oral examination) of the paper can cover that scenario.

"They are wanting you to learn through the act of writing the paper."

And you did, only earlier.

"is like thinking"

Funny. Do you also think that writing exactly the same paper many times is making you learn something each time?

Self-plagiarism can also be an issue in the real world. Sure, I've cut and pasted an explanation of something into a different white paper or other docs. But there have certainly been times (don't remember the details) where journalists being paid to write a new article have taken a hit because they copied from an earlier piece. It's very situational but self-plagiarism can absolutely be a thing.

You can't just "rent" the software. They're going to try to verify that you work for an edu. That's way more hoops than even a semi-motivated plagiarizer is willing to jump through.

What stops me from buying acess via a sketchy edu?

It's work.

Now hold on a minute, this whole industry has developed around shirking work. So surely, i can automatize part of this with a webcrawler and sci-hub.

Eh, I'll just wait for someone else to automate it and use theirs.

Men, i would con some Venture Capital guy into building this, by giving the ilusion somebody else has already built a version, but im just too lazy. Would hire some students, but those would just copy paste it together from stack overflow.

    > That's way more hoops than even a semi-motivated plagiarizer is willing to jump through

If you're willing to go through this amount of work, why not just not plagiarize

If you were plagiarizing you could compare your paper with your sources. Might not be a bad idea even if you aren't to make sure you didn't fuck up and cut and paste something from your sources.

https://smallseotools.com/plagiarism-checker/

I mean, yeah, there should be a real person in the loop. But the issues are arising from when these types of tools aren't being used properly.

It's not a problem when a professor is checking mid-terms from an upper-division class of 30. It is a problem when a grad-student with too little time is grading finals for a freshman class of 500.

Sure, the purveyors of these tools can say: "Use this thing properly! If you don't, bad things will happen." But time and time again, bad things keep happening when people don't use the tool right. After some amount of incidents, the purveyors of the tools should just re-design the dang thing [0] such that it's harder to use it wrong.

[0] Provided that the people that make the tool have any sense of morality and put human learning above dollars.

That's pretty much my experience too. A good paper will use lots of citations, and hence have lots of flags from these services (eg turnitin), but this doesn't mean plagiarism - it means the student's doing the right thing: gathering supporting (or negating) evidence and quoting it for the purposes of argumentation.

I think the problem is the testing methods they may use to verify the service.

They pass it the same text, it comes back as 100% plagiarized. Cool.

They pass it text that is close, but clearly just slightly modified original text. Comes back as plagiarized. Cool.

They pass it text that isn't close at all. Comes back as not plagiarized. Cool.

It works. Right?

Nope. We forgot to check if there are any cases where two different texts will show as being plagiarized despite knowing they've been independently created.

We also didn't check to see if a text that was essentially plagiarized can return a result of not plagiarized.

And I think those checks fall into the halting problem. You only need to find one case to throw the entire system into doubt. But to make sure there are no cases, you have to check every piece against every other piece.

And if you have false positives and false negatives, you can either double check every result. In which case, you might as well not use it. Or you give up on one the case of false negatives. If a plagiarized paper passes the system, oh well. Focus on checking every paper that gets tagged as plagiarized. Double check them and make sure it truly is plagiarized.

"And I think those checks fall into the halting problem."

Metaphorically, if you mean that there isn't going to be a concrete metric that really determines whether two bits of text are "similar" with bright shining line, sure.

If you mean halting problem literally, no, this isn't affected by the halting problem. String similarity evaluation won't be strong enough to invoke the properties that result in a halting problem. For any two strings, you can certainly terminate in a reasonably period of time, and have the "correct" answer as defined by whatever the local meaning of "answer" is. (Barring a particularly perverse implementation of a string similarity algorithm probably explicitly constructed just to screw with my statement here, that for some reason starts executing the string as instructions directly or something.)

Reply on news.ycombinator.com