Hacker News Reader

Comment by RegnisGnaw | Hacker News Reader

>> People that want to cheat or game the system will find ways, it isn't hard to do.

As someone who taught a college, that's a flawed statement. Let me propose this to you. You teach Calculus 1000 and there is an end of term exam. Your normal end of term exam is one where everyone sits in the same room, proctors are looking for cheats, things checked, etc. Instead this year you tell your students that this year it will be a take home exam with the following rules: 1) they have 2 hours to do it during the team home peroid; 2) no cheating by the honor system.

Do you think the rate of cheating will be the same? I mean by your logic, it should be.

View on HN

I had my fair share of uncheatable open book exams, calculators allowed back in college. One I had on Calculus (or was it Linear Algebra?) was way harder than a comparable closed book exam. You really needed to know the subject to pass vs just memorizing a couple of formulas and plugging them in the right place.

Another professor devised an exam that used your student ID as a variable of the first question, and subsequent questions used the previous answers as inputs. Impossible to cheat.

> I had my fair share of uncheatable open book exams, calculators allowed back in college. One I had on Calculus (or was it Linear Algebra?) was way harder than a comparable closed book exam. You really needed to know the subject to pass vs just memorizing a couple of formulas and plugging them in the right place.

We're discussing this in relation in relation to COVID, so no large gathering. This means no open book in person exams, I'm specifically talking about take home exams.

> Another professor devised an exam that used your student ID as a variable of the first question, and subsequent questions used the previous answers as inputs. Impossible to cheat.

Really easy to do so in a take home exam.

If someone can figure out how to cheat on an exam where each answer depends on the previous, and the original seed is unique to each student, wouldn't it show a pretty thorough mastery of the subject matter?

I think the larger point is that it is fairly easy in any subject to design a test that is very hard to cheat on. It is much harder to find the resources in modern education to grade that test since each submission is likely unique.

Tests that are easy to grade (like multiple choice), tend to be tests that are easy to cheat....

The problem is COVID. You can’t have a large gathering. So everyone is doing their exam at home.

Please tell me how you would structure such an exam without a proctoring system as described.

It depends on the subject. I studied computer science and econ.

Computer science: Solve a complex problem in code. Include a git history. Be ready to defend your program design if I get suspicious.

Econ: Long answer question: Pick 5 concepts that we learned about that you think are most important. Explain them as you would to a ten year old.

One option, if you have a reasonably low student-to-instructor ratio: make it an oral 1-1 exam for each student. A video call with the instructor and the student; you ask questions, they answer. If you have 20-ish students, it will eat up half a work-week or so, which is more work than grading 20 exam papers, but not _that_ much more.

Of course if you have 50 students per instructor this is not going to work...

How do you propose that this should work e.g. in proof-based maths courses? You can't just tweak a theorem to prove by the value of some "unique seed", the theorem might become wrong.

It's true that you usually can't cheat your way through such an exam provided you actually write the answer yourself, but in a take-home situation you can always ask someone else to solve it for you.

You do what my teachers do and have unique problem generation software.

what? how does that work?

You give it a few hundred problem classes, and constraints for possible answers, and it will randomize the class of problems and generate a unique problem as well as calculate the answer. Then you submit the answer as well as your work and it gets corrected.

For things such as proofs, it might give you a problem for which the theorem is needed, then ask you to solve the problem, indicate which theorem you used, and then prove the theorem.

Do you have any example of any software that can generate problems for actual proof-based courses (e.g. abstract algebra)? I'm having a really hard time imagining this, we can't even fully automate theorem proving - how are we supposed to automate theorem generation? And this ignores the fact that you also need to make sure that all the proofs are "of the same difficulty" in terms of fairness.

Ah, no, the theorems would have to be manually selected. But given a high enough number and an automatically generated context it makes cheating much harder.

> had my fair share of uncheatable open book exams, calculators allowed back in college. One I had on Calculus (or was it Linear Algebra?) was way harder than a comparable closed book exam. You really needed to know the subject to pass vs just memorizing a couple of formulas and plugging them in the right place.

You can always just pay someone to do it for you.

The idea that an honor code is enough is belied by the fact that software like this catches cheaters, no?

I mean, we're obviously upset about false positives, and we should be. But I'm presuming that some people are caught who were cheating, and without the software they would have cheated and not been caught.

We can suggest that with an honor code in place, maybe some of those cheating students would have not cheated, because... I mean, if they were willing to cheat with software in place, I'm not sure why would have been deterred by an honor code.

I think in about 98-99% of cases, people who claim an honor code prevents cheating are deluding themselves. Yeah, if you don't have any way of catching cheaters, then you can pretend you have a 0% cheating rate. But it's pretend.

P.S. I'm not speaking for my employer at all.

It's not certain that the software catches cheaters. It could be security theater. Even the false positives could just be to make people nervous.

Where would they get reliable training data?

There's a pile of money to be spent on this stuff, and virtually zero accountability. What's that a recipe for?

I have taught at multiple honor-code institutions (and still do). It does not prevent cheating. However, it shifts focus: I can go about my teaching life starting from an assumption that students are not cheaters—and I'm personally convinced that most aren't.

The flipside is that when you do catch a cheating case, you completely throw the book at them. It's legitimately easier to cheat under an honor system, if that's what you're wanting to do... so my assumption is that if we catch you at it, it's likely part of a pattern, and if we catch you multiple times the pattern is irreformable. It is not uncommon at honor-code institutions to expel students on the second offence (sometimes even on the first).

I do think that cheating is less prevalent at my institution (and my previous institution) than it is in the larger university population.

I perhaps wasn't clear enough but I meant solely when it comes to these type of anti-cheat systems being used. Obviously it would be different for in-person vs remote/take home.

I don't see why we should allow for the possibility that the ratio of cheaters would stay the same with or without the surveillance. That could only be true if the ratio of potential cheaters were so low that the threat of detection introduced by the spyware can't reduce the ratio further.

We may hate the software on ethical grounds, or because it degrades the exam-experience on many levels, or because its use can be considered abusive, but obviously it has an effect in the intended direction.

Reply on news.ycombinator.com