It's actually very disappointing that this whole monitoring system was deemed necessary to begin with. How screwed is the system that anyone one would believe this level of intrusion should be part of the education process.
I think at the very least, all exams ought to be designed as "open book" with a time limit. that at least eliminates the issue of students using unauthorized resources. I'm not sure what can be done about students who collaborate on exams or go so far as to hire someone to take it for them. perhaps a few students could be randomly selected for each assessment to explain a few over their answers over zoom?
No one wants to hear this answer because it means actual work for the teaching staff (compared to, say, automated scoring of multiple-choice tests), but it is, in fact, the answer.
It's not just "actual work", writing high quality tests and projects and then grading them is multiple full-time jobs worth of work. It's equivalent to interviewing 100 job candidates every week, forever.
Projects they had to demo, and they had a published rubric. While it was possible sometimes to play to the rubric, it still meant there was very little work to figure out a grade.
Tests, on the other hand, I had to dig in, understand what they were trying to do, figure out if it worked, and award partial credit. I guess the prof could have done multiple choice to make grading easy, but that has all kinds of problems with actually testing anything.
The fact that the university doesn't provide adequate staffing is not an excuse for short-changing the students still further.
Actual work for the already incredibly underpaid, over-worked teaching staff. As the money dries up in university funding, teachers are stretched thinner and thinner. That forces them to rely more on automated testing instead of having the time to actually know more about their students' performance. That in turn makes it easier for students to cheat.
The economics (as usual) are perverting the system. The individual participants are merely suffering from the perserve incentives of the system.
So I think I actually understand the system pretty well.
Edit: teaching is part of the job, dude. It just is. If you asked the average taxpayer (you know, the ones who pay the base salary before grant money is taken into account) whether teaching is an important part of the job, what do you suppose they're going to say?
If you don't want to teach, you need to find a job that doesn't involve teaching. If you accept a job that requires teaching, you're morally obligated to provide the best teaching you can.
Or so I see it.
1) Version control all assignments. I force them to make periodic commits throughout the week. If they want to copy a project wholesale, they have to do so in a way that takes a lot of time and effort.
2) Comparing bytecode. If they change the names of variables, comments, spacing, etc. these superficial differences will be lost in compilation.
3) Oral explanations. Now that everyone has guaranteed access to screen recording software, I force my students to record an explanation of their work. They go through their code line by line and explain it to me. I can tell very quickly when a student is explaining code they haven't written. If they are explaining code they haven't written to a satisfactory degree, then they've at least demonstrated they've learned something.
4) Remove incentives to cheat. Give them all the resources they need and more than enough time. Provide easily accessible venues like chatrooms and forums where they can ask questions anonymously. Provide opportunities to improve prior poor grades so they feel like failure is okay.
This doesn't eliminate cheating 100%, but from my experience it does seem to cause cheaters to fail, as their cheating does not pass for acceptable work. The students who earn an A come to see me during office hours and I know they are doing the work. I would say very rarely does someone earn an A who is not on my radar as being an obvious high-performing student. If a student is cheating and earning an F or D, I can't say I care much about that.
Notice none of these methods involve an invasive surveillance regime. I don't require cameras on at all time, 360 degree views of work areas to prove no one is helping, software to monitor tabs and processes... all this is completely futile. The students hate it and it incentivizes them to find ways to thwart it. They feel justified in doing so. I saw one post that advocated running a high speed fan to drown out typing noises, wearing reflective glasses in a dark room to foil eye tracking software, putting Vaseline on the camera... all to what end? To cheat on a psych exam? It's an arms race that's not worth fighting. The solution here is to adapt teaching methods with new technologies and testing methods.
Not everyone does projects over time, not everyone commits all the time. I don't like committing in progress work and I usually do work in big chunks with large breaks in between, especially for solo projects where I don't have to share code regularly with others.
Based on your criteria, I'd likely be considered a "possible cheater" just for having a different style.
Regardless though, my students usually are at a point where they don't have a commit style. They come to my class with no knowledge of git or version control -- I have to teach it to them.
Secondly, labeling you a possible cheater doesn't condemn you to anything, it just puts you on a shortlist of assignments to examine more closely.
Thirdly, this style of committing has helped more than a few students out when their computers die mid assignment and they need to move to another computer. It also allow me to help them with their assignments. When they come to me for questions, the first thing I ask is "have you pushed everything to your repo?" Instead of trying to debug over a virtual desktop connection, I can run their code locally and get to the cause much quicker in my own environment. It really works out well for everyone involved.
Question: if (when) a student hands in an assignment without the periodic commits, what happens? How do you measure & set expectations for sufficient frequency of commits? (i.e., 1 single commit is too few, I assume that once a day is enough, but where's the threshold?)
Thanks again for sharing - I'll be thinking about these for my own classes!
In terms of getting them to follow the commit guidelines, I give them a carrot and a stick. The carrot is, if they start N number of questions half way to the due date, they can get a penalty-free extension if they need more time to complete the assignment. This is to incentivize students to start early. I've found one of the biggest indicators of successful students is that they start assignments early. Students who start their assignment on the due date usually run into blocking issues (software not configured properly, computer not working right, network is down) that would be trivially overcome if they had started a day or two earlier. Students who have this habit quickly correct within an assignment or two.
The stick is by the midterm, I start requiring it more strictly by subtracting points for not following the guidelines. I think of it like a math assignment; if all you provide on a math exam is the solution, you'll get little to no credit for that. Showing your process is part of the work in math, and I treat it the same way in my classes.
I think it could possibly scale to larger classes as long as the grading team scales with it. I'm not sure how public universities handle their grader allocations.
Every semester, students fell into the same buckets. - Those I rarely saw who nailed everything (and were getting an A or a high B because they understood the material), - Those I saw constantly who were going to get an A or a B (because they sometimes didn't understand things from class, or when they did, wanted to make sure; or for a few, understood it all, but knew the importance of showing they cared about the material to the professor and TAs) - Those I saw constantly who were going to get a C or a B (they rarely understood from class, but were willing to put in the work to try to) - Those I never saw who were going to fail (they didn't show up to anything).
It was pretty much invariably this latter group that got caught cheating. That's not to say the other groups didn't, but they at least got working code, did really well on the exams (pre-COVID, proctored in person), and in conversation were able to show they understood, so I'm a lot less concerned about them.
The best however, is a 1:1 code review with a TA or Prof. Randomly jumping to files, reading the code and asking questions about it. Extremely hard to copy a codebase and learn it well enough you can explain it but really easy to do if you wrote it yourself.
re the 1:1s with TA/professor, I agree. if you can explain code that you didn't even write, you might be even more deserving of that A.
Sure the later can copy their assignments. But once they'll hit a non-trivial assignment or a code review in a more advance class they won't be able to fake it anymore. You are just delaying their transfer to another major. And that gives the CS department one extra course they can collect tuition from and justify hiring TAs for...
The sky's the limit when you think about it really.
It isn't a minority. It is an overwhelming majority if they think they can get away with it.
https://www.cleveland.com/metro/2017/02/cheating_in_college_...
https://www.cbc.ca/news/opinion/opinion-academic-cheating-1....
Maybe skills tests in an interview process is not such a bad thing.
As an employer, I don't care what piece of paper you have from what school. I only care about how well you can do the job I need to be done.
I'm fine with this in theory, but lots and lots of places bungle it so badly that they're no longer testing programming skills, they're testing rote regurgitation of memorized code under pressure. If your job legitimately requires that, for everyone's sake get out of the industry.
Personally, I think a degree doesn't say much about a person's ability to do what is required in the job.
Maybe we should just get better at giving skill tests in interviews.
Testing whether they can remember a specific leetcode question is worse than checking for a degree.
Is that controversial?
To be blunt, duh. I agree degrees are very imperfect & crude indicators of what you're try to measure, but what is your alternative? I'm getting DoS levels of applications blasted at me, how do I filter to an effective level that removes the same cheating mechanics mentioned here?
Not defending Protorio here, but I didn't really understand how rampant cheating was, either, until I became friends with someone who works at a University.
Growing up, my friends and I would never dream of cheating and would fear the consequences of getting caught, so the entire concept feels foreign. We also grew up in the era of in-person classes, before cellular phones were common.
Some of the cheating stories I hear from my friends in University are mind-blowing. Everything from (college) students thinking the proctor won't notice them using their cellular phone during tests, to people trying to hire test-takers who show up with fake IDs. They had to start giving proctors photos of each student because checking IDs wasn't enough any more.
The situation is much more complicated than it sounds. As soon as one person gets away with cheating, it becomes a cheating arms race as even the good students feel the need to cheat to keep up. If an entire institution becomes notorious for rampant cheating, the value of every graduate's degree goes down.
Obviously the Proctorio solution is terrible, which is why I expect we'll see Universities push to return to in-person classes sooner rather than later while keeping remote as a 2nd class option.
That's basically what happened here: The school expected students to cheat and looked for solutions to address it.
I think it's very important that universities have very significant consequences when students are caught cheating. Cheaters don't generally start by hiring people to write their papers and take tests for them. They test the waters with little cheats here and there, pushing the boundaries over time.
If Universities made examples out of students who were 100% confirmed cheating beyond reasonable doubt, the amount of cheating would decline significantly. Instead, we're stuck with this game of half-baked anti-cheating systems which some students approach as game.
I don't think that actually follows. As I understand it, many universities and zero tolerance plagiarism and cheating policies. The problem is, when you have no middle ground and your policy is extreme, you leave yourself very few options in the cases where you don't want to or can't easily expel the student. If your policy is expulsion for cheating, then not doing so and providing a lesser punishment is seen as favoritism if the student has resources, or is used as ammunition for their own case by students with resources if offered to a deserving student otherwise.
I think the actual solution is well defined, less flexible, but not overly harsh punishments. Any student caught cheating or plagiarizing is not expelled, but immediately either fails the class or drops two letter grades (and a second time in the same class would be failing the class no matter what if the more lenient option was chosen initially).
With extreme consequences teachers are going to be hesitant to report small and borderline cases, because the consequences are so large. Make the consequences manageable for the single occurrences but problematic if they keep happening, and you'll correctly catch those serial cheaters and those occasional ones that are better off just taking the hit (and maybe dropping and retrying the class) instead of letting them slide because the punishment is disproportionate to the crime.
Think of it this way, if the crime for stealing a candy bar was life in prison, would you call attention to the person next to you that you just saw steal a candy bar? Does the store attendant actually call the police, or just take the candy bar back and tell the person to leave? What if the person is rich, and it actually causes you problems to turn them in, because it's worth it to them to make sure they don't suffer that major negative consequence be exerting their influence? If the punishment is seen as disproportionate to the crime, people will make their own decisions to avoid what they see as a problem with the system, and it also means that people with resources are more likely to exert those resources to avoid those problems, to the detriment of those around them (and they'll mostly get away with it, because who wants to die on the hill of making sure someone is punished for something so inconsequential?).
When I went to school they made a big deal out of the student ethics board, and if you were caught cheating you were brought before it. It was implied that you would be kicked out of school and your cheating conviction would be appended to any other school's request for your transcript.
I don't know how prevalent cheating was at the school.
Completely agree. For many students their lives and futures will literally be determined by the result of a single test. Under these circumstances, not using every resource available to win is simply stupid. Why would someone not cheat if possible?
The truth is academic integrity was doomed the second degrees became necessary for employment. People who care about academic integrity are those who are there by choice, not those who are pursuing a degree to improve their chances in the job market. The only possible solution to this problem is to end the "everyone must go to college" meme. Good work opportunities that don't require decades of education would go a long way.
A friend of mine copied an assignment to the point that he even included the other person’s name in the code.
He got 100% and a note telling him not to copy again.
I can understand why it happens though. For the professor to take it further, they'd have to escalate it within the school, involving a whole load of bureaucracy, meetings, production of evidence, and likely a formal hearing with the student, etc. Unfortunately, there's often no incentive or institutional support for professors to do any of this.
Having taught at several institutes in the Anglosphere, a disproportionate amount of cheating I've witnessed has come from those who speak English as a second language. Difficulties with English and the need to 'save face' probably push otherwise honest students into doing this. Which also says something about admission tests. It's something of an open secret that administrators turn a blind eye to suspicious admission results, as enrolment and tuition money are a strong incentive.
It's not an easy problem to solve, but I suspect proper admission filtering would eliminate the bulk of cheating, and the remainder could probably be dealt with by empowering professors.
If the professors or TAs are paying attention, they already know who the cheats are. It's not hard to spot when weak students suddenly start acing tests and mastering skills they didn't understand the week prior. There needs to be more leeway, institutional support, and incentives for academics to challenge suspicious students, e.g. via oral examinations.
Trying to solve this using software is just an attempt to avoid tackling the real problems.
Money talks. TOEFL (Test of English as a Foreign Language) scores tend to be treated as gospel, even when it's clear from emails that the student has nowhere near the level of proficiency indicated by the test results.
...and yet I never thought to wonder how they managed to keep passing exams.
Maybe some institutions are simply selecting their students wrong?
Pre-pandemic I would give them a paper exam in which they had to read and identify bugs, or write code by hand. This was the tradition. Now they actually use their tools to the best of their ability, and I get to see what they are capable of.
You might think this testing paradigm would result in everyone getting 100%, but that's just not what I've seen. The distribution of grades the last exam was normal with a mean of 80%, pretty much in line with what I get during a typical semester. As we transition back to a classroom environment, I don't expect to move back to paper-based CS exams. They are just wrong on so many levels.
I've had professors that did more damage by 'teaching' (I use that term loosely) than if they had just said 'learn it from the book/online and show up for labs/tests/exams'. Mostly it was because they had no idea how teaching worked.
It's a very faculty-centric view because from a student's perspective, the teaching is 99% of the school. Very few will go on to research. They come to school to learn and earn a degree. If the University cannot provide good teachers, they are not a good university. If a professor cannot teach, they are not a good professor. That isn't a moral judgement, its just the subjective judgement from the other side of the school-desk.
The distribution is one thing, but grade correlation with other statistics might be more revealing. In particular, how do you know that wealthier students aren't hiring people to take the exam for them?
The other thing I do is I don't make exams so weighty. If you ace the exam but you don't do well on the assignments, you're not getting an A in the course. So the hypothetical wealthy cheater will have to keep their body double on retainer for the semester.
But I will say that the students who earn an "A" are ones I interact with fairly regularly. "A" students typically come to class, ask questions, engage with the assignments, start their assignments early, etc. And before anyone accuses me of playing favorites, I don't do any grading, and grading is anonymized. But I can easily predict the "A" students just by their work ethic. The idea of the brilliant student who doesn't come to class and effortlessly breezes through the exam is as far as I can tell after teaching thousands of students a myth. At least at my university. Maybe all of those kids go to MIT.
The university will pay repetitional consequences for claiming that the student learned particular things which they clearly didn't.
Or are they cheating at the "take these courses to become a more rounded person" stuff, like the insect biology courses for the CS students?
Still, you're paying a fuckton of money to supposedly learn stuff in college. Cheating your way through is a huge waste.
I maintain that cheating is almost always a pedagogical problem first, and a trust problem second.
Cheating becomes a convenient solution to a problem when you're dealing with a course with inadequate teaching, a difficult learning curve, or a lack of motivation for students to do their work to the best of their ability themselves, or a nonsensical curriculum. Fixing cheating doesn't involve surveillance - it instead involves removing the incentive structure that exists for cheating in the first place. This may involve rethinking grading, or course material, or assignments; but is certainly not impossible.
We act surprised when students "cheat" in CS exams that's expected to be done with only pen and paper - nearly any real workplace will give you an option of a text editor or IDE of your choice. So give them an IDE! Give them the API documentation! Don't create an incentive to test the waters to fix the broken rules of assignments.
Another relevant area of work is ungrading, or self-graded courses in general - when you remove the friction that grades cause in the feedback loop of learning; learning becomes an organic process for everyone involved. There's a lot of interesting pedagogical research, and just "cheating is rampant" doesn't scratch the surface of "but why is it?"
In addition, cheating is a game. Every second you spend drumming up cheating in front of your students is another second they think about trying to get away with cheating you. If you tell students they're not to be trusted, they will not give you any reasons to trust them; in many cases it's as simple as that.
A combination of good pedagogical design, and building a relationship of mutual trust with your students, is certainly more fruitful than creating an academic police state (of which Proctorio is only one part of). There will always be people slipping through the cracks, but there are other safeguards in the world to catch them too.
Another important thing is that conversations about cheating always assume a very specific framing of higher education - that they exist primarily as a gatekeeper or arbiter of who-knows-what; the university also has the purpose of providing an environment for learning. And in many cases, cheating is just a result of a failure to provide that environment.
In addition, if the primary beneficiary of university degrees are the employers (or the people who care about the who-knows-what stamp), then why do students foot the bill for tuition? If you choose to accept this framing of universities primarily as arbiters, isn't access to a degree just a head tax to enter the skilled labor market?
Higher level education is largely voluntary, and it's up to the person taking it up to decide what they gain from it. If they are only in it for a diploma, they'd get there one way or another.
To me, focusing on finding "cheaters" makes education a competition. I never felt cheaters got anything over me in my studies, and I never felt like I've got a lesser grade because of them. Does that happen in cases where you've seen "exam-sitting-for-hire" in action?
Even if you normalize your grading scale based on the students taking the exam right then, if your claim that "most students are honest" is true (and I believe it is), that should not affect any non-cheater significantly (unless you've got a small, non-representative group, but then normalizing grades is unfair to begin with).
So my question is: who are we trying to solve the problem for? What is the expected outcome, knowing that there will always be people who "cheat" their way through life too?
You might have worked very hard for a diploma from University X, but if it was found that University X was handing out diplomas like candy to other students that were not as academically rigorous, it weakens the value of your diploma.
It’s basically up to the instructor on how to run the class.
Some require proctorio, some are on Zoom with lots of TAs to watch the screen, some issue an exam on your own time where the browser monitors what you do / visit, and others just offer take home projects.
Handing over the solution for the problem to a software company shouldn't be an accountability shield for the Universities. Impacted students should sue the universities, and then they should sue the software company.
While I am against these surveillance software, making sure that tests are fair is very hard, especially since everything is online now thanks to COVID.
A sibling program at the university exposed a cheating ring of ~15-20 people, and I think many were surprised that the result was to simply zero their grades for that class instead of more severe action.
/e I mention incidents and sizes since these programs are not thousands of students but tens or hundreds.
It sounds like the definition of cheating was very broad there.
I think cheating is quite common in the undergrad years, but gets less later.
As the original commenter notes, here the problem is being solved for "well-behaved" students, in a way which is easy enough for dishonest students to bypass and present themselves as well-behaved.
And suddenly, privacy invasion gets off the table quickly (as soon as you are not aiming for 100% non-cheaters, the cost becomes obviously too high for everybody else).
As far as reputation, I'd rather see schools focus on the successful students, which is somewhat done with all those research-paper-grading systems (not a perfect system by any means because of gamification, but at least idea in the right direction), but mostly done with bragging about scientific break-throughts to come out of their students and staff.
Still, what is the purpose of a reputation or "well-respected degree"? The goal should be knowledge and applicability of that knowledge to actual problems in life (known as "jobs"): it's not like anyone accepts any graduate without interviewing them first, which is to say that nobody trusts any school to have done a proper job of evaluating them. Most of those schools don't trust themselves, so they hold interviews for post-graduate studies too! :D
Nobody looks at the "lemons" coming out of a school to consider it a bad school (I am sure you can find plenty from "top" universities too), but on the successful ones. Do the successful ones change with more cheaters at all? (Sure, there is a turning point, but catching all of them is meaningless)
[1] There's quite a bit of research on this. Bretag is a key author.
> Unfortunately custom projects and the like won't work for teaching fundamental maths skills.
I am not sure I agree. One of the projects I did for my projective geometry class was to do an inverse of a projected drawing (in AutoCAD) and save a 3D .obj file. In differential equations course, I was given a project to prove a theorem that is generally missing from the school books (or usually given as an exercise). You can also let people devise proofs in a closed system of axioms and a few theorems.
It is a hard and different type of work, but I am certain you can both teach and get to know students with theoretical math projects. If they get someone else to do it for them, yet they are able to present it in front of the class (online) convincingly, they have likely understood the concepts, which is what teaching is all about.
> So my question is: who are we trying to solve the problem for? What is the expected outcome, knowing that there will always be people who "cheat" their way through life too?
For one thing, all the harm that incompetent professionals can cause - and in a moddern society, the scope of that is not inconsiderable - in fact, Proctorio is probably an example.
Let's not lose focus on the real problem here, which is Proctorio being an aggressive vendor of garbage.
I would go so far as to say that the information age has made educational material so widely and easily available that the only value universities provide is signaling and wealth/class filtering.
Perhaps there is value in knowing someone has been able to work in a team setting and complete given tasks.
But there is another way to look at it: can all non-cheating students build bridges, design microprocessors or design rockets? I can give you an affirmative no.
coming soon to a tech interview near you.
Higher level education is largely voluntary, and it's up to
the person taking it up to decide what they gain from it.
If they are only in it for a diploma, they'd get there one
way or another.
Then again, I have a hard time envisioning what would someone cheat about in a legal exam, other than not memorizing the things, but that's to me just a signal that the content is badly presented (instead, put students in a pretend courtroom in a case that covers the study material, and they'll have to learn it, and learn to apply it).
But even if we accept it as so, it is not an argument to be so vigilant in catching cheaters majoring in other subjects.
Think of it another way. If you knowingly contract a member of the KKK to do your hiring, you can't pretend not to know why people of color don't get hired in your company.
Proctorio's issue may be more subtle than the "in your face" example I gave above but they're there and whoever contracts them does so with full awareness of them.
P.S. Because I'm sure the wave of downvotes is less about people supporting racism and more about ignorance, let me further support my point about such tech with real life examples:
https://www.vice.com/en/article/g5gxg3/proctorio-is-using-ra...
https://www.theverge.com/2021/1/28/22254631/university-of-il...
Why would you say that this is evidence of a 'screwed' system?
So your preferred solution would be to just stop all education for the duration of the pandemic? Although online education sucks, I'm not sure if most students would agree with you.
Proctorio reflects the cost cutting, hyper systemization of education, which is a trend that— every time I see it— seems to be antithetical to actual education.
Or in physics (my field), I do want a student to know U=RI. Not google it. Not look it up. I also want them to be able to transform it to U/R=I.
Any of these anti-cheating systems are trying to cut costs by enabling weaker teacher-student interactions.
What a scam
It's all a lot of smoke grenades to cover for:
- an education system/process that at its core is not about actual learning but "bulimia learning" aka memorizing the facts the profs deemed relevant for the exam and forgetting them the very second the exam is over to make space for new useless stuff that will be forgotten just the same way.
- an employment system that has "optimized" to needing as-standardized-as-possible papers that certify potential employees of having skill X so that hiring managers can easily separate between candidates that are "worth it" on paper without having to waste time on "unworthy" candidates - something that obviously fails as it chucks out a lot of the people that aren't built for bulimia learning but can't prove that (unlike an awful lot of the "certified" people) they actually know what they're doing
- "education" institutions that are more interested in getting grant money and income from student tuition rather than on training actually talented students
- and as a root cause of all of that: employers believing they need "university graduates" when the good old German-style apprenticeship system works just fine... with the side effect that apprenticeships cost the employer actual money for years for training the apprentice, whereas with university the students and their parents pay the bill for, sometimes, the rest of their lives
Education and employment is in dire need of reforms. Universities should be serving only those who are actually interested in science, companies should pay for educating the workforce they need instead of forcing generations of young people to take on unsustainable debts, and schools should be reformed to actually provide stuff people are going to use in their later career.