Can't this be done in the US as well ?
Do you feel it is effective?
It seems to me that there is a massive asymmetry in the war here: proctoring services have tiny incentives to catch cheaters. Cheaters have massive incentives to cheat.
I expect the system will only catch a small fraction of the cheating that occurs.
It'll depend a lot on who/where/how is doing the screening and what tools (if any) are permitted.
Remember that bogus program for TI8{3,4} series calculators that would clear the screen and print "MEMORY CLEAR"? If the proctor was just looking for that string and not actually jumping through the hoops to _actually_ clear the memory then it was trivial to keep notes / solvers ... etc on the calculator.
In the years since, I’ve only ever heard mention of older models, not newer ones which makes me wonder if this is a special case and situation where technology is frozen in time intentionally to foster learning.
Oh yes, they're frozen in time, but since the people who pay for them are not the same people who demand they must be used, they're not frozen in price. It's the most expensive kilobytes you'll ever buy.
At my high school we were allowed to have TI-83s but not TI-89s, because 89s had built in CAS (computer algebra system) and could do your algebra homework for you. When I went to college I already had an 83 so I didn't feel the need to upgrade.
I ended up displaying "M" "e" "min(" "c" "log(" "e" "a" "r" "e" "d". Then covered up the "in(" with spaces.
Then you lower your contrast for the full effect.
If they don’t catch them they don’t have a business model. They have one job. The University of London, Open University and British Council all have 50+ years experience on proctoring university exams for distance learning students and it’s not like Thomson Prometric haven’t thought about how to do it either, even if they (mostly?) do computerised exams.
You put your stuff in a locker. They compare your face to some official photo ID and take your photo. You sit the test. They print out your results along with your mugshot. That's it. It was very painless.
The difference is running exams is a small part of a teacher's job, and almost certainly not the part they're passionate about.
Also proctors demand things I've seen no teacher at any level demand (or be able to demand).
There is definitely a war between cheaters and people catching them. But a lot of people can't be bothered and if learning the material can be made easier than cheating then it will work.
You can imagine proctoring halls of the future being Faraday cages with a camera watching people do their test.
I've been running a programming LLM locally, with a 200k context length with using system ram.
Its also an abliterated model, so I get none of the moralizing or forced ethics either. I ask, and it answers.
I even have it hooked up to my HomeAssistant, and can trigger complex actions from there.
So to your point, it’s easy to cheat even if the proctor tries to prevent it.
You wanted to "ace the class", which is an "A" on your final report card? But your crush's exam tanked your grade? You passed the class anyway, right?
Did you swap Scantrons, then, and your crush sat next to you, writing answers on the dgfitz forms?
She wouldn't pass without an "A" on the exams, so her running point total was circling the drain, and your effort gave her a "C-" or something?
In what ways did your teacher make the exams "fair"? What percentage of the grade did they comprise?
Were the 3 tests administered on 3 separate occasions, so nobody caught you repeatedly cheating the same way?
I imagine that it would be utterly trivial for two people to nearly-undetectably cheat in this way, by both of them simply writing the other person's name on their exam.
Not sure if that impression is accurate though, or if it's true of mathematical writing.
If you're just filling in bubbles on a scantron, then handwriting isn't very visible and each person can just write their own name on the scantron they're submitting as their own.
The main kind of cheating we need them to prevent is effective cheating - the kind that can meaningfully improve the cheater's score.
Requiring cheaters to put their belongings in a locker, using proctor-provided resources, and being monitored in a proctor-provided room puts substantial limits on effective cheating. That's pretty much the minimum that any proctor does.
It may not stop 100% of effective cheating 100% of the time, but it would make a tremendous impact in eliminating LLM-based cheating.
If you're worried about corrupt proctors, that's another matter. National brands that are both self- and externally-policed and depend on a good reputation to drive business from universities would help.
With this system, I expect that it would not take much to avoid almost all the important cheating that now occurs.
I get why they use it, without it there's no way to know you're not on your phone or another device cheating since they can only really see what's on the device you've installed the proctor software/rootkit on.
Sadly Linus Tech Tips video of him taking the CompTIA A+ exam has been taken down after threatening letters from CompTIA but they demanded a basically baren room, 360 photos and spotless web cams.
Anything short of all the requirements and they do not administer the test. In the case something is broken, there is usually a window rather than a specific time when you can take the test, so you have an opportunity to resolve it.
If a webcam is required, students are informed about this requirement before enrolling in the institution, and are usually required to purchase hardware capable of doing this as a condition of enrollment.
- Make “homework” ungraded. Many college classes already do this, and it has been easy to cheat on it way before AI by sharing solutions. Knowledge is better measured in exams and competence in projects. My understanding is that homework is essentially just practice for exams, and it’s only graded so students don’t skip it then fail the exams; but presumably now students cheat on it then fail exams, and for students who don’t need as much practice it’s busywork.
- Make take-home projects complex and creative enough that they can’t be done by AI. Assign one large project with milestones throughout the semester. For example, in a web development class, have students build a website, importantly that is non-trivial and theoretically useful. If students can accomplish this in good quality with AI, then they can build professional websites so it doesn’t matter (the non-AI method is obsolete, like building a website without an IDE or in jQuery). Classes where a beyond-AI-quality project can’t be expected in reasonable time from students (e.g. in an intro course, students probably can’t make anything that AI couldn’t), don’t assign any take-home project.
- If exams (and maybe one large project) aren’t enough, make in-class assignments and projects, and put the lectures online to be watched outside class instead. There should be enough class time, since graded assignments are only to measure knowledge and competence; professors can still assign extra ungraded assignments and projects to help students learn.
In summary: undergraduate college’s purpose is to educate and measure knowledge and competence. Students’ knowledge and competence should be measured via in-class assignments/exams and, in later courses, advanced take-home projects. Students can be educated via ungraded out-of-class assignments/projects, as well as lectures, study sessions, tutoring, etc.
There are a LOT of people that don't take exams well. When you combine that with the fact that the real world doesn't work like exams in 90% of cases, it makes a lot of sense for grades to _not_ based on exams (as much as possible). Going the other direction (based on nothing _but_ exams) is going to be very painful to a lot of people; people that do learn the material but don't test well.
The in-class assignments should also be easier than the take-home projects (although not as easy as the exams). In-class assignments and exams would be more common in earlier classes, and long projects would be more common in later classes.
But it stops a casual cheater from having ChatGTP on a second device.
I did a remote proctored exam for the NREMT last year. They had me walk the camera around the room, under the desk, etc. All devices had to be in my backpack. No earbuds. They made me unplug the conference tv on the wall, lift picture frames etc. I had to keep my hands above the table the whole time, I couldn't look down if I was scratching an itch. They installed rootkit software and closed down all of the apps other than the browser running the test. They killed a daemon I run on my own pcs that is custom. They are recording from the webcam the whole time and have it angled so they can see. They record audio the whole time. I accidentally alt tabbed once and muted the mic with a wrong keyboard, those were first and second warning within 5 seconds.
When you take the test in a proctored testing center location they lock all of your stuff in a locker, check your hands, pockets, etc. They give you earplugs. You use their computer. They record you the whole time. They check your drivers license and take a fingerprint.
Those methods would stop a large % of your attack vectors.
As do the repercussions:
A candidate who violates National Registry policies, or the test center's regulations or rules, or engages in irregular behavior, misconduct and/or does not follow the test administrator's warning to discontinue inappropriate behavior may be dismissed from the test center. Exam fees for candidates dismissed from a test center will not be refunded. Additionally, your exam results may be withheld or canceled. The National Registry of EMTs may take other disciplinary action such as denial of National EMS Certification and/or disqualification from future National Registry exams.
At a minimum you're paying the $150 fee again, waiting another month to get scheduled and taking another 3 hours out of your day.
> When you take the test in a proctored testing center location they lock all of your stuff in a locker, check your hands, pockets, etc. They give you earplugs. You use their computer. They record you the whole time. They check your drivers license and take a fingerprint.
While attending there, I also took a virtual Calculus class. The instructor was based in the satellite campus, several miles away. The virtual class required a TI graphing calculator, used Pearson textbook & video lectures, and all the tests and quizzes were in Canvas. I worked from home or the main campus, where there was a tutoring center, full of students and tutors making the rounds to explain everything. I received tutoring every other week.
But then our instructor posted the details on our final exams. We were expected to arrive in-person, for the first time of the semester, on that satellite campus at specified times.
I protested, because everything I'd ever done was on the main campus, and I rode public transit, and the distance and unfamiliarity would be a hardship. So the disability services center accommodated me.
They shut me into in a dimly lit one-person room with a desk, paper, and pencil, and I believe there was a camera, and no calculator required. The instructor had granted an extended period to complete the exam, and I finished at the last possible moment. I was so thankful to be done and have good results, because I had really struggled to understand Calculus.
I used a spare laptop I wipe.
ex “1% of test takers were disciplined for attempting to contact someone for help using a disallowed electronic device surreptitiously”
minimally as deterrance
LLMs and remote exams changed the equation so now cheating is incredibly easy and super effective compared to trying to morse code someone with a button in your shoe.
Also: I think your suggestion is excellent. We may see this happen in the US if AI cheating gets out of control (which it well).