ProctorU is dystopian spyware
shkspr.mobi
shkspr.mobi
I'm not good at things because I'm always able to magically materialize the right answer out of my mind inside an anechoic faraday chamber, I'm good because I get the right answer in a reasonable amount of time using all the tools available to me (the internet, my brain, books, whatever).
I get that there are a few professions where you're really going to need to know the right answer quickly without looking it up under high-pressure situations, and it seems like testing and certifying doctors and lawyers is important enough to society that we can arrange to proctor those tests in person (and still maintain covid safety, disability/poverty access, etc). That level of extreme proctoring security seems pretty dumb for random exams WHERE school <= undergrad.
If you elaborately cheat your way through all online classes all the way through undergrad, that seems mostly bad for you
I agree with the premise, but I think you're downplaying this quite a bit. Pretty much any job that you're not doing asynchronous work in front of a computer would apply here.
> If you elaborately cheat your way through all online classes all the way through undergrad, that seems mostly bad for you
Well it's more than bad for just you. It's bad for the university and everyone else with a degree from there since you are degrading the value of having that degree.
Just a thought about how it's weird we argue about how testing should be done online, but not the fact we test online things which are not really online in their nature.
It's lame to ask for examples, but do you have any? Because honestly I don't think it applies to many jobs, unless it's something that a) puts you in high pressure situations, and b) requires specific in-depth knowledge. There aren't that many jobs like that, and for the ones that do exist (medical, military), you don't get in just by doing a bunch of multiple choice tests.
As someone in the medical field, I would say upwards of 80% of the graded portion of a nursing license and it’s certifications (trauma nursing, cardiac life support, etc), you’re graded entirely on multiple choice questions. People die from mistakes, yet I meet a lot of people who proudly admit to having cheated or googled their way through a much of their studies. Some of them are good and quite a few are downright dangerous at their jobs.
A lot of places won't let you take the exam without practical experience, under the supervision of clinical personnel. Harder to google your way through that.
A truck driver, a crane operator, a musician, a teacher, a chef.
Danger doesn’t have to be physical. Embarrassment, potential failure etc are all reasons to need to know something without looking it up.
The first two have specific government licensing requirements that require in-person examinations in use of the equipment in question.
The third has no need of any sort of proctored test.
The fourth _may_ need some sort of proctored test, but most teaching licences require in-person or otherwise monitored practica—and those are far more viable than anything proctored. Teachers should _often_ be ready to turn to books, depending on what it is they are teaching.
The fifth also doesn’t need a proctored test, and the type of immediate no-book knowledge required when preparing something is something that isn’t readily memorizable, but is instead only something that is achievable via long experience. Most chefs work from recipes and plans.
I don’t think that any of the examples you have given fit the mold.
Probably a key detail - I’m in New Zealand, and Health and safety is taken increasingly seriously. Company directors face stiff fines/imprisonment for H&S failures and while things have a long way to go, they have also come a long way in a short time.
However, health and safety &c are mostly _orthogonal_ to the professional standards of the professions you listed—and in some cases (musician, teacher, and chef), are not the primary responsibility of the professional. (A tour manager / stage manager may have H&S responsibilities around a musician’s show, but musicians themselves have almost no such responsibilities.)
Structural engineers need to know how to calculate the safety of what they are building, but they may (and probably _do_) need to look up the specific regulations and requirements for different localities or structures or materials quite regularly, unless they work on a single type of structure and material in a single location that never changes its regulatory requirements.
Looking up the specifics to an answer isn’t a failure, and our training and certification regimes need to recognize that it isn’t a failure. If having split-second responses to a particular situation is required, then the training needs to focus on what is absolutely required and reflect that (much like military training does with respect to weapon safety).
Bad waiters would just have no clue and won't know what to do.
Mediocre waiters would offer to go to the kitchen and ask about it
Great waiters will tell me yes or no, will offer me to cook it with X instead of Y, or give me one of the other options in the menu (they already know by heart) that do not contain dairyMany places will have waiters follow up the "does X have {{ allergen }} in it?" with clarifying questions as to whether the customer has an allergy or if it's a preference. If they are indeed asking because they have an allergy, it's critical that everyone dealing with that customer's food knows in advance so they can double check all ingredients and prep/handle the food in a way that limits any potential cross contamination.
Personally I found studying philosophy extremely helpful to solving business problems. Not because there is an exact 1 for 1 concept match. Rather it teaches you how to frame and break down problems so that you can be more adaptable and efficient when faced with various unknowns.
There's always going to be domain specific learning needed for any position. But to dismiss any tangential knowledge as useless is extremely short sided.
I agree that “studying philosophy” is a worthwhile endeavor, but I align more closely with the parents in regards to memorizing the semi-random litany of data bits that show up in a typical exam. The information is ultimately forgotten and is usually only relevant in the context of the current textbook chapter. Nearly every useful concept that applies to my two separate careers was learned on the job. The time spent in the classroom is nearly irrelevant and could have been replaced entirely by several weeks on the job.
This applies to pretty much any position that needs to communicate in real time about a base of knowledge (doctors, lawyers, logistics, any type of management position). If you believe that any career can be substituted by "a few weeks on the job", you're not really aiming that high for yourself.
Being fluent in your professional vernacular is very important for productivity. And a closed test is the best, scalable way we have to test this.
True story: when I became a programmer in the 1990s, I used to buy the printed manuals for the Java APIs as they were released. At one of my first interviews (for a Perl job), I was given the Camel Book as reference for the (handwritten) programming test.
The medical industry is not a solutions provider. You should view them as a useful tool, but one that still leaves you with the responsibility to utilize the useful tool to solve your problems.
I am not making a normative claim here; I'm making a descriptive one. The medical system is an incredible toolset, but you need to be ready to assemble it into a solution. Maybe it should be a solutions provider. Maybe it's really discriminatory against the people who won't or can't operate this way. No argument. But it observably isn't a solutions provider today, whatever "should" be.
In this particular case, if you care you should have scheduled a followup with a different doctor. ER doctors don't do that sort of analysis.
My current requirements, just this month, are:
* Python
* PHP
* Javascript
* React and Vue
* TypeScript (but not in every JS project!)
* Blade templating
* Liquid templating
* Whatever Algolia uses for it's inline templating
* Bash
* C# (games!)
For apps and web development, I wouldn't fuss too much about specific language syntaxes since if I were to cull my applicants based on language experience I'd be throwing away plenty of talented developers. I'd hire a talented Python developer for a job writing JS for example, if they're applying then they're willing to learn it.
By scalable you mean cheap.
An actual test would be: design a whole system/experiment to do a thing, in 3 hours.
Can't look up basic things, you'll be too slow.
But then it can't be auto marked, so isn't cheap.
Exams: The very last time in your life you won't have internet access.
Sure I could, it'd be fine, but why store the entire syntax structure in my mind rather than a pointer to the location where I can find it in its most updated form?
There is a surprising amount of complexity behind switch/case.
That ship sailed a long time ago.
Edited for snark removal.
Thinking is not "answering questions".
In order to build a mental model of a problem, you need a rather good model of the environment in your mind, and this comes from memory and the habit of thinking (which is what drill exercises facilitate).
Rote learning is an essential tool in being able to create adequate models of the environment.
The problem is lots of people think it is useless because "I did not need to memorize", when the issue is that "they are sufficiently able to memorize that they do it unconsciously". But most people need to put the effort.
Unless things have changed in the 30+ years since I was in college, I seem to recall exams happen after weeks of study and not "on demand". Part of learning is memorization. There's no way around that. It sounds more like you aren't cut out for the pace of high-caliber institutions which move very quickly. I have two friends with literature degrees, one from UCONN and one from Dartmouth. The Dartmouth friend had to read 4x the number of books than the UCONN friend in the same period of time for the same degree. C-students still need jobs! Just because you aren't a straight-A doesn't mean you can't still be a serviceable employee.
The need to manually solve the formulas has since been superceded by basic calculators. But it's still all (personal opinion) unnecessary rote equation solving these days. I certainly could've used more emphasis on structure and less on longhand math.
Good to know it's the same over there!
Memory is useful because it is the very basis of reasoning. Rote learning is important because intellectual habits facilitate thinking.
Google has not all the answers, but especially: it does not have any questions.
There's not reason to fail a candidate just because they can't remember the signature of a method in a library they haven't used in a while.
Having said that, I do think that many of us have become too conditioned to having access to Google/the internet/the ability to always look stuff up, that it does become a problem (I am utterly incapable of getting around the city I live in without Google Maps, as an example), and that, I think is a negative for humanity. Having access to information and reference is great, but never taking the time to actually learn/memorize/understand concepts and content so that you feel confident doing something without checking the answer is still really important in lots of roles. There should be a balance.
But none of this excuses the malware/spyware that ProctorU and other systems employ. Frankly, people who are that committed to cheating will cheat anyway.
I've heard of similarly dystopic sorts of bullshit even when it comes to remote job interviews, often (but not limited to) code tests. I cannot imagine any job that would be worth me installing spyware/malware on my personal machine for the interview. Even for a university exam, I would push back hard on this sort of thing (or at least the onerous requirements and access it has) or demand the school issue testing laptops or at the very least re-think how/why they proctor exams the way they do. Individually, students don't have a lot of leverage, but if enough people complain -- especially in high dollar degree programs like a MSc -- universities will rethink their behavior -- or at the very least, force the software firms they pay millions of dollars to to support a fucking Chromebook.
Additionally, for any structure of sufficient complexity, the "lead" firefighter will _absolutely_ be requesting and receiving building plans to direct the firefighters to the appropriate locations. Also, like London taxi drivers, fire truck drivers probably have to train on the streets they are protecting. (That said, I do recall stories of firefighters getting lost due to bad GPS directions.)
Quoting myself:
> but I'm not sure how many of those careers (air traffic control would be another) would have remote-proctored exams anyway.
The same applies to the drivers of fire trucks. They’re going to both be trained to drive around for locations and tested on that, but they will also use appropriate tools to get the location. It might be an "enhanced" GPS, but it isn’t going to come down to someone’s memory all the time.
What I did was ask them to write real code. It’s a course on programming. If you can get the computer to accomplish the given task in whatever way you know, including gluing together parts and libraries from online, then that’s a demonstration other learned something!
And as it turns out, C and D quality work is still submitted even when students have all the time and resources in the world. The internet is not a magic wand to solve all problems for you. For example, my final project is to write a file server. Yeah you can find tutorials for how to do this on the internet, but it’s still going to have to be customized to use my specific protocol. For someone who knows what they are doing, they can do this assignment in 10 minutes leveraging the right tools. For others, they might write it from scratch as we discussed in class. Others still will barely get past writing a makefile, even with step by step instructions and examples on how to do it. So in the end the grades worked out as they usually do, distributed normally with a B- average, and about half a letter grade stdev. That’s just the way it is.
So this fear that access to the internet will lead to rampant cheating is only relevant if the questions are easily gamed: like definitions, facts, and contrived math problems. Well it turns out this is like 90% of prepared materials for the low level intro classes.
I mean, if you can get through a number of semesters of college by looking up answers on Google, the problem is not with Google, it’s with the curriculum. Because what value are these courses really adding if they are just cramming facts into your brain and then you regurgitate them, and promptly forget?
Take chemistry for example. All of chemistry should be in a laboratory setting. All of it. It’s a perfect hands on discipline, yet the standard sequence is just year after year of memorizing facts and definitions with very little comparable lab time.
And I get why this isn’t done, it’s probably not practical and there does need to be an acquisition of fundamental knowledge, but that isn’t really done by going through prepackaged web content (that can’t be resold) with all the answers available on Chegg, as is the case for a great many 100 and 200 level courses in all major disciplines at most universities.
I mean this is just... Incorrect. High school chemistry (generally the first time it's called chemistry and not science) is often taught in a lab (I assume mainly a function of whether lab facilities are available). Undergraduate general chemistry and organic chemistry generally have a required lab component that is required to be taken concurrently (this is even true at community colleges). Using general chemistry as an example, the lab portion is usually 1 three hour (at least) class per week, while lecture is 3 one hour classes per week (organic chemistry is essentially the same). More advanced chemistry might not have lab components (e.g. physical chemistry), but the foundations are almost always at least combined with a lab.
>the standard sequence is just year after year of memorizing facts and definitions with very little comparable lab time
And I compared the lab and lecture time... and they are very similar (essentially equal)...
I didn't choose to respond to your overall ideas about a chemistry degree because that would require convincing you that there is quite a bit of fundamental information needed to make those labs useful (and efficient) and there was already a great misunderstanding regarding the amount of time chemistry students spend in the lab.
Well you don't need to convince me of that since I admit as much in my original post!
> and there was already a great misunderstanding regarding the amount of time chemistry students spend in the lab.
I can only speak to my experience advising dual CS and Chemistry majors. The ratio of lab to class hours for them is 1:2, and worse when you add electives. So that's the basis for my statement. YMMV.
Ok, I guess in your degree program the only difference is that you sit around in a lab instead of a lecture hall learning those fundamental things then? Because I'm not referring to some trivial amount of fundamental information... I'm referring to, roughly, the amount of information covered in 3 one hour lectures a week.
> I can only speak to my experience advising dual CS and Chemistry majors. The ratio of lab to class hours for them is 1:2, and worse when you add electives. So that's the basis for my statement. YMMV.
Ok, I'm citing directly from my university's course catalog (and also the courses I took at several universities) so YMMV.
The most obvious example why you're wrong is learning a second language. Even learning a language outside of a classroom generally requires memorization of fundamental vocabulary (and sometimes a writing system); it therefore makes sense that a class teaching a second language would test students on this memorization.
Now, other fields might not be so obvious, but they require many things to be quickly accessible, much in the same way that one might need quick access to a word or a character with a second language. A biologist needs to know and understand the central dogma, an engineer needs to understand mechanical stress, and a chemist needs to be able to read a structural formula; they need to be able to do these things essentially instantly (and understand them intuitively) to be able to even discuss more complex topics, because complex information builds on simple information. Hence it makes sense to make sure those pieces of information are understood and readily available before sending a student off to a more advanced topic.
Writing is a skill. It is NOT memorization. It is not taught or learned that way, even at the beginning, unless you are talking about handwriting.
No one says, "Memorize this phrase, and now write it out again."
Not to mention memorizing how to physically write (i.e. what you're talking about with handwriting)...
I assume you think I mean composition or something, which is of course extremely obviously not what I meant... But to develop that skill of "writing" that you're talking about requires memorization of the fundamentals... Which applies quite broadly to other disciplines...
You are just wrong in every possible way.
Memorization can be done on a cellular level, generalized pattern recognition - not so much.
No they say "Write this word 10 times to memorize it" then give spelling tests.
I also think the way we test people is set up to be measurable rather than effective, and all this hand-wringing about cheating is an attempt to prop up a premise about pedagogical techniques that simply do not accomplish their goal (and a way to scam universities out of a lot of money for tools like this one). Most of the market for standardized tests is a racket built on market dominance and irrational beliefs about its value built on a (pervasive, policy-level) misunderstanding and subsequent cargo-cult style blind worship of metrics per se
On top of all this, the sheer level of security and privacy violations the school system seems to tolerate for this dubious purpose is ridiculous, and speaks to a deep gap in knowledge about, or devaluation of these things that causes far more problems in our information-driven world than cheating on a dumb test possibly could
I've gotten advanced degrees and consider them to have been overall a valuable experience, but even then most of the education system as we have it was pointless torture, and I probably would have quit school if this nonsense were around when I was in it
And I feel like it's drilled into instructors too.
-More oversight and efficiency in this process is breaking down the unofficial means through which anyone got anything else out of it (e.g. real mentorship from educators who care that is not encouraged so much as tolerated by education policy, opportunities to engage with new ideas in a meaningful way, good reading recommendations, an incidental avenue into a social life)
-The structure and function of public education is increasingly infecting university curricula, structure, and priorities
I think this likely has the effect in broader society of creating less fluid competency and more blank-faced compliance, and this disproportionately affects people in key leadership roles as schooling becomes more of a selection pressure on people's career paths
This is not said often enough and it is a shame.
Universities are not intended to be vocational schools. They're research institutions where the value they generate is the knowledge they discover. The knowledge discovered is often frivilous and arcane with no real use outside of a very specialized subsection of a field of study. However, there's also the knowledge which is 'useless' at the time of its discovery but changes the world later on as its studied further and applications are discovered.
E.g. https://math.berkeley.edu/~gmelvin/math54f12/math110su12_gra...
I don't think there's much more to add than Turchin's ideas of "overproduction of the elites" and it rubs me the wrong way when I hear intelligent people gripe about how university credentials and research have marginal vocational utility. The problem's with a culture that celebrates credentialism and gets people to take on crippling levels of debt to avoid an impovershed future, not scholarship.
That famously changed under Ronald Reagan. Then came "everyone should go to college", while maybe 10%-20% of the population usefully should.
Lawyers spend most of their (billable) time researching. And the great majority of lawyers never see a court room. No quick reaction time needed.
Doctors... The average doctor follows a script to triage the easy 80% of problems. And the average doctor stops at that.
I agree, if you can look at available resources and competently perform the task that is likely not an issue in the field.
However paying someone to tell you all the answers to a test is cost effective for an exam that takes a couple of hours but will not get you by in the field. The university doesn't want to be known to give out degrees to incompetent workers so they need to prevent this case.
As a teacher, I would love to be able to dispense with tests except as informative diagnostic tools. Unfortunately, I am required to give at least a final exam by my university, and that and other grade-based requirements come from the confounding of what the university's mission should be, which is education of the interested, with what most (at least US) universities' missions have become, which is credentialling (of the often unwilling—and who can blame them, when the requirement is artificially foisted upon them?).
Unfortunately, there doesn't seem to be any way to get away from that drift towards credentialling that doesn't involve a shrinking student body, and administrators are addicted to the idea that we so badly need an ever-increasing student body to prop up … well, salaries for administrators, for example, whom we clearly need because without them we might not do so well on this endless treadmill of pointless dedication to growth.
Everything allowed Math I + II ate the dreams of students like candy.
Still, even if you are good at math and have all the tools at hand, you still had to develop a general understanding of the topic since otherwise you would just be way too slow to work through the assignments.
I opted out of using spyware in the university physics courses I taught last year, and caught my cheating students the old fashioned way. Proctor-spyware, like airport security, is more about theater than effectiveness. You aren't giving the USMLE or a Bar Exam, so you can take the time to write a good exam and evaluate it correctly.
The total time spent in creating a few good, easy to discuss questions, the answering of which would demonstrate understanding, and then reading them, thinking about them, and coming up with a grade, was probably actually less than the amount of time it took to create meaningful multiple choice questions that didn't have any ambiguity, and which weren't easy to intuit the answer even without understanding. Doubly so when we did away with "partial credit" answers, but instead made it so each question was 1 point (or otherwise all or nothing), you had, say, 5 of them, and what we really were looking for was a paragraph that showed understanding (rather than checking off boxes in a rubric of "mentioned A, B, and C"; short essay questions, if you will), an expectation we communicated to the students.
And that's aside from the actual project based grades, which were better still.
By the way, where I work, management pushes for stuff like proctoring, and more students, and "measurable" results, and so on. As a teacher, I don't care much about the whole grading show.
Then the students are not getting what they pay for.
Must all of the cards in the deck be played at each turn? Things cannot simply be bad/user hostile/privacy invading, etc.
I would especially consider that in the United States we do not have a broad legal right to privacy but there are potentially much stronger tools available if that software skews negative outcomes towards protected categories like sex, disability, or race. From the perspective of a student, job applicant, etc. being asked to use this, if the legal risks cause an organization to stop using it they'll enjoy that as a win even if the outcome isn't a blanket ban.
Not at the expense of grounded reasoning. When I see poorly substantiated claims, it shouldn't, but it really drags the whole rest of the argument down. The argument presented about lawsuits is actually a great example of why I think that's a broken system. They resolve that issue of lost credibility by considering each issue with total, clear, and mandated separation. Outside of that legal world with very well-defined rules, using such tactics reduces credibility.
I should note, that in this particular case, the claim of racial biases is at least substantiated by a believable anecdote.
EDIT: To clarify why I think the legal methodology is broken, its only because the same principles apply to criminal trials. IMO, prosecutors should NOT be throwing poorly substantiated charges at a defendant just to increase their winning probability and make the required defense more expensive.
It should. If someone is willing to make wild, unsubstantiated, claims, it should detract from their credibility.
If the parent comment had said something to the effect of "I should give the benefit of the doubt to a sitting US senator" then you'd have a point, but that context wasn't part of their statement.
Edit: Also, frankly, I wouldn't give the benefit of the doubt to a US senator. If anything, it makes the identity politics feel even more irrelevant.
I wasn't reacting to a summary, I was reacting to an independent premise. The sentence I reacted to, at least the way I read it, was broad to the point of being more of an axiom with which the commenter interpreted the post. As I see it, disagreeing with an axiom is an intellectual engagement. Feel free to counter my disagreement.
> It’s kind of hard to see any definition of “identity politics” which includes the reports but not your emotional reaction to accurate words.
Not entirely sure what you're saying here, but none of what I said was emotional. Simply pointing out that pattern matching is a viable way of filtering other people's thoughts and ideas. If someone makes wild claim, it should change the way you view other claims which may have seemed more rational in their absence. Not really an emotional statement in my mind, but again, feel free to point out which part of this you disagree with and I'd be happy to engage.
As it stands now you aren't really responding to anything I've said, but rather disagreeing that what I'm responding to warrants responding, which is rather tangential.
Case in point, here's someone saying, 'How can you even handwave away systemic racism?' in reply to a comment agreeing that bias exists but is not deliberate: https://news.ycombinator.com/item?id=29164295
Also, the identity politics argument seems to hinder a simple, moral argument against surveillance software as a violation of privacy. The logical implication of the IP argument is that this surveillance software would be okay to use if we manage to work out all the kinks.
It's not the same with general discourse - when you raise a bunch of issues that aren't especially relevant and seem designed to be inflammatory, you damage the credibility of your other arguments. Arguing that test proctoring software is transphobic is such a stretch that it makes you question whether they author has such strong biases against the software that their evaluation of it is just generally too biased to be trustworthy.
It seems like the software is matching people against existing images, based on the issue with the black student, and trans people are I would assume more likely to change their appearance, including as a result of taking hormones and having facial feminization/masculinization surgeries.
Yes, but with some evidence. Otherwise, as a society, this is a bad direction to head in.
That being said, there's a rotating list of "badnesses" that are in the zeitgeist and I agree that it's annoying to see them flogged at every opportunity (often w/o much insight).
The statement is at best extremely misleading, and at worse, mostly false. It also represents a juvenile, immature, and myopic perspective on reality.
I would suggest reading "Wealth, Poverty, and Politics" or "Discrimination and Disparities" by Thomas Sowell. He has been debunking the "inequality of outcome therefore racism" logical fallacy for decades.
B) Even if it were simply physics at play, requiring the use of a system known to have physical constraints against darker skin causing failing grades purely on that metric is still pretty racist.
I wanna pull this apart a bit because I think it's a good opportunity to talk about how systemic bias gets started. Digital sensor evolution is path-dependent. Technologists developed photo-sites that have "enough" dynamic range for most uses before moving on to increase the resolution on a sensor. What exactly is "enough" depends on your test data.
The sensor on a webcam is only "so big" as you say - but how that sensor balances resolution and photo-site count depends on what conditions they consider acceptable. We could build web cams that would see more pigmented faces better - there is no fundamental limitation in the technology itself. It's that a series of decisions have been made over years of development, generally without people thinking specifically about race at all, and we've arrived at a status quo that has adverse outcomes for people with different skin tones.
There was a similar process that happened with film photography[1]. Not that film, as a technology, is unable to capture dark skin - but that the development standards that were tested and distributed were designed for lighter skin.
Like, I agree that the webcams we have aren't intentionally 'racist.' But I do think that the status quo that has led everyone to accept this balance of dynamic range and resolution is reflective of valuing people with lighter skin more.
[1] https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b...
https://www.newyorker.com/culture/cultural-comment/a-century...
It seems like you're reading a specific thing I said about a specific scenario and universalizing it in a way you imagine I might universalize it. I'd love to hear a critique of what I actually said, or we could talk about our views of society in a wider way, but I can't respond to this combination of generally dismissing what I said and attacking what you imagine I might think.
I sense you’re tired of discussions that mention the “cards in the deck”, likely because you aren’t affected by them and therefore care little for them. That’s honestly fair, but there’s value in writing that way to channel outrage into action.
White supremacy: darker skin tones tend to photograph worse than lighter skin tones. Laptop webcams are notoriously crappy and can make this effect worse. Is this white supremacy or physics? Can the test instructions be modified to ask all users to have an appropriate lighting setup (ie lit from above and from the side to ensure that your face is foregrounded properly)
Ableism: for users who require assistive technologies, is it better to take a test in their own space with their own equipment or to travel to a test center and use shared lab equipment? For users with mobility challenges, is it better to be in their own space or travel to a potentially non-accessible testing center?
Sexism: for working mothers, better to take a test in your own home or travel to a testing center and arrange for child care?
Two "isms" seem more relevant but weren’t mentioned: ageism - because fuck boomers, right? Socioeconomics - not everyone has access to a PC that meets the specifications or can acquire one on short notice.
Lots of context is discarded when one engages in polarizing categorical rhetoric. I’m not here shilling for proctoring software but rather for nuanced discourse.
I’m not a photography expert, but it seems to me that cameras are physically equally capable of overexposing an image and underexposing an image. If a particular camera which is used in a facial monitoring system tends to do one rather than the other, I would ask why that is the case.
The choice to use typically terrible cameras in a proctoring system disregarding that it might work ever worse than normally for a subset of people is suspect, yes.
White supremacy is not just bad faith things done by bad people, it is also the assumption that whiteness is the default experience and the failure to account for that not being the case. Similarly with ableism and sexism.
That being said, complaining about the accessibility & inclusiveness of our required surveillance technology does have a dystopian feel to it, lol.
This only works with "new" definitions of racism. It is in fact plainly racist on it's face to demonize a group based on immutable characteristics. It is even worse when actual diversity of though is ignored and people of color are demonized because they don't agree with a race-marxist ideology.
It is in fact plainly racist on it's face to demonize a group based on immutable characteristics.
It is even worse when actual diversity of though is ignored and people of color are demonized because they don't agree with a race-marxist ideology.
>At the beginning of a test, these products ask students to verify their identity by matching their appearance with a photo ID. As Os Keyes has demonstrated, facial recognition has a terrible history with gender[x]. This means that a software asking students to verify their identity is compromising for students who identify as trans, non-binary, or express their gender in ways counter to cis/heteronormativity. If a student’s gender expression or name on their ID are different from their current gender expression or name, the algorithm may flag them as suspicious. When this happens, they may have to undergo another level of scrutiny to authenticate their identity, an already common and traumatic experience for trans and gender non-conforming students. If these students are not alerted of this possibility before the test begins, it may force them to either discontinue the test and risk their grade, or out themselves to their course owner when they may not want to, risking more trauma and discrimination including being denied financial aid, being forced to leave their institution, or have their lives put in physical danger.
>The Eugenic Gaze is a combination of white supremacy, sexism, ableism, cis/heteronormativity, and xenophobia. When we apply the Eugenic Gaze using technology, the way we do with algorithmic test proctoring, we’re able to codify and reinforce all of those oppressive systems while avoiding equity-based critiques because of our belief in the neutrality of data and technology.
Their recommendations are quite reasonable:
>Don’t use algorithmic test proctoring. Instead, focus on pedagogical techniques that you can use to design assessments, online or in person, that draw from personal experience or require students to apply concepts in unique contexts. If you have to use algorithmic test proctoring, make sure students know about the test settings and ID requirement well before they take a test, and assure them that you will not take any behavior flagged as “suspicious” into consideration that isn’t described explicitly in the syllabus.
The GP link[1] instead calls out "Facial Recognition Tech", and "Algorithmic Proctoring" as being too biased and follows up with a petition[2] to ban these entirely.
[0]: https://hybridpedagogy.org/our-bodies-encoded-algorithmic-te...
[1]: https://library.auraria.edu/news/2021/why-online-test-procto...
[2]: https://www.sheaswauger.com/post/petition-to-ban-facial-reco...
[x]: https://ironholds.org/resources/papers/agr_paper.pdf "The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition"
I think you just reflexively dismissed this because you saw a basket of words that normally go along with things you disagree with, but the argument is pretty solid.
Really? On the male part? I'd have expected it to do best with women, because I'd have figured facial hair is more difficult to deal with than a wider varieties of hair styles.
Again, I wouldn't chalk any of this up to deliberate bias against any of these groups, but it's all bias anonetheless.
You could read the research and refute it, or you could just bluster about things. I know you tried to expand on it in your comment below, but minimizing the specific concerns raised to "isms" and ignoring that at least two of the references in the articles linked and their references for the actual research addressed at least the socioeconomic portion of it, illustrates that you only applied your surface level perspective and criticism.
In this case, the algorithm is actually bad for all ethnicities [0]. It's just that it's extremely bad for black students (fails half the time) and just regular bad for everyone else (fails a quarter of the time).
[0] https://www.theverge.com/2021/4/8/22374386/proctorio-racial-...
Still, ProctorU it's a major deterrence to cheating, I'm just not sure it's worth the cost.
Article proceeds to use a lot of words to not show any of these being true.
I am against it for the general dystopian surveillance normalization it encourages. We don’t need to throw a word salad of made up progressive insults against it to resist its implementation.
Shockingly, this was due to some JavaScript relying on an older Chrome proprietary API so there's no possible way they actually tested it against their alleged support matrix.
Lawsuits don't cost millions. Court fees are absolutely never that high, and lawyers, while some may be expensive, are generally affordable for ~middle class (or even lower class if someone wants to do pro-bono work for you)
The whole "lawsuits cost millions" thing is a myth perpetuated by big corporations and further relied by normal folk who hear it from somewher else, which probably heard it from somewhere else, and so on.
When you read in the news "X company wasted $XX million in legal fees", what it actually means is "they stretched out the case with a team of very expensive corporate lawyers whoses price ranges are in the millions".
That may be, but they can easily cost many tens or hundreds of thousands of dollars. Lawyers typically bill at multiple hundreds of dollars an hour so it doesn't take a lot of hours to rack up five- or six-figure costs. That's high-stakes poker for most people.
I once sued a neighbor for their barking dog. It cost me over $10,000 before I pulled the plug.
> "they stretched out the case with a team of very expensive corporate lawyers whoses price ranges are in the millions".
Yes - The company stretched the case out with expensive lawyers: Do you think the other side is somehow not obligated to also continue dealing with that case?
Who pays my lawyer while the company stretches the case out? Oops - that's still me.
----
As someone who has actually retained a lawyer for dealing with a previous employer:
1 - Most places had zero interest if the money at play was less than 100k (ie: They would not take the case unless I had a potential win of 100k or more)
2 - They charge ~$350 an hour. Sometimes billing for "intern" work at ~$150 an hour instead. I make good money (~200k) and I can afford less than 23 days of lawyer time a year, assuming I spend my ENTIRE yearly income on it.
The advice I have gotten from actual lawyers is that it's literally not worth it if you expect to get only a few thousand dollars.
This is especially work considering with this particular company, which has a history of using legal threats to silence critics:
https://www.gofundme.com/f/stand-against-proctorio
I would DEFINITELY not jump at the chance to incur a similar reaction.
Turns out, their JS minifier was creating code that contained a syntax error. Chrome was able to make it work, but Firefox would silently error out. Rather than try to solve the problem, they blocked any browser that wasn't Chrome.
-_-
Me, to <litigious tech company>: “Your JavaScript fails on IE8 because it now throws an exception when it attempts to set an invalid CSS value. I made a tiny patch but do you have an ETA for the fix?”
LTC support: “We don't beta test Microsoft's products for them!”
Me: “Okay, it was released this week. How's testing going?”
[a week passes]
LTC support manager: “Hey, can we get a copy of that patch to give to other customers?”
My employer at the time paid 7 figures annually for support.
Seeing the backend of this tool was much more worrying than being subjected to it. Simply put, the platform is SO bad, that it could not be used as evidence even in the most blatant cheating cases - for example, the screen capture feed and the webcam feed were two separate files, neither of which was time stamped. If a student had a poor or marginal connection, these two recordings would get out of sync and could never be reconciled. It was so primitive it was laughable. That's on top of the issues you'll find documented elsewhere.
https://www.eff.org/deeplinks/2021/02/student-surveillance-v...
In my SO's case, they told her wearing a sleeveless shirt was "inappropriate" during an online exam and told her to put something that covers her shoulder on so she ended up wearing a sweater in 80F inside. When we reached out to their support, they said there is nothing they can do about it and rattled off some standard script.
For an adult how is that not a discrimination lawsuit?
I worked with her to try to get it escalated and at least get the test fees refunded or compensated but didn't work. She just accepted it and scheduled another exam.
I go to UOPeople, which is a tuition free online school and I am getting a 4 year computer science degree (for like $5k which is crazy), anyway of the 40 courses you have to take only 11 are proctored. UOP offers 2 choices for proctoring, A) find a real life proctor, or B) use ProctorU. I don’t have the luxury of finding a real person + with covid its more unlikely, so ProctorU it is.
All in all its not the worst thing in the world, but when I first read the requirements for the ProctorU testing environment and technical requirements I almost quit school completely to look for alternative paths. Some of the crazy requirements include: Testing space must have nothing on the walls, or floors. You can’t wear glasses while taking the exam. Your desk must be clear of everything besides the specific testing materials (calculator if your lucky, and maybe a pencil and paper) Your device needs a webcam, so that they can not only watch you for the entire 1 hour and 30 minutes where you take a test that determines 40% of your grade, but also so that you can show them each wall, floor and under your desk. And thats just the physical space requirements. I had to empty my closet and use my laptop to take this test because their is no way I just have an extra room for testing…
The digital requirements were pretty intense as well, access to folders they had to right too, chrome settings and a whole bunch of wack stuff. I created a dummy account just to take tests.
When you go into the program they have you download it acts like a 1 way mirror, you can hear the proctor(if you are lucky to have a human proctor) and they can watch you, your screen and hear you. I had some tech issues once and I was grateful to have a proctor with a sense of humor who was able to help me through it. I can’t say my privacy is worth a cheaper degree, but I hope that this doesn’t become normalized, because it is not a pleasant experience.
Talk about security theatrics ... I'd just stick the answers I want on a piece of paper just under the cupboard in reach of your feet or with a tape on the bottom of your desk and get it off when scratching your crotch.
> access to folders they had to right too
Why? They are recording all processes and the screen already. Again just theatrics. And if you can hide it from that they are never going to find it anyway.
How is that legal? My face would need to be less than 10cm away from the screen. So - there goes using the camera to monitor where I am looking.
How is this not massively illegal? This is a clear ADA violation. I cannot see without glasses. Not like, things are a bit blurry, but like I have 20/800 vision that's correctable to 20/20 with glasses. Forcing me to take an exam without glasses is forcing me to fail an exam for a reason that has nothing to do with my academic abilities.
Actually surprised they don't require you to take the test naked.
This shit is crazy.
Very unprofessional, if not illegal due to discrimination and even though it was one of Sans's entry level certs, I barely passed, versus 90 + on all of their advanced ones without these issues.
Their reasoning was "The rules say no facial obstructions, your glasses block your face.". They have to hire the dumbest people to do these proctors.
I've never had any issues with other proctoring services, and things like pearson for Comptia and Microsoft were actually enjoyable. With proctoru each proctor seems to find some issue and you have to argue with them since it's completely unreasonable.
Don't know what class/proctor you took, but I took several UoP tests with ProctorU and never had issues with glasses.
Nearly every college student during the pandemic had to use ProctorU in order to complete their classes, or a similar alternative. Quite disturbing the experience is normalized, and I wish there were an official OS level feature for "report all activity on the system from time X to Y", without having to use a sketchy third party app.
I wish the author the best of luck fighting the requirement to use ProctorU.
Oh please don't. This type of espionage should be discouraged, not officially supported.
https://www.amazon.com/American-Plating-502N-Valve-Instrumen...
There was no limit on how many notes one brought in to an exam. Some of the weaker students turned up with rucksacks full of ring binders and the invigilators had frequently to admonish them to make less noise rustling the papers! Those students almost all failed or attained only a pass degree.
In my opinion this successfully weeded out those who thought that memorisation was enough. The exams typically never asked anything that could be answered simply by looking up the answer in notes or even the textbook.
Which is a long way to "yes, I agree."
Perhaps it's about the type of question asked. Knowing the Sicilian Expedition happened after the death of Pericles is different than knowing it happened in (checks Wikipedia) 413 BC.
But then knowing the year is very important for the world historical context. At that era in Greek history, events in Persia had more impact than events in Italy and Britain was essentially unknown to them. Yet here we are living in a world where the Parthenon friezes are in the British Museum. It's hard to put together the different moving parts without dates.
I'm not following. If you don't know the material, you have same incentive to cheat.
But QEMU/KVM which is the de-facto hypervisor on Linux is harder to detect. Even the others which I mentioned before can be hardened to evade detection.
And if you do a little bit of tinkering and intercept traffic, you can make it so that all the cheating reports from the "AI" never leaves you computer. I've never played with ProctorU but have experimented with a couple of other similar software. They usually send regular reports every five minutes and some anomaly reports (some extra software running on your computer, another person in room, face not visible, etc) when something happens. You need to intercept and modify traffic to not send these anomaly reports. This is easier if its browser based, but you need to install systemwide certs if its install-able software, and a lot more work if they utilize certificate pinning inside binary install-able software. I have never encountered the last one though.
That's after fighting with the software to have it installed in the VM to begin with.
Defeating malware's VM detection is very interesting.
Links for others if they're interested:
https://github.com/a0rtega/pafish collects all the best-known detection methods into a test suite.
This issue is interesting/has links for sure: https://github.com/spender-sandbox/cuckoo-modified/issues/45...
I heard that MS is requiring notebooks to have a HD front facing camera. Maybe they would still sign my device driver? If they don't, wouldn't that be a lawsuit waiting to happen?
This is the digital equivalent of forcing students to be naked to take a test in person.
Impossible requirements have been placed on at-home exams and proctored testing; and this company stood up.
Are they violating many moral and reasonable privacy codes to do it? Absolutely. It is a huge breech of ethics.
But universities and their professors asked for it.
Plenty of online educators already know how to (edit: lead) classes and give tests without it; but many, too, are either lazy or overburdened and have asked for this.
It is interesting that institutions that often have "department of ethics" are the first to be OK with awful products like online proctoring software.
Coming up with a replacement is the hard part.
[1] https://www.nytimes.com/2021/05/09/technology/dartmouth-geis...
> Surrender to the power of complex technological systems — allow them to oversee, track, quantify, guide, manipulate, grade, nudge, and surveil you — and the system will offer you back an appealing share in its spoils. What is good for the growth of the technological system is presented as also being good for the individual, and as proof of this, here is something new and shiny. Sure, that shiny new thing is keeping tabs on you (and feeding all of that information back to the larger technological system), but it also lets you do things you genuinely could not do before.....The danger, however, was that “once one opts for the system no further choice remains.”
Shockingly, smarmy ed-tech hucksters don't have a good answer to this one.
This extends to coding interviews as well. Using the resources at one's disposal to get a sense of the landscape before diving into algorithms must surely be part of the job, right? What do I care if a developer needs a quick reminder before diving into a solution, or even reads up a bit and scans someone else's code before answering?
What is the value in ensuring that people have perfect recall if this is something that will almost never be necessary in a real world job?
On the other hand, at least a basic level of recollection is necessary for quick thinking in meetings, you don't always have the time to look up documentation.
Plus, it doubled as a study guide for the actual final which was only a couple of weeks later. I thought it was a remarkably kind thing to do. Took out a little stress. Gave even the struggling students an easy "A". And it worked as a comprehensive guide to almost everything we covered.
It's not an odd question. "Okay, the whole subnet where your credentials server used to be is now a smoking hole in the ground, and IT forgot to pay the fiber bill last month. What do you wish you'd done three years earlier to address this problem?"
To set people at ease I tell them up front that only one candidate has crossed the line (googled the solution) and everyone else has made perfectly appropriate choices; "elseif or elif?" and small details like that.
Basically I think that online evaluations have to be completely different than in-person ones. Proctoring is fairly trivial and non-intrusive for in-person tests - don't open your laptop, don't talk to the person next to you. For big tests in small rooms I assign seating.
Online is different. Basically there is no reasonable way to keep someone from hiring someone on Chegg to do their entire test for them, and the most horrible proctoring software in the world won't stop a determined cheater from balancing a cell phone at the bottom of their laptop screen...
You really need to use a different approach with online assessments, and honestly I don't know if it's possible to use online tests for some of the things that we use in-person tests for.
Which brings us to what a university should be about:
* Understanding principles
* Being able to draw connections from principles to novel ends
* Being able to argue your point
* Being able to discern, whether a point is argued properly or not (according to these rules and the rules of the particular discipline/field)
* Knowing about the current state of the art/ideas and their relation to principles
Nothing of this includes a use case for spyware like this. It actually indicates that the institution does not manage at least one of those aforementioned goals. And it it's an indication for that institution rather answering to those challenges by a strict regime of established procedures, which is exactly what a university is all not about.
ProctorU is absolutely, positively insane, but the alternative sounds quite reasonable.
You could make the argument this is how real life works, but we'll need to radically redesign the current curriculum if the internet becomes fair game. Long term this is a must, but short term not making an effort to stop cheating will corrupt the entire institution - anyone that doesn't cheat is at a severe disadvantage.
In the real world my learning of a new piece of software proceeds as follows: the github repository/manual of the software -> stackoverflow/biostars -> google with whatever you want to do.
I agree! I wish more institutions viewed the world as Stanford's Honor Code does. It predates the Internet by many decades:
> Open-book Requirement: As stated in the Interpretations of the Honor Code, “If take-home examinations are given, they should not be closed-book examinations…” Open-book exams place no limitations on the materials or resources that a student may access during the exam.
https://communitystandards.stanford.edu/resources/faculty-an....
Customers can't defend themselves against such intrusions of privacy from their school/government.
And employers too. They want you to respond to emails at all hours, and ask for way too much control over your device in exchange. This is slowly being changed with Android work profile and Apple "user enrollments", thankfully.
One alternate way to prevent this would be liability: if the institutions using this had to reimburse all of their users for every security hole in their mandatory software or the risk due to the security settings they require you to disable, it'd complete change the calculation for them.
> Seems they have no problem hosting software for ProctorU.
The restrictions are on what their apps can do, not on who published it.
In the "old" days, 20 students walked into a room, sat down, and took a test in silence with pencil and paper while the instructor stood there. Now, 350 students go wherever they want, with classmates and many internet-connected devices around them, and take a test unmonitored.
Sure, you tell them the exam can be open-note as long as the work alone, that's easy. But they can work together, and that's hard to detect unless they literally copy.
The #1 problem is Chegg. They can screenshot and post the questions to Chegg and get answers back very quickly. And they do. To add to this, many schools have a large culture of cheating. Like > 10% of students or more will cheat on unproctored take-home tests.
That 10-30% basically ruin exams for everyone, but most of all the instructor. You can't give people relaxed take-home exams, too many will cheat. You have to give a strict time limit. You have to put a ton of effort into making questions obscure or idiosyncratic, rather than just giving standard problems from years past. And remember you have 350 exams to grade, so you can't ask very deep questions.
So I can't get that mad at my colleagues who use proctoring software. I can't really offer a reasonable alternative.
After looking at their system requirements and the draconian measures needed to just be able to take my exam (they also require you to have a webcam, which I don't have and to real-time broadcast your private room, where you will be taking the exam, which is a non-starter for me), I realized it would take me longer to set up correctly, than it would take me to drive there, take the test and be back home.
I heard some people say we live in a high trust society, but that trust only seems to be going one way. If I don't trust a corporation, I have no agency or power to act on that. But if they don't trust me, they impose these insane draconian measures without any oversight that preclude me from progressing in life, professionally or otherwise. It's fucking insane.
1. We don’t require the student to install any software - we use the video streaming feature of the browser on their laptop to record the student’s screen and webcam
2. We get the student to place their phone camera behind them at the 4 o’clock or 8 o’clock position with a view of the student’s desk, then we record the scene using the browser on the phone
In this way we get 3 video feeds - the screen and two cameras - which we can monitor live and record. You might argue that this is still too creepy, but if we cannot see the student while they work it’s just too easy to cheat and that would make our credentials worthless.
That's more or less the only silver lining I can think of here.
Buuut, after a while, we were like, why subject people to this? This is crazy. And why even charge for certs anyway? I'm happy we're done with proctoring!
At the moment these are taken in person in secure buildings where you are identified at the entrance, your belongings are stored in a locker, and the exam itself is taken in a secure room on computers with no/limited internet connection, etc. You are timed and monitored.
Naturally, it's next to impossible to remove once installed. I speak from personal experience
This is so GenY!
----
Keys to a successful Exam Day:
Run the “Computer Requirements Check” at least a day before the appointment
Disable your pop-up blocker
Click “connect to proctor” to begin
Be alone in room
Have a clear desk and workspace
Be connected to a power source
Your mobile phone and headphones must be out of reach
Have your government issued ID ready
Be prepared to perform a room scan
You must be in the webcam view throughout the exam. ----
Same dystopian bullshit. A room scan?! What business of theirs is the contents of my room? What if I live in a studio and it's my whole living space? (it is) Massive privacy invasion that is completely unnecessary.
I told them no thanks. I don't need to pass a test like a college student with ~20 years of experience under my belt, which they knew.
Create an exercise that contains a technical term that does not exist, and create a page containg it with a plausible, but wrong solution for the excercise. Make sure that the page is easily found with Google. Give everyone who solved it using the wrong solution a failing mark.
Personalize every exam. Create a pool of exercises and choose n exercises per student, based on the student id. Easily done if the sheets are already LaTeX anyway.
Create heavy time pressure. Cheating is very hard when even completing all exercises regularly is almost impossible. (Lovingly called "Zeitklausur" in German, lit. "time exam", it's normal that students are unable to finish those in time)
Create exams that don't just test the ability to vomit knowledge, but test the ability to use that knowledge, and let students explain in their own words.
Replace the exam with multiple small projects and presentations.
All of those things were used by different chairs / departments in my university :-)
Nothing will prevent "someone else writes the exam for another student" with absolute certainty, yes. But neither does proctoring software.
I don’t see a problem with that, you’re taking a test.
That said, I would not like to install this on my computer and would 100% use a separate user to do so. As long as it runs in the home directory I will trash it afterwards. I even use Zoom from my phone rather than installing it on my computer, but if I had to I’d do the same.
Examind.io
Unfair having to spend such money, but textbooks aren't that much more expensive, and you can use it for more than one course too
Edit: For example, remote sensors placed in a power plant or foundry where people work also would constitute surveillance. But it is in an environment where carefully calibrated machines can otherwise fail catastrophically.
To add, I suppose what I am really getting at is that it may be more useful to address ProctorU in particular rather than bring it under the same umbrella as other less used surveillance software.
For me, it made me realize that the word 'dystopian' in the title adds no facts to anything, its just a judgement. A negative one.
Any suggestions?
I have never seen it used over here (Poland). My friends from other European countries haven't either, at least when I asked them.
I wouldn't be surprised if GDPR prevents them from collecting most of that data. The fact that most colleges here are state-run might also contribute.
There are better ways anyway, IMO. At my (german) university, most exams are so personalized (random exercises from a pool in random order, student ids used as const values in calculations, groups A / B / C, etc) that cheating is pretty hard.
It made me not want to work at the company anyway. If they treated me like that during the interview, imagine your day to day.
Sounds quite depressing if true.
Example: design a database schema for a grocery store with such and such requirements... you either know or don't know how to do it but you won't find the answer on the web.
Well, I guess there's no way for me to use it anyway!
> Virtual Machines
It's my policy that closed source software gets installed in a virtual machine. Others need to abide by my policy if it's running on MY equipment.
Now I get why Facebook (Meta, whatever) decided to get into it. I'd not been able to piece it together until just now. Of course there's a spyware angle front & center. It all fits now. Their main market's probably intended to be business, but education makes sense, too.
There are many things which are technically possible which are not a favorable cost-benefit for most people. This is in the same category as those guys who relied on technically being able to fly without showing ID to the TSA — there's a reason why it was mostly affluent white men flying solo, because the potential downsides are much greater for most other categories.
The reality is that a sufficiently motivated cheat will cheat and will more than likely get away with it regardless of counter-measures when there's not a proctor watching over their shoulder[1]. If the goal is to test proficiency, there has to be better ways that work remotely.
Back in High School, "open book" tests were common in my Chemistry and Physics classes. The "open book" rule wasn't to help over privileged kids pass the test -- these were among the hardest tests; the book was useful only for referring to the myriad of formulas which a High School student was not expected to commit to memory, but if you were relying on that reference to tell you exactly what you needed to do to solve the problem, forget it.
This, obviously, falls apart in many contexts -- "open internet" tests are difficult to write in many subjects, simply because there are tools designed to answer those questions, immediately[2], but it likely just requires a little creativity.
From my own personal experience: I ended up taking a certification exam which was offered online due to COVID. They attempted to re-create the security of "testing in a center" by spyware and procedure (photographing the room from every angle, myself, my drivers license), all trivial to defeat if I was so motivated -- the procedure was even communicated in advance. And it was a certification for writing software[3].
I took practice tests, online, which turned out to be nearly word-for-word what I saw on the test. I knew the answers, anyway, but had I wanted to, I could have spent my entire study time memorizing the test answers -- not technically cheating, but the test has abjectly failed to indicate anything about my expertise. A more useful approach would have been to present me with a program utilizing the features that they wished to test me on, then present a set of multiple choice questions asking questions about that code. With a diversity of test programs and frequent changes, it would reduce the probability that a Google search would yield an immediate answer, testing the candidates ability to solve the problem using all of the tools that are available in the field.
Assuming that "testing is still the most reasonable way to assess skill", which is another argument in itself that falls victim to footnote "0", the point is to make irrelevant the forms of cheating that the spyware is attempting to prevent. Both are losing battles against cheating, but the former is less so, and certainly less consumer-hostile.
[0] The classic "Why do we do this?" with the most common answer being "Because that's what we've always done"
[1] Just off the top of my head, a small camera in the room aimed at the screen and a discrete -- in-ear, like my daughter's (Bluetooth) hearing aids -- ear-phone Bluetooth connected to a mobile phone to a third party in another room with a computer being used to research answers.
[2] And in a lot of contexts it still doesn't matter. If there's a tool that changes how that question is solved, and you're testing whether or not someone can solve that specific problem -- not whether or not that somebody can write a tool to solve that problem -- wouldn't it be more intelligent if they solved it using the most appropriate tool?
[3] Before I get grief; it was requested that I take it out of the expectation that I would require no study and was needed due to us being a MS Certificate Professional shop.
Watching the video that this article links to, a couple of the requirements are, among other things:
* To show the proctor all four walls of the room, and underneath the desk,
* To take out any earbuds you may have.
These requirements alone would probably mean your cheating solution wouldn't work.
(I hate that I have to say that because I do not want to advocate for ProctorU here, but in this case these requirements would probably do what they were intended to do.)
The problem is of course regional accreditation rules of proctored exams