Students have to jump through absurd hoops to use exam monitoring software
vice.com
vice.com
At least twice a month, their systems would be down, and nobody could take tests at all. We'd have to send home 30-60 students at a time, and reschedule their tests for another day. These were students who were also working jobs, and took time off of work, likely unpaid, to come in and take the test. So we wasted their time, and lost them money. Not to mention the constant waste of IT's time dealing with this trash software.
It's totally fucking criminal that these companies sell such garbage to our schools and get away with it. It doesn't matter how bad it is, it checks a box on some requirements sheet so we will never switch to something else... Not that the other options are any better.
I imagine it wouldn't be terribly hard to run the machines so that only one full-screen window is shown, or a user for whom the internet is disabled. Is that not enough?
Many of the anti-cheating software will also run the webcam and yell at you if it thinks you aren't paying attention to only the screen, while recording everything on your screen.
And students are being forced to install this on their home machines.
Wait, what if I work from home and my kid is sitting in the living room as well. I get recorded then? WTF???
Who owns these machines? Is it the parents or the school or the government or who? What if they decline? Can someone explain why a child has to accept that they are being watched on a webcam without their consent? Without the consent of their parent? And, "either accept it or don't get allowed at exam" isn't a choice. They ain't going to high school yet, but I don't envision I would allow my children to be scrutinized by this. Then again, who expected COVID-19. I can only hope this is temporary, but perhaps its part of 'the new normal'.
If the software detects that you're in the room your kid fails.
Your child will not be allowed to take the exam.
Can you explain this defeatist attitude to me and what purpose it does providing the already known negative outcome?
To me, the language you used suggested that you are already bought, sold, and controlled.
You take the tests, with the software, or you fail.
It's simply a compromise.
However, not everyone has that option, is willing to take that option, or understands the ramifications of not taking that option.
I hate the current system. I also understand why people choose to follow it.
Edit 0: It looks like maybe this conversation has hit the maximum depth for HN? If you'd like to continue talking, please feel free to email me.
Edit 1: Turns out I was wrong. The reply timer just hadn't passed. Still feel free to email me if you like.
Your question is about the current process though, and the answer is if you really do not have access to a room with internet that you can use for a couple of hours, the best thing you can do is to contact your school and see what arrangements can be made to accommodate you.
Not much. When lockdowns first started I remember seeing _tons_ of first and second hand accounts of all the ways anti-cheating software failed when exposed to the real world.
> I imagine it wouldn't be terribly hard to run the machines so that only one full-screen window is shown, or a user for whom the internet is disabled. Is that not enough?
It can be a little tricky. We put the test in a single full-screen window on top of any other windows, and...the students alter the transparency to view their notes behind it, or they notice that we only check windows in the current desktop or for the current active user. Maybe "full-screen" on the system in question only fills a single monitor and leaves another open for research.
All of that assumes that the student is only using a single device and doesn't have a hi-def pet silicon rock in their pockets at all times, hence the webcam monitoring and whatnot to check that a student is actively staring at the screen the whole time.
The webcam solution also seems entirely bypassable if the student is determined. (Speaking from my experience as a TA, students who cheat are usually pretty determined) Just put a phone to the level of the monitor, and there's nothing a webcam can do.
So, obviously, everyone just used their phones for cheating lol.
Once they gave one of their research subjet on dictionary optimization and "the best they could do themselves" compiled to show it was possible. I went far enough (we had to also be constrained to 16-bit memory addressing) and decompiled their stuff with IDA to find a few more tricks and got a little bonus for telling them and reimplementing them.
Another time they told us "do whatever with OpenGL and we ll grade you relatively", ensued an enormous competition between students to beat each other with my team ending up implementing a counter strike map renderer with skybox, full texture, fast enough to walk around, and we got the best grade.
The last project, on the 5th year, was a semantic web video search engine a la youtube, with fetching, indexing, auto translation, knowledge graph, for a team of 40 students who had to split in various department with one team acting as product owners, for a few months.
I think there are ways to do exams that are meaningful and actually make you interesting in job interviews.
The having to scan the room, scan my ears, scan my "desk" (or in this case my crotch because I was using a laptop) was all very silly, but didn't really bother me. The worst parts were all very human - they sent me the wrong link for the test, shared incorrect information about the allowed materials (IE: scratch paper, a 4 function calculator, etc), and then I waited for more than 20 minutes for the proctor to finally show up. Then of course there's the problem that the test provider has wonky weird software, including forcing you to install their "secure" test software. Which then yells at you for having a browser open that it necessitated to start the install process.
I guess what I'm really saying is - I find it funny that this is deemed "secure" when really I suspect most people taking a test have the same emotions as myself - we just wanna get to it and get it done.
So even a well designed
Sure the situation was still extremely contrived, but it wasn't simply regurgitating facts
For example you learn a depth first search and in the exam you have to store some information on each step to print it in the end.
You have to learn different proof schemes and use the easiest one for a given hypothesis. Sometimes you get with the wrong one in an endless recursion for example.
Being able to search things up is well and good, but dont you run into situations where you don't even know what to search for in the first place?
Every few years I need the quadratic formula for something, and I just derive the thing instead of remembering or looking it up. I’ve essentially traded some memorization for some understanding. We’re surely going to do it wrong, so I’d err on the side of too much understanding and too little memorizing. If you have a real feel for how a thing works, it’ll stick with you longer than the date of such and such battle.
Most people can't do this or have a very hard time learning much easier things.
I find that the crowd that talks about "pointless learning/testing/education" are often either ones that struggled mightily and were never really that smart, or they are so smart that they are above it.
It's much more important to know that there is a quadratic formula, or more fundamentally that every quadratic equation has 0-2 roots (exactly 2 if dealing with complex numbers) and what the different cases loom like (does the parabola touch or intersect the x-axis?). It's typically more important to be able to solve a quadratic equation through guessing, factoring or completing the square. Because these things teach you something about how mathematics works, regurgitating some formula serves no purpose, you can just ask Wolfram|Alpha instead. And even if you can't complete the square etc., I'd much rather people understood the conceptual side of it instead of remembering formulas.
The importance has always been "how its applied." Tons of people failed classes despite knowing the formula. It has always been about the basic application.
I don't think many math tests consisted of "write down the quadratic formula" and that's it.
But nowadays, you can just feed the equation to Wolfram|Alpha (or Sage, or Mathematica, ...), so there is no point in blindly memorising formulas.
And I don't think I've ever had a formula sheet in one of my maths exams in school...
As for the “most people can’t derive the quadratic formula,” you might be right about my blind spots, but I think it’s equally likely that I’m right and have the necessary point of view to see that most people can’t do it because it’s taught and tested poorly. Both explanations would equally explain it being easier for me to derive the formula than memorize it.
> to see that most people can’t do it because it’s taught and tested poorly.
Or they just don't want to learn it. Some people don't like math. Or maybe people are lazy - they like it but can't overcome procrastination to really learn it. Or they are more focused on something else, like I was as an adolescent (computers.) There are certainly people who got sick, or went on vacation during that week of school, etc.
My point is, we always like to blame things that aren't actionable, i.e. "the system." It was just "taught poorly." There are certainly cases of that being the truth, but if you look at the time constraints and all other details, it's hard to just blame the system. How do you actually fix the system?
Yeah, given all the constraints, I agree completely. My complaint is that there is a giant emphasis on testing in really scalable manners that take people and dialogue out of it. So which constraint is the system (the way things are taught and tested) most sensitive to? I expect that it's teacher to student ratios, which my thesaurus says is a synonym for money, simultaneously the easiest and hardest constraint to change :-(
If you think you understand something, but can't actually solve problems without referencing anything, you don't actually understand it.
The difference between learning X and understanding X
Often the problems had some trick that you had to apply to solve it. And without knowing those certain tricks from routine finding right solution was pretty hard.
Anecdotally, my favorite professor from undergrad (born in China, PhD from the best department in his field [my personal opinion]) said he thought the reason for Russian/Chinese dominance in certain areas of math was due to how those areas benefitted very much from rote practice. He advised all of us (American undergrads) to drill and kill certain techniques in order to build up our pattern matching.
I don’t think they’re advocating doing hundreds of worksheets on the power rule or trig substitutions or memorizing line by line proofs. Our brains do follow formal rules when doing math, but the insight necessary to find a way to solve a problem that isn’t straightforward isn’t through application of rules, it’s through a tacit intuition that you build up by doing lots of math. There is no other way.
It’s like how everyone feels like they understand physics to a PhD level while watching the Feynman lectures, but if you were to hand them any of the problems afterwards, what seemed like such a natural stream of thought is just simply out of reach. It’s much easier to go over something and declare “this makes sense” than it is to come up with that something in the first place.
The only way I got through my undergrad math was by doing problem set after problem set until the concepts were second nature, and the courses that didn't have a sufficient breadth of exercises to drive home fundamental concepts ended up being the ones I struggled with the most.
Applying derivations that someone else invented doesn’t require “pure genius”; it only requires you to be able to follow the person who discovered it, which is a lot easier than finding it yourself.
> The only way I got through my undergrad math was by doing problem set after problem set
It sounds like you didn’t really understand what you were doing then. It’s hard to phrase this without just sounding like I’m bragging, but I never had to practice doing anything I actually understood. If I felt like I needed practice, that was a sure sign I didn’t get it, which I always tried to fix with careful thinking instead of repetitive memorization
Obviously, but that’s not what we were talking about. We were comparing memorization to understanding, not inventing to learning.
If you’re good at math, you should be able to re-derive any formula or procedure quickly (up to, say, constant factors) without having to memorize it (after the derivation has been explained to you).
If you run into a problem that you can’t solve because you didn’t drill the steps hard enough, you don’t actually understand the problem. This isn’t necessarily your fault - many math courses teach by symbolic manipulation without the conceptual grounding required to actually re-derive the symbolic procedures yourself. Few students will seek that understanding on their own outside of class, in which case they’re stuck with memorization.
> the reason for Russian/Chinese dominance in certain areas of math was due to how those areas benefitted very much from rote practice.
I think this supports my point - these “certain areas” are small. There is a relative paucity of mathematical/physical innovation from China (especially per capita!). The west still dominates mathematical invention.
That is it is better to understand how to solve problems with references than without. You'll forget most you know but once recorded you can't forget. Then you just need to know it exists so you can find it.
On a side note, I really wish we had more emphasis on the conditions under which the linear approximations broke down. I remember sitting at the front of the MIT 2.002 class, and there was a demonstration of metal fatigue using a hydraulic press at the back of the classroom. Professor Sanjay Sarma stepped up to the front row in order to better see the demonstration at the back, and so I asked him about my intuitions about which way the model diverged from reality under vibration frequencies high enough that the quasistatic assumptions built into the model broke down. He looked to both sides of us and told the students on either side of me not to listen because they might get confused, and then we had a little discussion about conditions beyond which the model applied and which way the model's error went under those conditions. It was simultaneously one of the best and most disappointing moments in my education. It was an exciting discussion, but I was sad that the world beyond the linear approximations was considered to be likely too deep a rabbit hole for most of the class. Sanjay (as he preferred to be addressed) was an excellent educator, and I'm sure his judgement was based on past experience... each semester has a given complexity budget, and the field of Mechanical Engineering is so broad (statics, dynamics, thermodynamics, fluid dynamics, mechanisms, control theory/sensors/OpAmps, manufacturing techniques, design for mass manufacturing, numerical process control, destructive/non-destructive testing, etc., etc.) that undergraduates need to spread a limited complexity budget across so many subjects that they can each only be covered relatively shallowly.
Either you will understand permutations and combinations almost instantly, or you will have to do tons of examples.
Likely not applicable to all kinds of math but relatable to some concepts for sure
I have a terrible memory and I excel at math (and majored in physics in undergrad) precisely because it doesn’t require memorization.
I don’t know anyone who is good at math who operates by memorization.
No amount of reference sheets will help you with mathematics if you don't understand something.
If you think you understand some mathematical rule, but can only show 100 problems from memory, you don't actually understand it that well. If you can convincingly show why no counterexample exists, then you understand it in the strongest possible way.
Also, it was not that unusual for exercised based university math tests to allow references. Precisely because it does not matter all that much for difficult exercises, remembering everything is not the point.
Proctoring is not a solution that I am comfortable with. I do not want to peer into the private lives of students and their home environments. Not every student is well-off, and has a private space all to their own for 3 hours.
I think taking tests from home does not really work with any of the models I have seen discussed this year. Cheating is real. It has nothing to do with rote learning. I am out of ideas which are foolproof.
Ex: student is to implement a K&R C compiler + code generator in 1 semester. Help them break up the project into manageable chunks (tokenize, AST, register allocation, etc). Each feature includes tests, comments, docs, and is presented to you by X date. Code review and cycle as many times as needed as long as it gets in by X date. By the end of the class the student hit 100% of their deadlines but only pass 70% of some test C files you create so they get a 70% (or higher if their code was consistently good and did not need cleanup) in the class or something.
It would never work because "you need to have a final exam" but it's the ideal I think we should strive for if you are "teaching programming" since this is how most work I've done in the real world has gone.
There is some kind of a massive waste of money going on.
From the students perspective all they need to do is:
1. Write a "project proposal" where they define a project, define goals, schedule a checkin with you to see if that project is large enough (1/20th the grade of the class)
2. Watch lecture videos and go through exercises on their own time.
3. Write code and bring it in to teacher. (18/20th of their grade)
4. Demo their finished project to professor (1/20th their grade)
This does work for later classes. I never had the ability to test this workflow out for earlier (100-level) classes as all of those classes I TA'd for followed the same model you're talking about.
The 200+ level classes that followed the model I'm talking about had ~10% to 30% pass rate which was in line (~2x) with the pass rate of similarly leveled courses from our college's Math and Physics department which was used as a sanity check.
edit: that's not to say you're incorrect that it would be difficult to do at scale, this is just the way I've seen it done at moderate scale (20 to 40 students).
If you're asking the students to define unique projects themselves, you run into new problems as students dig up the most obscure blogs they can find on the internet to download and give you. It's a never-ending adversarial struggle.
This is especially true of junior data scientists (in my anecdotal experience so far)
While having individual projects is very nice and it works for smaller classes, it does not scale to average students.
I have once used individual projects to grade the ~100-student class and the amount of effort it takes is unbelievable.
For example, just your number "1." point, write a project proposal, is not so easy. It takes hours to write a good proposal and several back-and-forths between instructor and student to agree on a reasonable (not too advanced/not too basic) project.
Do you mind sharing roughly where this was (country and quality of university)? I thought we were in an era of grade inflation where everyone gets an A.
Part of it was related to their expectations of the course: "you mean I have to write code and the class is based off of that and not memorizing test answers? I'm out"
What I mean is - certainly you don't want to harm the reputation of your school, and you want your students to have actually learned the material and be prepared for whatever they do in life that requires that knowledge... but what I'm wondering is... if there's presumably some tiny percentage of a class that is going to actually cheat or act in ethically questionable ways, are we in fact putting more effort into the prevention of that cheating than is warranted?
I personally experienced this in undergraduate with a professor who chose to ignore obvious cheating and I think it negatively impacted the motivation of students to learn the material.
According to the International Centre for Academic Integrity, from 71,000+ US students surveyed between 2002 and 2015 at least 68% of students admitted cheating in some way.
It should instead be worked on making cheat resistant material
(unfortunately I don't know how to do it but I feel like the later is where the energy/brainstorming should be spent)
Stop giving bullshit tests with bullshit restrictions, which is usually what drives people to cheat in the first place. Have you considered asking: what can I change so my students are able to successfully learn the material such that they won't feel compelled to cheat? Recognize that cheating students are a reflection of your apathetic educational strategy and ineffective testing model.
Here's an example of a bullshit test which exemplifies a few of the issues with apathetic educators: single-try tests. Students are only given a single attempt to demonstrate they've learned the material, with the result ultimately reflected in the final grade. The apathetic educator is unbothered by having failed to adequately prepare his students, despite his knowledge of their inability to accurately self-asses in an unknown topic. A caring educator would provide a way for students to diagnose and fix any gaps in their understanding in addition to as a path for them to raise their grade to an A if they demonstrate an understanding of all relevant course topics.
An ideal situation would be allowing students to use online resources but not to communicate with other people during the exam.
Edit: typo
a) make cheating in effective, (no need to sneak in a formula sheet when everyone has it)
b) More like the real world, I always have google at my fingertips, it is just how efficiently I can use google and my existing knowledge to solve a problem
Looks like from ProctorU's docs[1] that they got around to probing for the (blatantly obvious[2]) signatures of a VM and stopped that trick.
I wonder if those signatures exist in a Windows Sandbox[3] instance, or if you can detect/block the Enhanced RDP Session Windows leverages when running a Sandbox...
[1] Under "Not Supported" at https://support.proctoru.com/hc/en-us/articles/115011772748-...
[2] https://bannedit.github.io/Virtual-Machine-Detection-In-The-...
[3] https://docs.microsoft.com/en-us/windows/security/threat-pro...
At that point, it seems to me that there isn't much left to distinguish the virtual machine from the physical. Behavioral properties of the CPU? Anyway, that's what I meant when I said it seems like after a certain point, it becomes impossible to tell the difference.
EDIT: based on the link above, it looks like current state-of-the-art doesn't go far beyond making sure names and common virtualbox performance shortcuts aren't present. https://github.com/hfiref0x/VBoxHardenedLoader/blob/master/B...
What else is there that could possibly indicate virtualization? Available instruction sets? Strange limitations of the given CPU? (e.g., the cpu presents itself as some Intel chip that's known to have 4 cores, but there's only 2 available)
[0] https://secret.club/2020/04/13/how-anti-cheats-detect-system...
- You run into performance issues - virtualization's speed depends on paravirtualized drivers, which are readily identifiable (e.g. "virtio" disk controller). These can be emulated instead of paravirtualized but will cost performance. Windows drivers are signed so you can't just change names of things.
- Chipsets - some are often emulated by VMs. For example, if you are running a system in 2020 and are using an Intel 440BX chipset, it's probably a VM.
ANGLE (Parallels Display Adapter (WDDM) Direct3D11 vs_5_0 ps_5_0)
My non-standard screen resolution is another dead giveaway. There may be others as well, but those two seem to be the lowest hanging fruit. And many of these proctoring suites run as browser extensions, and I'm not sure what additional information extensions have access to.
But yea, they're not too hard to trick. You could get fancy, around listening for mouse clicks and keyboard taps, without detecting any corresponding actions within the VM. But that sounds like far too much effort for a higher education software vendor.
A VM doesn't help you when your professor is requiring you to rearrange your room, set up mirrors, bash earplugs etc.
If you can generate convincing real-time fake video based on verbal commands like “ok, now show me behind the desk… now your left ear” you probably don’t need to pass an exam any more.
Not that it would really help anything.
Tests, like software interviews, are a poor way to determine an individual's ability in anything but synthetic tasks. They are cheap, though.
Except higher education software is known more for milking it's market than for it's robustness and sophistication. The type of proctoring software used here was trivially bypassed only a few years ago, when the last of my friends were finishing up college. I haven't been exposed to the topic since the last of them graduated, and from ProctorU's help center it looks like the vendors have wised up to the VM trick (in some fashion). But it can still likely be subverted relatively easily, rendering all of those onerous environmental requirements moot since they all explicitly ignore the computer itself.
I sort of wish I had access to it, so I could probe around myself and write up a tutorial. Not that I'm an advocate of cheating, but more so as a big "F* You" to the entire draconian process.
This is far less about cheating than petty power tripping. (They should, of course, feel free to abuse each other in what ever ways they find enjoyable. But this is really not my thing.)
Can I ask, in a totally non-confrontational way, are you successful in your field and used to getting what you want? Based off of nothing other than this post I would guess that is the case
Not because I encourage cheating, but because I should never have to install proprietary software on my personal machine to take an exam. I will however on occasion grudingly be okay with installing closed-source software inside of a VM.
I usually use VMs if I need to run anything closed source.
You don't need to be entirely vegan about closed-source to care about this issue though; there is a notion of a person trusting certain companies' closed source code over others. That is a thing. For example I would probably trust something like Photoshop slightly more than some random exam company's outsourced exam app.
Alternately, it seems that QEMU on linux does a pretty good job as well.
If that is any consolation, keep in mind that both you and they have a long way to go before graduation. Exams at your level are meant to assess whether students are ready for advanced classes. Cheating on them does little more than removing that safeguard, which can very well backfire. You're doing the right thing if you focus on learning the basics.
If you make exams online, you should change the format to one that is better suited to people having access to the internet. They did that with our exams, and it worked perfectly fine, sidestepping the issues with these exams.
My Organic Chemistry class had tests that were open-book, open-note, open-internet. Pretty much the only thing you couldn't do was talk to other students. Even if you tried to contact other students by the internet (which they didn't even try to police), there were several versions of each exam, so you were unlikely to find help.
They were still by far the hardest exams I took at University, because the questions were crafted such that you couldn't easily find the answers. Trying to look up a question in the book or online was more likely to waste test time than get you the answer.
Basically you could look up a very low number of things in the amount of time per question allotted
An exam that has too many people timing out just isn’t a well balanced exam.
Perhaps its unfair that they get no benefit for being better, no?
It would be better to just provide tons of padding to everyone.
Why make people ask for accommodations when we can just setup the infrastructure to make it unnecessary to begin with, at least in a wide variety of cases.
I’m sure there are some exceptions, but anything but the most generous time limitations in academic courses don’t seem to the improve the measurement of anything meaningful.
That sort of thing can't be done for all subject matter, however.
Basically, all work submitted, whether tests or homework, require a pledge with a signature that the work is your own and you didn't cheat. Violations of the honor system -- cheating, plagiarism, etc. could be reported by students or faculty and would be investigated by the Honor Board with disciplinary sanctions for violators.
One upside to this was unproctored exams -- although if cheating became widespread, the Honor Board would threaten to bring proctors back! The idea in general was that students were expected to become professional engineers, who are expected to have integrity enough when working in the field not to deliver/sign off on stolen or slipshod work. Proctored exams keep everybody honest while the proctor is watching, but do not address the concerns of personal integrity when you might believe you can get away with cheating.
The thing is, if it's perceived that it's easier to cheat than to learn, people will do that, honor system or not.
I admit that there is a difference between an institutionalised honor system, designed with respect toward and in collaboration with students, and a last-minute arrangement by an unprepared organisation.
Students have been putting up with a lot of nonsense from universities this year. Their trust might have eroded. If universities are not perceived as honorable themselves, a honors system is not the answer here unfortunately.
The credibility of your school went down by the same amount.
If cheating is rampant, either the culture is broken or you are selecting students wrong.
That was the point I was trying to make.
This is not a problem of a single school, or a single university. This is happening across the sector. The success of contract cheating websites, and the inability of universities worldwide to deal with them, attest to that.
> If cheating is rampant, either the culture is broken or you are selecting students wrong.
Sure. People are willing to risk jail to get their kids into top universities. A kid whose family borrowed money to send them study in a Western institution might not see honor as their top priority. It is not a student selection problem: the culture is broken all the way down.
Lest I come across as pessimistic, I am not. Academics may not be able to fix the structural problems, but we have more control over our own little part of our world than in the private sector. I, for one, don't rely on exams anymore for the time being and have not seen a GPA increase in my course. A honor-based exam system is not something I rely on, however.
I've heard of rampant cheating abroad, and it's part of the reason I believe some foreign degrees to be worthless. Truth is western institutions shouldn't admit from places where cheating is rampant. Else we're pretty much encouraging this broken culture.
(or maybe I'm exposing my weaker moral character than the rest of you upstanding citizens).
I think the vast majority of people who have entered some program of learning, want to 'learn' - and will try to.
Where it gets tricky is when you find yourself backed into a corner. Say you know (for whatever reason) you're 90% going to fail - do you accept that? Or do break out the Hail Mary cheat - what have you got to lose?
What's strange is that the moment you leave academia, suddenly this behaviour is acceptable. From the "Fake it 'til you make it" in acclaimed books, to the "carefully positioned demo that wireframes over the chasms to focus on the bit that exists - but you fully intend to fix the rest if the budget appears"
There's some collective cognitive dissonance that imagines all are equal, they go through the same trials of education (ignoring what people can afford, whether they have to work another job, they're commuting in from miles away and a thousand other variables) - but they can all be compared by their meritocratic grade at the end.
(before they enter the cess-pit of the working world).
I've reached the point where I've even lost my own point. Maybe it's just that the person who managed to cheat themselves to a pass against a barrage of technology, unveiled at the last moment, is a more rounded and useful person than the one that swallowed the book and scraped a pass.
This is almost the critical pass/fail decision in my opinion, because in the real world, on actually difficult or unsolved problems the correct decision is to escalate, clearly communicate you have an issue, and either be directed to someone who can get you unstuck, or inform resource allocators that additional resources will be required to fulfill their request.
There is nothing more destructive to organized, value producing work than to vouchsafe that it's a'ok hunky dory when it isn't.
I'm pretty sure I made it through some of my own coursework by the skin of my teeth because I could at least describe in enough detail where I was coming up short that the instructor had confidence that with more time and experience I'd get it.
At least, I think. I'm still a bit fuzzy on thoss last couple years of college from the sleep deprivation. I get bits and pieces of lucidity in between the recollections of "Oh God, I need sleep."
Theoretical Comp Sci is still "that one class I'm not sure if I really got anything good out of besides intuition into what is and is not a candidate for being computible", and Linear Algebra was "I've killed more trees writing out the work for these problems than anyone should reasonably have to."
Lol... so true. Some procedures take forever. Like finding the eigenvectors of a 3x3 matrix, for example. First the eigenvalues calculation, then for each eigenvalue there is a nullspace problem. Some trees will be hurt for sure!
Then I realize I have no trees in my yard, and that I should correct that.
I wonder if the extreme cost of university in the USA worsens this? If education was cheaper they might lament the lost time as opposed to lost time and a lot more money.
The honor system is also about expectations. Other universities expect you to cheat, and put extensive administrative controls in place to prevent that. The Stevens Honor System expects you to be honest, and to hold yourself and your peers accountable. So investigations and hearings for honor code violations are conducted by students, not staff. The administration doesn't get involved until it comes time for penalty enforcement.
So yeah, the end result is the same as any university: you can not expect it to bode well for your academic career if you cheat or plagiarize. But the structure is different, as is who ultimately bears responsibility for inquiring and investigating claims of academic dishonesty.
Professors frequently copy tests from other sources or previous semesters, and this obviously saves an enormous amount of effort, but ironically their copied work is what makes cheating on their tests so much more accessible. Professors make loud noises about cheating, and some are so hyperfixated on preventing cheating that they demand ridiculous requirements like proctoring software. They clearly aren't really prioritizing cheating prevention in their efforts, otherwise they'd write new tests more frequently (or at all!). Cheating prevention measures that don't require greater instructor effort are far more commonplace. When a professor literally copies a test, they shouldn't be surprised to see their students doing the same.
Tests that are suddenly ineffective with open internet access are already complete garbage for 3 reasons:
1. Some students will always cheat, and as long as teaching is remote, some students will always have internet access. Open access means that honesty is no longer discouraged.
2. Verifying granular details/vocab on the internet is completely normal in real work, even among experts.
3. Any test without resources like notes or internet relies far too heavily on memorization. This might be justifiable if there were evidence that standard test studying strategies (cramming) were effective at guaranteeing long term memorization. Anecdotally, I would fail the majority of my finals from previous semesters without time to (re)prepare. Students should be encouraged to understand the inner workings of the topic, in terms of analysis and potential states of that system, not to memorize dozens of details and vocabulary terms.
Within my Computer Science curriculum, the most effective and important content (between tests, projects, and lectures) has been programming projects by far. I'm dubious that tests are effective for any curriculum with relevant real world projects, though CS may be better than other subjects at making projects the primary measurement of understanding, and replacing the rigor that testing creates.
But if you do have repeated questions, there is software that will basically grep the internet for you to catch plagerism. All the students know that going in, so copy and pasting is no go. Sharing answers can land you in trouble if two students don't rephrase their answers well enough.
All of this can get you kicked out of college and potentially ruin your life with that mark, not to mention the money spent. Still, people do get caught cheating with those risks even before the pandemic.
Edit: my plan is to home school in a lumber mill / cnc machine shop. The strong will survive.
Caltech had tremendous success with a culture of honesty and take-home exams.
The shortcomings that seem obvious to me are: - Penalizes honesty - Implied notion that Caltech students are "honourable" and honest. How is this achieved in practice?
This is harder for professors to do, and maybe fails in intro courses, but generally is quite effective.
Else you might as well call the degree "rote" because that's what you measure.
> maybe fails in intro courses
If someone cheats their way through intro courses they are up for a bad surprise since the upper level courses assumes a mastery of the intro classes!
I'd still point out that the system is not robust against: - Working in groups. (Bad if tests are supposed to assess individual performance.) - Asking outsiders for help.
Also, your downplaying of "rote" learning feels misguided, no matter how advanced/abstract/high-level the domain in question is. Cue the Bruce Lee quote about 10,000 kicks..
We are so used to exams being about doing stuff that literally nobody in real life would ever even consider.
However, I'd assume that doing either during a test would be against the honor code.
Students hate carrying dead weight so to speak. Sure they might be tempted to swap favors (A helps B for a certain subject and the reverse is true for an other one). If the exam is properly constructed you might be able to detect plagiarism however.
Rote is absolutely necessary for a well rounded education, but I really feel the need to overcorrect in the opposite direction because testing for rote is the lazy approach. Over time a lot of testing has the tendency to shift to simply measuring rote.
Although most homeworks and exams aren't nearly that cool IME, they're virtually all about synthesis and fundamental understanding rather than rote memorization. This "first principles" approach to most everything is one of my favorite things about Caltech.
Sagan told Thorne he was writing a science fiction novel, needed FTL travel, and asked Thorne if he could suggest something that would be reasonable. Thorne agreed to look into it.
After the conference, Thorne spent a while working on it and came up with a wormhole approach and worked out the physics of it.
In addition to giving it to Sagan, Thorne also put it on the Ph 236 (General Relativity) final exam. He didn't tell the students on the exam that it might imply FTL travel. He just set up the conditions and had them work out the physics. Most of his students succeeded in that, but he was a little disappointed that none of them happened to notice that it implied FTL travel.
(I got the above from an unpublished book Thorne was working on in the early '80s. It was a collection of biographies of and interviews with physicists, astronomers, cosmologists, etc., written and conducted by Thorne. He had the draft chapters and the raw interview transcripts in a world readable directory on the physic's departments VAX, where they were widely read by the rest of us with accounts on that machine).
The real problem is people paying others to take exams, or simply copying answers from others. I work as an independent tutor, and I get contacted far too much by people looking to have me take their exams for them.
I guess what I'm trying to say is that the "implied notion that Caltech students are... honest" is, in my experience, largely true (possibly because Caltech tries to select people who think the Honor Code is good from the get-go).
Clearly, any society collectively benefits from honesty, whereas an honest act is, at least in a strict game-theoretic view, a loss to the individual in the short term.
The question "Are you honest?" does not necessarily filter out dishonest people. (Incidentally, I always felt like I would not have been admitted to Caltech as a student.)
I considered this an enormous success. It created an atmosphere where we were treated with respect, and where we were expected to treat others in the same way. If someone did cheat, then they would certainly be too ashamed to admit to it. It was a community where I was proud to spend four years.
Yes, I would say that Rice students are honorable -- without the scare quotes.
How is this achieved in practice? Wish I knew. This is the sort of thing that every university, and every organization, wants to achieve. Trust is very difficult to build if it's not already there; and it's much easier to erode.
You were accountable to each other, not to an authority. Authority gets you enough to do the bare minimum to not get fired or not get caught.
Being accountable to each other feels very very different. It's tough to explain in an HN post but anyone who has felt the difference knows what I meant. You care what your peers think. You care about that authority only enough for them to leave you alone.
Never let it be said that shame isn't an effective tool.
This is true, but at the same time I think shame as a motivator is often really unhealthy. People can do all kinds of terrible things to avoid shame: at the extremes, witness things like family annihilators or so-called “honor killings”.
I’ve been reading a book about Midway lately so this is on the brain, but a culture of shame is part of the complex that prevented the Japanese from reeling in insubordinate but “patriotic” junior officers, and from realistically assessing their own and their opponents’ capabilities. Meanwhile the modern process for evaluating air accidents (constantly applauded on HN) explicitly removes the question of blame and shame.
So, I’m not really saying anything about Rice university. I’m sure they don’t have to worry about honor killings on campus. This has just been on my mind lately. Just saying that without social release valves for shame you can get some really wild consequences.
PS for anyone interested in Midway, read Shattered Sword for everything you ever wanted to know about early-war Japanese flight deck operations.
I think my take-home from this discussion is that the honor code can be made to work in the right circumstances that exist, at least, at Caltech, Rice, etc.
At the same time, I believe it is impossible to induce the requisite "cohesion" in other contexts such as 100% remote learning or high-stakes mass testing (entrance exams etc.), even if the student body stayed the same.
That's because it's done wrong.
There are countries where they simply sort by descending scores on whatever standard test they came up with to admit. As if the test was perfect and results could be compared at an infinite decimal place (hint, stats and physics disagree with that!).
So the test basically becomes a measure of how someone is good at taking the test and you start to see min-maxing behaviors. It's a contest to see who can pour the most time into maximizing its results. The incentives for cheating are simply so high. The downside of cheating is that you don't really learn, but since the only skill these tests teach you is taking the tests, you are really robbing yourself of a skill that becomes useless 3 minutes after the test.
My advice is to throw these tests in the trash.
The council does have teeth. My friend was sent in and found guilty of cheating on a test. They got kicked out in short order. The straight-A student they'd allegedly copied the answers from got a few months' suspension. I never got an explanation as to why the sentence differed.
No system is perfect.
A professor questioned me as a witness at the "student council" hearing. I expected a student to do the job. The questions were all worded to make my testimony skew towards a guilty verdict, calling my own integrity into question. This is expected if there was a mechanism for an organized defense, but I was never questioned by the "defense attorney". I left the questioning disturbed and frankly afraid for the safety of my own academic record despite not having done anything wrong. Guilt by association. It felt like a witch hunt and a front for implicit faculty power. My friend still denies wrongdoing and had to rebuild their degree from scratch at another institution. They are still proud of the time spent at Rice, even though they feel their degree was taken away unfairly. The only evidence I heard against my friend was that they had made the same mathematical error as the other implicated party.
My first take-away from the situation was that organizations based on trust work well, but it is critical to have robust mechanisms in place for dealing with moments where that trust gets called into question. It is human nature to take trust away faster than it is given, even if it turns out there was no wrongdoing.
My second take-away is that organizations based on trust tend to punish violations disproportionately, especially if the violation reflects badly upon the group.
My third take-away (only realized years after the fact) was that trust-based systems tend to create a tyranny of implicit rules that tend to exclude newcomers (like students) unless specifically addressed.
I'm still very much in favor of a high-trust environment, but it is far from a cure-all. I find it's a prerequisite to a robust system, but must be paired with counterbalances.
For my beef with the Student Council, Rice was an overwhelmingly positive and nurturing experience. If I ever decide to get another degree, that's where I'll go, and I always recommend it to prospective students with an independent streak.
Yet they will be less likely to commit suicide, you know? Or some other act of desperation.
It’s a success because it lets students quietly cheat - if they need to - without reducing the graduation rate.
This is simply because the set of problems a person can solve in 3hrs limits the scope of the questions that could be asked. In fact, my friends and I would make lists of things we were sure wouldn’t be on the exam due to time constraints.
I have to say that nobody cheated. This was partly due to my physics graduating class being 12 students.
"Can I use Feynman?"
Imagine how much harder interviewing would be without having a one on one conversation. And if it had to be perfectly standardized nationwide. It would have a ton less bias, but you’d have a terrible signal and have to do all these crazy things to prevent cheating. In other words, these seem even worse than the state of tech interviewing.
Just enough steps to have a degree of assurance that I'm not cheating, but not going overboard.
I do not do anywhere close to my best work with someone staring at my every move.
For undergrad, I'm much more in favor of deliverable-based approaches. Term projects, problem sets, etc. I think it's cruel to boil such a high-stakes decision to a single-point-of-failure event like a big test. I've heard too many stories where someone had a bad event happen before a test, or just slept poorly, or made a trivial mistake, or got sick, only to be told "too bad".
If cheating is a huge problem, locking down the final step isn't going to fix anything in the long run. It destroys all trust. We need to take a look at the motivations to cheat, even if these reasons are uncomfortable.
And some people thrive with others around.
Eventually you'll find an excuse for why anything is unfair..."too much pressure!" "not enough time!"
And we wonder why, by and large, college degrees become more worthless, and academic vigor sinks like a stone.
Maybe we should just accept that college should be hard, that tests should be challenging, and instead, remove the extra requirement for college for most people. Because if we just make it a joke it's pointless anyway.
Because oral exams are quite time limited, you really only have one shot at each sub-question. And how good you are, is 100% subjective to the people judging you - and since this is in person, lots of things can influence their judgement. Hell, they may even have some latent bias.
And as others have mentioned, it brings a bunch of stress on the candidate.
I guess it's a very personal preference. I always did better on home exams, the stress-level was pretty much non-existent, compared to the others.
I'd much rather just assign take-home exams and final projects, or use a synchronous 1-hour exam that everyone takes at the same time.
If your a professor that cares about Google give a rapid oral exam. If your a college professor giving multiple choice or other memorization based formats, what value are you providing over free online courses? Frankly, if your institution "has" to use this sort of software, it's not worth attending.
> if your institution "has" to use this sort of software, it's not worth attending
Because of the sheer number of applicants in many places, whether a person has a relevant college degree and how much they scored in college is often the first "filter" jobs apply while sorting candidates, and if you can't get past that, it's essentially impossible for you to get a job.
Students will stop going to college for courses that don't prepare them for the job market when the job market stops requiring them to do so, which definitely isn't the case right now.
Being tested by people who mostly have never even worked the job you're supposedly preparing for using methods that don't represent the way you'll actually be working seems a complete waste of time and money. Increasingly the more talented individuals I'm working with in my job I see more and more that never even went through any of that in the first place.
I can imagine a way that it would be valuable but really you'd have to start from scratch, there is too much cruft.
The more time I spend working in my chosen field, the more I appreciate the time I spent studying unrelated subjects.
Nitpick but noise-cancelling is by definition an active electronic process, and so will never be a purely foam ear plug. This is Vice misreporting what the professor said (professor's words were "ear plugs for noise reduction").
This does of course put the burden back onto the teaching staff, give them a nice bonus to mark the work. Given school can shutdown their buildings they can save on costs such as heating, air conditioning, cleaning, electric, gas, maintenance, etc - there should be a little extra budget left-over for a marking bonus.
What if the client side of this program was open-source, and all it did was record the screen and web camera, and upload a copy to a server that was run by the school on premises.
Would this change your view on the software or make you more likely to agree, however begrudgingly, with the use of this kind of program?
It's good. I remember learning about networks to pirate stuff back in the day and that's much less of a thing now. Now kids have their own incredibly stupid invented problem they can easily work around if they learn some tech stuff, and a lot of them are familiar with obs from streaming anyway.