Hacker News Reader

Would the solution be a shift in the expected outcome of testing to:

A) Project-based testing proving critical thinking rather than memorization, and

B) Testing with the understanding that as long as you can find the answer via Google, notes, or a book in a reasonable time frame, this shows you understand enough to proceed? Is this not how much of the working world works as it is?

The problem is that you must make sure you actually test the student, and not some other student, or a team. That's even harder for "take home" projects, because the time for exchange is so long.

I try to formulate most of my questions to not be just knowledge. With higher classes, I care less and less about how much they know, or just know how to find it, because the real problem is figuring out what they need to know, so to speak. For lower classes, this is a lot harder, because these classes often teach the basics required. You cannot be a good physicist if you have to google F=ma every time. The same way you can't be a good programmer if you SO-search for -loops.

That would be better, but it's harder to set and mark the test. Also it doesn't solve the problem of having a better student take the test or coach you through it.

I did this in my university teaching with (in my estimation) some success, even with large cohorts (1000+). But it takes a lot of design effort and friction with university administration. One extra element was to incorporate reflection into the project specification, teach students how to write reflectively and to assess it.

> How good is their internet? How do you handle "my internet went down"? Or: I'm in this other timezone, your test is in the middle of the night!

At Warwick, we gave 24 hour windows for most online assessments. This resolves the timezone issue. Internet/connectivity and other claimed technical issues are handled by the existing mitigating circumstances process. Students are expected to provide evidence, and the development team is also involved in these cases (providing log data which may or may not support the claim). We have pretty good technical data, so it's usually fairly obvious as to whether the student's claim has any merit or not. The main advantage we have here is that we built the entire platform in-house when the realities of Covid-19 became apparent.

Some departments have looked at "viva voce" interviews after the exam as an anti-collusion measure. We already had the infrastructure to do source matching on e.g. coursework, so typed-up exam solutions were also fed into Turnitin in the same way.

Is it flawless? No. Students can still technically start the exam early and share the paper with others who start later. But I'd have expected to see much higher mark averages if cheating was widespread - and some of the informal discussions I've had with teaching staff have suggested worse performance this year, if anything. Part of this may be due to the incentives to cheat not being particularly strong due to the "safety net" policy which was implemented - students cannot be disadvantaged by taking the exams - their average mark cannot decrease, and will revert back to a grade based on non-exam assessed work if they perform worse.

Yeah, we also have seen generally worse performance in most courses -- I think it was really hard for the students to study at home, for many reasons. At the same time, we saw clear evidence of cheating in some courses, like rapid point increases at the last quarter, timewise, of a test. And people uploaded their exams for help, in a way they could be identified. These cases are now with the administration. Not sure what will happen to them.

Students cheat because incentives aren't aligned - if I don't learn my intro physics material and I'm not taking any later classes that depend on it, the only difference between cheating on the exam and not is whether I end up with a degree. Employers, graduate schools, etc. want to see that degree, but they don't actually care whether I know intro physics.

As you say, this is less of a problem in higher-level classes: you ask them deep and meaningful questions that build on the intro material and you see if they can give a cogent answer. This process already has its natural conclusion when we ask Ph.D. students to write and defend a thesis - but almost until that point, we continue evaluating students based on homework assignments and tests with clear right or wrong answers.

It seems to me that the answer, then, is aligning the incentives. For any classes that have only textbook work, tell students that everyone passes them, and while homework and tests will be offered, they're mostly there so students know for themselves if they're ready to take higher-level classes. If you want to teach students to be students (which is a valuable skill on its own, although most of them should have figured it out in high school...), find some freshman-level classes that don't have textbook work - say composition or research - and grade those. As the years progress, move towards classes where students offer free-form responses and assign each student different work. If students seem like they didn't actually learn the intro material, go look at whether they took exams and how, and then send them back until they do. A few will party all freshman year, sure - but they'll learn that their sophomore classes do actually mean business now.

This does, incidentally, involve getting rid of the general requirements classes unless you can evaluate them in a non-textbook way. You probably can just work it into the major - biology or computer science or music composition students will genuinely benefit from understanding introductory physics, calculus, etc, so test that in those classes (and make it clear to them they need to achieve a "passing" grade in those classes). If you can't, and you also can't ask students meaningful independent questions, there's no point in having your degree require the ability to memorize "the mitochondria is the powerhouse of the cell." I passed my college biology class with a C, and I am unsure what exactly would have been different if I'd put in the work to get an A.

I think they do care if you know intro physics, well I guess it depends on what you are doing but if you are going into a related field in graduate school or industry they probably do want you to know this. Unfortunately, they don't have a good way to assess this skills/knowledge so they use your grades as a proxy.

This is a neat idea, but basically puts very high stakes on fewer assessments, and these assessments differ per student. So they will naturally vary in difficulty, and depend a lot more on the instructor, introducing more subjectivity into the process. I could imagine many serious issues of fairness "I got an impossible problem" and "who is the easy professor" in these cases.

>There are only proctored exams, but they only help if you use a locked down browser. I'd rather not put what amounts to spyware onto my students computers. AND they are pretty easy to fool anyway.

Couldn't they also just use a second computer?

Often times these browsers also require microphone and webcam access and track eye movement. Huge privacy violation, but I also don't have a proposal on an alternative other than proctored in-person tests.

Yes, or their phone. I mean, they could monitor your eyes with the webcam, and if you start rummaging on a different keyboard/mouse and looking away, this might be detected. But a phone? Easy to hide! What do you do if they want to go to the restroom?

Additionally, it puts another "barrier of entry" in there: Rich kids can buy more electronics to cheat better. Or pay somebody to do the manipulation for them.

All of the proctored exams I've ever been aware of have either a human supervisor in the room at all times, and/or a video camera pointed at the student the whole time.

If you open a second computer they should know about it.

KVM switch with PIP?

Soon, a locked down VR headset with eye tracking. It's step one for the matrix, maybe, but it does make it hard for the student to check their phone.

But employers will be tempted to use the same tech to measure employee time-on-task, and write employment contracts accordingly. Parents and teachers would want to use it to monitor study as well as exams.

The potential for social control is rather alarming.