Hacker News Reader

Comment by andrepd | Hacker News Reader

It's a fine balance. One one hand you want to let the students google material, consult books, use a calculator... on the other hand given free internet access what stops them from just asking (or paying) someone to solve the exam for them, or googling long enough until they found a solution someone posted online and just copy it, or cribbing from their mates?

A good balance, I found (pre covid), was having the students sit a test with textbooks of their choosing and/or their own cheatsheets and lecture notes. No bullshit cramming and memorisation (you can consult literature as you would elsewhere), but also no cheating.

View on HN

> One one hand you want to let the students google material, consult books, use a calculator... on the other hand given free internet access what stops them from just asking (or paying) someone to solve the exam for them, or googling long enough until they found a solution someone posted online and just copy it, or cribbing from their mates?

DING DING DING QUESTION OF THE CENTURY!

I don't mean that sarcastically, my point is this:

There is a clear, vulgar, mismatch of values here (and there sort of always has been). Shouldn't the purpose of schooling (especially higher education) be learning things, and not maximizing your score on arbitrary measurements of "performance"? I think it obviously should be, but that's not the game, because its non-trivial to measure and counter to the raison detre of the system at large. The very phrase "Competitive School" is a perversion.

That only makes sense if the goal of college and education is pure learning but we all know it's not.

Learning is part of it for sure but Companies use college as a filter is another.

Until jobs are unnecessary there's going to be this tug between learning and performing that's never going to good for either.

All I'm saying is the answer is never so simple.

Companies use college as a filter because it's difficult to assess proficiency during an interview and they'd like some way to cull the wheat from the chaff, and avoid wasting time talking to and hiring charlatans and posers.

Meanwhile, if you ask google-style brain-teasers during interviews (which can have a legit purpose, but it's become a cliché), people complain about that.

On the third hand, if you ask candidates do actual assess-able work during an interview, the extremes are "white board coding" and "take home programming problems", people balk at whatever you do for that too.

None of these are necessarily good, nor necessarily better than the others.

HN is lousy with complaints about all the different ways to filter and assess candidates.

Sure all the systems suck currently (except hire by internal reference, that one works great). Just because all the systems suck doesn't mean that something better might be possible. And part of finding the something better is understanding what doesn't work about the current systems.

Companies use university as a filter for long-term conformity.

That's much more pronounced in non-programming jobs, of course. For many white collar jobs you just need any kind of degree, doesn't matter too much in what.

Learning how to look up someone else's solutions to problems, or what someone else thinks about something, isn't much of an education. It might well be something to add to the traditional 3Rs, but it would be what you call a mismatch of values to expect a degree for it.

This is not intended as an endorsement of Proctorio, and its use of the DMCA in this way would be ridiculous even if the criticisms were without merit.

So a formal education is supposed to meet two needs: one, to educate the student. For this goal, we don't need to worry about cheating at all.

The second purpose, however, is as a way to signal other people that the student has been educated. This is what a college degree is: a certification that the person has been educated to a particular standard.

For that, we need some way to verify that the person has learned what they were supposed to learn. Tests are a way of doing that, and cheating prevents the ability to measure the success of the education.

It goes even further than your second point, for many professions:

We don't just want to signal other people that doctors, nurses, engineers etc. have been educated. We want to validate that they've actually learned what they need to know in order to not cause uneccessary injuries and death to other people.

Some skills truly require memorization. For example, being a good doctor requires incredible amounts of memorization. A traditional test is a good way to test memorization.

I would agree that not all skills require memorization.

Assessment is a usually necessary part of education. You need to figure out what your class knows. And a fair proportion of your students won't bother to really know stuff unless it'll be formally assessed. And, yes, also there's the issue with granting credentials: you don't want to give them to people who haven't reached a minimum standard.

I'm fortunate in that the classes I teach are centered project-based learning and ... not too easy to crib on. But if I were teaching core academic sequences, it's a ridiculous amount of work to make problem sets and exercises of appropriate difficulty and develop rubric for them.

You want to be able to leverage other peoples' work, and your past work, in building assessments. And you want to have a time advantage in grading: a 30 minute assessment that you give to 100 students that in turn takes you 30 minutes to grade each is a recipe for no fun.

But all of these things-- these economies of scale in instruction-- open the door to cheating. A well-proctored, well-designed exam can mostly patch over the vulnerability.

> Assessment is a usually necessary part of education. You need to figure out what your class knows. And a fair proportion of your students won't bother to really know stuff unless it'll be formally assessed.

I don't think this approach is right. There is an obvious conflict of interest when the same party provides (1) education) and (2) assessment. The better approach is actually that taken by the College Board -- they do assessment, and only assessment. If you'd like to do well on their test, you're free to get whatever training you see fit. And that training is, theoretically[1], evaluated by how you eventually do on the SAT, not by how well you do on practice exams provided by the educator.

This should be the practice everywhere else: credentials are given by assessment, and the assessor doesn't do anything other than assessment. If the assessor sells training, that corrupts their assessment.

[1] In fact, the system seems to have broken down; SAT training is widely accepted to have large effects, while actual measurements of those effects have difficulty distinguishing them from zero.

Hey, I love third party and objective measurements. But we can hardly give the SAT every week.

Educators need quizzes and other assessment measures. That way, I can know how my class is doing. My students can understand where their performance falls short. And their parents want to know how the class is going along the way. These let all the involved parties adjust for better performance.

And these interim assessment steps need to count for at least something to get a reasonable level of effort from students. (My classes are "fun", but that only goes so far: there's the fraction of the material that's intrinsically less fun, and there's the fraction of students that will not do anything without a grade incentive).

In turn, these quizzes and other assessment measures that I use, let me eventually score well on the metrics of engagement and scholastic progress that are used assessing me. ;)

> Educators need quizzes and other assessment measures. That way, I can know how my class is doing. My students can understand where their performance falls short. And their parents want to know how the class is going along the way.

Cheating on exams is only a problem for that last one, the parents seeking to know how their children are performing. If the educator isn't providing credentials, it's not a problem if someone cheats. They'll get less education, but that's their problem, not the educator's.

It really is a problem if someone cheats.

It's horrific for morale of students who are trying, and it's dispiriting as an educator to have no accurate idea of how your class is doing. And it starts a race to the bottom where cheating is normalized and everyone cheats.

Why? What are the cheaters getting out of it?

My answer above has nothing to do with what cheaters get out of it, but instead the harms it causes non-cheaters in the same class...

They get higher scores relative to their honest peers, satisfy their parents of having learnt material that they haven't, and erode the culture of doing the work in the classroom. They also leave their instructor confused about how to best address the class. (And, it's dispiriting, as a teacher, to have teed up a lesson based on an assessment that shows your students have this down pat, and then to find 2/3rds of the way through that they didn't actually learn that material at all).

I realize we're talking mostly about undergraduate relative to the article, and I'm a middle school/high school teacher so it's a little different. Still...

> They get higher scores relative to their honest peers

This is predicated on the assumption that, as an educator, you provide formal assessments of your students' capability, intended for third-party consumption. I keep saying this shouldn't happen. It makes no sense to assume, in the world where it doesn't happen, that it will nevertheless happen anyway. The assessment should be done elsewhere, by someone who is not responsible for educating anyone.

Under this structure, it would be possible for cheaters to mess up the teacher's assessment of their proficiency, and by extension the overall proficiency of a group of students. But it is strange to assume that they would do this, because -- I asked this above -- what benefit would they derive from doing this? If this action had any effect, that effect would be to move the curriculum onward from material they don't understand to material they understand even less. That is a harm to the cheater. The problem just doesn't arise.

> The assessment should be done elsewhere, by someone who is not responsible for educating anyone.

There's room for both educator-offered assessments and third-party assessments. As I said before, educators need assessments anyways. We have a fair amount of effort that graded assessments from educators predict future performance better than these third-party assessments, too (of course, the best predictor is both together-- e.g. SAT plus grades).

> Under this structure, it would be possible for cheaters to mess up the teacher's assessment of their proficiency, and by extension the overall proficiency of a group of students. But it is strange to assume that they would do this, because -- I asked this above -- what benefit would they derive from doing this? If this action had any effect, that effect would be to move the curriculum onward from material they don't understand to material they understand even less. That is a harm to the cheater. The problem just doesn't arise.

No one looks at middle school students' grades except the students and their parents-- well, and for eligibility for participation in athletics. Still, there's rampant efforts to cheat, with the negative outcomes I described above.

You're attributing a whole lot of foresight to a short term selfish decision. Yes, cheating may eventually bite you, putting you into a class you're unprepared for. But it provides a lot of short term apparent benefit, so...

> Yes, cheating may eventually bite you, putting you into a class you're unprepared for. But it provides a lot of short term apparent benefit, so...

What is that short term apparent benefit?

> No one looks at middle school students' grades except the students and their parents-- well, and for eligibility for participation in athletics.

You might be forgetting about high schools.

> You might be forgetting about high schools.

Funny, given that I teach in one! The middle school example was just to refute your point. Namely to show, despite the lack of degree-granting/admissions/pecuniary incentives to cheat, students still will cheat for short term perceived gains, with negative effects for everyone in the process.

I'm a-gonna go now, because I feel like you're being willfully obtuse.

Issue is, this system prioritizes assessment even more than actual learning. So this would be better for getting 'fair' credentials, but it would lead to people 'teaching to the test' even moreso than right now.

The current situation where the teacher is also the assessor allows the teacher to chose what to teach, to react to students, to teach for improving their students. Then at the end, almost as an afterthought, the teacher comes up with an assessment. This serves a bureaucratic purpose, the administration requires it. It also serves a motivational purpose "Study the material or you will fail the test", which helps the teacher to actually get students to learn.

This does mean the test isn't really focused on assessing how well the students have learned. But do we need more focus, in studying, on assessing how well someone has learned? I'd argue we need more focus on actually teaching well. Using tests to determine the quality of teaching seems quite fraught. Certainly the adversarial model of an assessment makes things harder.

To summarize, I think your solution would fall to Goodhart's law:

"When a measure becomes a target, it ceases to be a good measure."

Whilst also de-emphasizing the actual goal of teaching.

> Issue is, this system prioritizes assessment even more than actual learning. So this would be better for getting 'fair' credentials, but it would lead to people 'teaching to the test' even moreso than right now.

Why would that be an issue? Isn't that the point of the test?

The purpose of school is both learning things, and providing proof that you learned those things. In other words, education and accreditation. Therefore, you need some way to assess that the students did indeed learn the material so you can issue the corresponding certification.

The University of New South Wales (UNSW) wrote some case studies on online assessment during the initial Australian lockdown back in March.

https://teaching.unsw.edu.au/academic-integrity/case-studies

Some really interesting results:

> In my course’s final online exam, students have access to a Microsoft Teams channel as they complete their exam paper. Students are allowed to ask for help from each other, post specific questions regarding how to solve the problems, post photographs of their working out and indeed their answers.

> A near perfect asymmetric Gaussian (bell-curve) distribution was achieved in the final exam with an average mark, pass and HD all being typical for the course were it to be run on face-to-face on campus. Additionally, my course achieved 100.0% satisfaction in the myExperience student survey. My students commented that this assessment approach was a very forward-thinking approach to online examination.

Thanks for this, hadn't run into this source before. Passing along to some professors!

Happened to me a while back, but with an in-person test. They had a French teacher invigilating a biology test, and he didn't care at all. In the end, "every one" was working together, the solutions were written on the blackboard, and every one passed with flying colour, most of them with the same copy since they basically straight copied the blackboard. Of course, the test was redone (with a different invigilator) since the cheating was so obvious.

I am not sure how the Australian professor managed to get students to fail in these conditions. Either most of the students were inept and couldn't even cheat properly or the good students didn't want to share their answers.

After reading the linked list, that seems to be the case, the chat was monitored and the best students didn't want to share their answers. I think it works fine until someone cross the invisible line, and then widespread cheating ensues. In my case it was also high school vs university.

By employing such systems you are teaching not only the content of the class at hand, you also teach that cheating is worthwhile as long as you can trick surveillance and don't get caught. There is also some balance here.

Reply on news.ycombinator.com