take-home, closed-book type
What an oxymoron. I agree with the others here that AI isn't the problem.
What an oxymoron. I agree with the others here that AI isn't the problem.
We had lots of time, and a fair idea of the range of questions. It rewards actuality mastering the material vs memorizing it.
For the CS exam some people brought more books than they could physically carry, I don’t think it helped them much.
In addition I had to defend the report (120 pages of typescript and charts) of my final year project to my supervisor and another senior academic. And it was clear that they had actually read it.
All those exams were open note; anything in your own hand or a copy of a lecture handout was permitted. Again the weaker students would not have been helped by more time because they hadn't understood that you have to have enough familiarity with your notes to be able find the right information. Some brought in 50 litre rucksacks stuffed with ring binders and the noise of them furiously leafing through was enough for the invigilators to warn them to make less noise or risk being ejected.
In Norway it is typical that an exam of similar standard allows five hours.
Pretty much everyone still took three hours. But you felt so much calmer, knowing you weren't racing against the clock.
It's not a problem with homework assignments as they have multiple days to finish.
So the professor has to decide between poor data with high variance, or good data with lower variance.
> I'm pretty sure that having more time would not have helped the weaker students get through the Quantum Mechanics exam.
Funny you mention that. When I did my (undergrad) QM II exam, I likely got the highest score, and I'm sure my score was below 40%.
There just wasn't enough time.
Such scores were expected when I was studying. No one got 100% in the kind of exams we had, not even those who got firsts.
The problem is that the professor has little clue on who can and can't solve the problems. Everyone doing poorly on a tightly constrained problem is mostly noise.
Don't get me wrong - I'm sure people cheated even in my day. But this is the spirit - they're trying to give problems as challenging as they would for homework - and a lot of those classes have very challenging homework problems.
EDIT: Rate limited so: "smarter" here means how well the student will perform in a career in their chosen field. Fast + accurate is the ideal.
Assuming this is true (others have already addressed a few of the many reasons it might not be), wouldn't that imply that practically all existing tests are already flawed? If you want to grade on time, every exam should be graded with a formula taking both the time and the correctness into account. A binary "fast enough" vs "not fast enough" is about as useful as a pass/fail class grade.
The exams that felt like the fairest reflections of my own knowledge were proctored in-person, closed book, and time unlimited. Of course, being time unlimited works better for quantitative/engineering exams. I haven't put much thought into more qualitative/liberal arts type exams.
It always struck me as an ingenious way to get a feel for a new group of students, since it's quick to administer and equally quick to grade, while revealing quite a lot of information.
Certainly we hold speed to some regard, since a lot of academic accommodations involve granting extra time. If we aren't testing on speed, then surely we should give that to anyone that asks?
Dubious assertion.
> "smarter" here means how well the student will perform in a career in their chosen field. Fast + accurate is the ideal.
Still dubious.
Also, I don't know where you work, but in most of my jobs, career growth is not limited by speed with which you do the work. It's one factor among, say, 10. Most of the people who got ahead were not the fastest.
They're not trying to gauge who is the fastest. Or even the smartest. Just those who have the skills. In the real world, you'll rarely (as in, never) have to solve those same problems with the same speed you will in the exam.
In a lot (all?) of the jobs I worked, taking a day to solve a Medium level Leetcode problem was quite OK.
Lots of reasons a slow student can be just as smart or smarter than a fast one.
And then consider whether the point of the class is to test smarts, or something else.
I’d expect that’s not the intent of most undergraduate degrees.
Memorization is already hacking the rules of the game that's supposed to be gauging understanding. An ideal test is resistant to rote memorization as well as outright cheating.
I do agree that the idea of giving take-home exams and expecting students not to cheat has long since passed its prime. There may have been a time when it was reasonable to expect most students to behave honestly, but that does not hold at all in today’s climate. Especially post-COVID, for whatever combination of reasons, students just don’t seem to care at all about anything other than min-maxing their effort to grade ratio.
Which leads to a lot of outcomes like this one where students start using ChatGPT early, think it will continue forever, and then get completely crushed when they encounter the first assignment where they can’t cheat.
One particularly striking example was a timed, closed-book math exam where the exam paper just listed four problem numbers from the book. We were expected to open the book, copy down those problems without looking at anything else, and then put the book away. Honestly, that one felt like just asking for trouble and now that I'm a professor myself I think that the prof was unreasonably lazy, but it certainly captured the spirit of the thing. (Technically, that wasn't a Harvey Mudd professor but rather a professor at the adjacent graduate university. Maybe that had something to do with it.)
Some people get up in their jimmies about this but if they don't want me doing something they should make it impossible to do it
</bait>