Professor denounces mass AI fraud on an exam at Brown
english.elpais.com
english.elpais.com
I have written an article on how I have adjusted my classes to the situation:
https://htmx.org/essays/universities-and-ai/
Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.
In addition, at many institutions such tests are given infrequently and can be worth a significant component of a student's overall grade, increasing both student stress levels and the tendency for such assessments to measure short term knowledge students have obtained by cramming, not more meaningful longer-term knowledge gains.
I see you're giving quizzes every three weeks, which is better than twice a semester, but still not what I would consider an ideal cadence. In my course weekly computer-based quizzes comprise 70% of a student's grade, but that's supported by a significant institutional investment in high-frequency computer-based testing: https://cbtf.illinois.edu/.
Is there a form of assessment that is a good signal of the intellectual ability of all students?
Yes, students can raise their score by cramming, but won't be as fast or as fluent as a student who has learned and internalized the material over the term.
Forcing hand written should really not be necessary. It would be very cheap in terms of old computing hardware to set up a test room with old desktop PCs that have wired only NICs (with a network connection that goes to a switch in the same room with no uplink, connected to a decent size laser printer only), running something like lubuntu and libreoffice writer as a basic word processor.
Let people at least type their essay with the standard features of a word processor as usable as MS Word 2000 or better.
If you do let them bring their own keyboards, how sure are you that those are just dumb keyboards?
Even a setup with cheap Raspberry Pi's? And Vim. (built-in bonus points for students being able to save and exit). Was just reading about LEAP[0][1] that thought you may find interesting if you have not yet seen it.
Separately, wanted to say I enjoyed your essay and appreciate how you think about improvement with a focus on the practical usefulness for the student.
TL;DR: I especially think the concept of Interview-Based Grading would be the way to go.
A couple progressive Whiteboard interviews for CS a semester, power point for Business etc. If they use AI to prepare, create material or learn/improve from then they are gaining the skills that will be required in real-world work. If they choose to not learn the material (or understand/check AI output) it would show during the back and forth with a panel of professors (dunno if you guys like working together though); built-in load sharing with a wider range of questions.
I guess with issues you mentioned the whole system needs overhauled for it to work. Adjusted for different courses -- and I dunno how practical this is but -- but why not let kids do more guided self-learning, keep office/lab hours for individual help. Fewer lectures due to self study with repurposed time for live evaluations. Eliminating non essential courses a student takes for a major/minor would also lessen the collective workload of professors (fewer student, but ones that want to be there).
That learning & evaluation set-up would be my dream college -- I would have pursued completely different life in that setup. I wanted to learn and engage, not prove I can regurgitate rote-memory notes that are handed out while my employment required me to grasp changing context and modify the code while not unwittingly automating the deletion of production dbs.
[0] https://www.raspberrypi.com/news/leap-low-bandwidth-educatio...
If we get to the point where even that doesn’t work then we’ll be at the point where a camera in the room with an AI analyzing eye movements should be able to detect it. And no matter how advanced they get they’ll still need to radiate heat, so a thermal camera should work. If that fails, industrial CT scanners are getting cheaper and cheaper.
Heck if it gets too bad there’s always mm wave body scanners and a set of cheap glasses kept at school.
We had a 'fix' for this back when they checked your TI83. It's a 2 line basic program to display "MEMORY CLEAR" the exact same way as if somebody had spent the time to find the actual memory clear function in the settings.
Also I was in high school in the prime time for this hack and I never saw someone actually use it despite the stories.
Initial cost is almost certainly not a factor; the components could be so old as to be free.
Playing devil's advocate: one reason I'm good at typing on computers is that I learned on typewriters, which need much more force for each key press, especially the little fingers. Someone who learned on computer keyboards won't have trained the muscles on their fingers as much, and will have a hard time pressing the outer keys strongly enough (and if you press the keys too lightly, the mechanism won't hit the ink ribbon with enough force to transfer the ink to the paper).
I also challenge that "hand writing is harder to revise"; again, why is the speed it takes to write it at all relevant?
What? Suppose you want to fix the opening of your essay. Best case it's pencil and you can erase some, but worst case you have a longer sentence you want to put in there so you can't do it without scribbling all over and making a mess of the page. Word processors let you edit. How is this controversial?
> why is the speed it takes to write it at all relevant?
Okay, so from first principles:
1. Time is finite, we will all perish
2. Unless you are doing open book, needs to be supervised (proctored / or prof/TA is there)
3. That person is paid for a shift
4. That shift must end
5. Therefore, anything that enables people to write faster is good for the students, who can get more paper written, or the paper better revised, during the finite time available for writing.
You don’t write the test to fill the 60 minute slot. You time it so students are able to finish early if they’re really good. Slow ones need the whole time but can still do well if they understand the material being tested.
- someone with a good understanding will often come to a concise, clear answer while someone struggling will produce a convoluted paragraph.
- the way to get to the result will vary depending on your understanding (e.g., are you blindly applying some method or understanding what's going on). For instance, "hey, this is a vector field, I don't need to evaluate this complex integral, I just need to compute the difference between the start and endpoint of the curve!". Both answers will be correct, but one denotes a much better actual understanding (and will take way less time).
But then I'd use that to write the actual text answer, and consequently, it was mostly clean. I'd still have to cross stuff out or add a phrase between the lines every so often, but once the basic structure was done, that was mostly unnecessary.
And if I was ever pressed for time, I'd usually get part marks for the outline anyway. Depending on the subject, I'd sometimes race through the questions doing just the outlines, and then come back to write the full text as a complete second pass.
To be fair, this wasn't creative writing. I think it'd be harder when the expression is the content.
I think you're missing some logic in there somewhere. If a student fills in as many pages as they can, each with the number 1 written on it as large as possible, was this a good conclusion? The quantity of writing is not a good metric.
EDIT: to give a closer to reality example, an essay that's 4 times longer than a competing essay does not make it better.
Most curricula should probably feature both forms of assessment, demonstrating your knowledge of the basics in a closed book assessment and your ability to produce high quality final products using all resources available to you in take home assignments.
You're thinking of the output as the goal rather than as the means to an end.
Honestly, school essays, as a text genre, suck to read. They are not meant to be enjoyable, beautiful texts! Instead, they are meant to help gauge the student's ability at text planning, the quality of their text project, their grammatical knowledge, and orthography.
I must say I'd be open to seeing schools attempting other models, though, and measuring the results thereafter to see in practice what happens when students type their essays instead. In terms of text projects and planning, that could be interesting and train different skills.
But this is missing the point. The issue is not that people in general can't write by hand for hours, but that these particular people may have difficulty.
Define “worked” there are reasons we have abandoned quite a few archaic practices which turned to be highly ineffective in hindsight (and often quite misery inducing). I’m not implying that handwriting is one of them just that it’s a terrible argument.
> burden of proof is on the side saying that kids
Smacking children who fail to give a correct answer with a stick worker for thousands of years as well..
That's a strawman. I said bottlenecking testing by requiring hand writing is stupid. Put another way - you're meant to be using the test to be thinking and re-thinking the problem and articulating approaches and solutions, not use your time, effort and energy on managing quill and paper.
Testing for understanding requires the fastest WPM possible. Regardless of the method used, students don't all write at the same speed. To minimize the influence of this variance, they need to use the fastest writing technique possible. If a fast writer spends 30 minutes writing in a two hour test, and a slow writer needs a full hour, the fast writer gets 50% more thinking time. But if you double both their writing speeds, the fast writer's advantage drops to 17%.
The faster the writing technique people are allowed to use, the better your test can identify poor understanding.
It's fine (tho annoying when you lose points to "typos"), but it limits what kinds of problems you can reasonably put on the exam. You'll definitely lean a bit more into theoretical stuff than practicals. Which is fine for some courses, I think a bit less interesting in other courses.
Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example
Having said all that... "we've booked the computer room, you don't have internet, go type up all your stuff in this VM we have set up" feels fine if you don't like this constraint IMO
How practical is this?
1. Is your institution able to provide this support?
2. Do you believe you are able to supervise the room well enough that students will be caught if they cheat? (Eg, bring a phone and look up answers.)
I don’t know about 1s practicality. In my schooling it would always be doable but I have the impression US schools are a different scale
If you can escape the vm on the fly and manage to use an llm to cheat, you deserve that A.
This is basically how ICPC works (https://icpc.global/)
Contestant computers boot a Linux image that contains all the interpreters/compilers and documentation for the supported languages, some editors/IDEs(vim, Jetbrains, Eclipse, etc) and a web browser.
Network access is limited to the judging system only.
My experience in Europe is that most universities can support this.
If you're doing a good job, you have to do that anyway, or at least have enough of a spidey sense for broken code to know when to investigate and add an extra test case.
Something like 30% of the time at $WORK, interviewers report the candidate as having solved the problem when a closer inspection reveals UB, memory corruption, and other bullshit. The test cases pass, and I think that's part of the problem. You can't tune out and avoid deeply understanding the submission.
I think the problem is that the grader has to run your code in their head. That's a whole different problem.
I'd say compared to just typing the code on a disconnected computer: Not worth it.
The grader has to grade N exams.
How do you think English, philosophy, or history tests get graded?
Or were you just assuming your experiences (large university, I'm guessing) are universal?
There are many, many colleges that have no grad students. Period. They are undergraduate colleges.
Expecting any college or university to just be able to scale up the amount of manpower required to grade their classes/exams by a large multiple is absurdly unrealistic. I assure you that most existing colleges do have plenty of resources to effectively evaluate individual students' outputs, if they don't have to assume that anything submitted digitally is written by LLMs.
That new requirement is a massive change to how they operate, and it's incredibly rude and short-sighted to be dismissive and contemptuous of institutions that have trouble adjusting to it.
That Computer Science is somehow a privileged subject where the exam grading must be automated is elitism of the highest degree.
So in some theoretical hiring example, if I have to choose between two candidates: one which is really good at keeping the code in the head and the other who is less good at that, but who is very skilled at using programming tools like advanced type systems or formal verification tools, then I'd prefer the latter.
Of course in practice, I would of course take a student that I knew had passed their paper test with their own knowledge over a student that likely ran their digital test through an LLM.
> or whatever is the tech of the day.
Rote memorisation has been the standard technique probably for way longer that anything else so I’m not sure that every innovation is inherently inferior to what we had before
No wonder some people are so easy to replace with an LLM.
this would also end up valorizing particular ways of thinking or developing, so those whose development methodology is more exploratory until you get a certain point where you get an overarching view of how everything should interact and then go into a quicker iterative mode would be penalized.
I guess it's ok it does this, some things after all must be valorized whatever is chosen, although I've known many people who are quite good using this way of working who would then be downgraded.
Interesting. This reminds me of COBOL and FORTRAN "code sheets" that were used back in the day...
We had 3 hours to make the test. The papers the teacher delivered had differences between them (I think it was four different versions of the test?). The Teacher didn’t stay in the class also, you had to deliver the test to them later in their office.
IME, that kind of nitpicky grading mostly happens when the exam was designed too easy and does not depend on the medium.
> Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example
Which you really need to do anyway because code that is broken due to a brainfart and code that is broken because of a fundamental misunderstanding should not receive the same grade.
Additionally expecting whoever is reading the paper to comprehend everyone's (likely very sloppy, in this era) handwriting is an exercise in frustration for the person who would be evaluating the papers.
Not that tests/exams can't be given on paper, ever (multiple choice still works), but for something where people are expected to provide multiple pages of coherently written essay output, I would struggle to do it by hand. And I'm old enough that we did do a lot purely on paper when I was in school.
There will likely be a period where those who went through high school with computers struggle with hand writing stuff, but the next generation will have done it all their lives.
If you did it before then, barring physical limitations which have occurred since then, you would struggle for a short while and then you would be fine. I also did coursework purely on paper, got out of practice, and then once my kids were in school got back into the habit of handwriting. I can even write cursive again. Cursive makes it much easier on the hand and wrist. There's a reason it was invented. :)
For righties.
I'm right-handed, but I had to learn left-handed cursive at school when I spent 5 months with a cast on my right hand.
I think you're missing the point. This is a physical limitation. People don't write as much as they used to. I used to be able to write for hours, taking notes, writing papers, etc... Now, I try to take notes during a meeting and my hand cramps. And as a bonus, my handwriting never was great, but now it's illegible for me.
Yes, we all "could" re-learn this skill, but how many people will? If you asked me to type a paper on a locked-down computer, I could easily. If you asked me to write a 2000 word paper by hand in an hour, I doubt it would happen.
If we expect students to take in-person tests on paper, then you should also do the rest of the classwork on paper. I am completely in the camp that we learn more when we write something. The physical act is part of the learning process. But you can't expect students to write an in-depth exam without having the practice of doing it often.
Are you using a fountain pen, writing cursive and do you take care to hold it properly? Those three things fixed all my cramps
Say what? Writing by hand is not only useful to write. It helps practice fine motor control. Something you need in many places in life, and most people have no practice of other than by writing.
Doing it now I really don't think I could deliver my intellectual best while worrying about if anyone can read my handwriting and whether I'm gonna cramp up by the end of the exam.
Pen and paper is just not a very good way to produce text.
Pen and paper are literally designed to produce text.
Can you say more? It's worked well for 4,500 years. Probably the most important invention ever.
So it’s like saying that clay tablets worked just fine for 1000+ years why would we switch to papyrus? Or who needs character for vowels? Or spaces between words, it’s not like anyone reads texts silently?
Then we got paper and pens and that was a pretty decent interim for a short period. Then about 100 years ago we got typing. Then about 20 years ago we reached a world where almost everyone is better at typing than they are at writing.
Obviously it's still important that people can write by hand, but expecting people to do it for more than a few hundred words at a time is idiotic. Would you like your clothes to be hand sewn too? That also worked for thousands of years (much longer than writing) but we stopped doing it for a very good reason.
Multiple choice can only tell if you reached a final answer (or guessed it). It cannot tell anything about how you reached that conclusion.
The mid-term and final exams were not multiple choice (at least not entirely).
You can't give someone a multiple choice test and then give a written test to only those who passed it, except at the granularity of an entire course.
(Where do you even get ribbons these days)
A college instructor turns to typewriters to curb AI-written work and teach life lessons - https://apnews.com/article/typewriter-ai-cheating-chatgpt-co...
I do think it's necessary. And I felt unsure at first of how extremely strong I feel about this -- I think everybody should be able to write cursive, and even doctors should be able to write legibly, which ALL of them could learn in one single day, an afternoon, if they had to -- but then I did a simple search for "the benefits of writing by hand studies" and now I'm even more radical.
It's like PE or brushing your teeth. Nobody initially wants it, so we, knowing better, force them.
As a specific example, all of my friends in Germany - whether born there, in Eastern Europe, or whatever - use print letters, not cursive.
In high school the most difficult part of the SAT was the honor pledge that you didn't cheat that you had to write in cursive. Nobody writes cursive.
Cursive vs printing (I'm guessing that's what you mean by "standard writing") is exactly the same, provided you can actually write in cursive. If you weren't taught in school, then sucks to be you, I guess? Modern pedagogy has a lot to answer for :(
The junk that used to be taught in US schools (a type of Palmer cursive) it not fun to read or write.
BUT, the above analysis only really applies to people who want to write, and want to write a lot.
(Apologies for repeating my reply from above here as well.)
As someone who has hated both reading and writing cursive since middle school, I'm curious what is significant about cursive specifically?
... And there are jobs that use those skills.
Correlation between handwriting, drawing skills and dental skills of junior dental students - https://pubmed.ncbi.nlm.nih.gov/22269191/
My dentist, while teaching dentistry commented that if the student did not learn cursive in school, it takes them another 3-4 months of practice in order to acquire the fine motor control for holding dental instruments.
What is the downside of learning cursive?
* Reading and responding to other people's emotions. * Emotional regulation. * Generalized sequencing. * Musical ability. * Counting. * Following patterns.
While you may see it as a waste, you may very well have benefited in significant ways from learning it at a young age.
So I would totally put it on the same level, theoretically. That is, you can write block letters with just as much skill and care as you can cursive, I think. Just like you can write cursive by just doing squiggly lines (as doctors write prescriptions).
Learning cursive (and other skills) in school is about trying to make sure opportunities for professions are not excluded.
While cursive is not a foundational skill (compare math and reading), it is a way to train fine motor control.
And it's not only the dental surgeons, but also dental hygienists... and doctors and surgeons (anyone who uses a scalpel). One could also look at various craftsmen such as jewelers and watchmakers.
Learning cursive is a way to make sure that we don't exclude people at an early age from professions that they might want to go into.
I few interesting bits — it does involve cursive, but it's Arabic and it's graded on a rubric that includes things like "Presenting the beauty aspects of Arabic writing'. Also, given a sample of 71 students and a p<0.001 means the correlation coefficient only needs to be around 0.40 which means handwriting and drawing may only explain about 16% of the variance of these dental skills. That's not nothing, but given the subjective nature of the test and the confounders (does this handwriting sample really measure motor skills or maybe it measures care and attention to detail, or conscientiousness), I'd be a little wary of using this to argue for education policy.
Still, glad you posted it and glad I read it. It interesting.
The moment I have to write stuff down my focus is gone and I might as well be taking a nap.
And having to read my own handwriting assures I’ll never look at that page, again.
Different strokes
I detest writing and have terrible handwriting but have seen first hand that typing or just listening is not as effective. In grad school I sucked it up and just typed up my handwritten notes so they were searchable when I actually needed them to be.
But writing by hand and just reading them over was usually enough.
I mean, I have no way of knowing if it's the former or the latter. But I've been noticing recently when people treat their traits as changeable and when they treat them as core to their being. I don't really have any faith that, in most cases, one can differentiate the two as much as one thinks one can.
For others, hearing stuff (and saying stuff) out loud is more useful. I had a friend who'd make nonsense songs of stuff to learn: just doggerel, but by singing it to himself when revising, he had a massive uptick in retention. He was so happy when he worked that out.
I imagine there might be other modes that work for other people too?
That's because "average" can mean either median, mean or mode depending on context.
Sorry! I'll see myself out.
The big advantage of writing in cursive is speed and less muscle fatigue. Writing in cursive requires far less lifts of the pen and far less tiny movements... a reasonable cursive script (Spencerian, but with a little less flourish) is quite easy to write legibly and with speed, with just a little practice.
The junk that used to be taught in US schools (a type of Palmer cursive) it not fun to read or write.
BUT, the above analysis only really applies to people who want to write, and want to write a lot.
What's really important is handwriting itself. Block letters are honestly fine.
Writing in a good cursive (like the Spencerian script) is a bit faster and much easier on your hands, as you don't have to lift the pen as often. But that matters only when you need to write pages and pages of text.
My native language is Russian, and in Russian schools cursive was mandatory. Writing in block letters was seen as a sign of illiteracy.
I started learning English as a foreign language, and we didn't bother with cursive. So I kept writing in block letters for quite a while. I then started learning German, and our teacher taught us German cursive handwriting. I'm now using it for English as well :)
How interesting. I've always thought the cyrillic block letters look beautiful, whereas the cursive variant not so much. I learnt some russian and I've never bothered writing cursive because of this perception. Odd to realise now that the average russian seeing my writing would be dismissive instead of impressed!
> Odd to realise now that the average russian seeing my writing would be dismissive instead of impressed!
There's a HUGE difference between learning Russian as a native and as a foreign language. Especially if you can write! Russian spelling rules are quite opaque.
People will absolutely be impressed by your writing.
i was taught cursive in 2nd grade. and my handwriting is gobsmackingly horrible. coming back to stuff I've written after I've forgotten the context, makes it impossible for me to understand what I've written.
and it's not for lack of trying. I spent almost every summer till 10th grade, practicing writing 30 pages a day. and still it gets reset to my horrible hand writing in weeks after school start. at this point, i just consider myself hand writing challenged.
i cannot tell you how much happy i am that, computers have made handwritten exams obsolete.
These days I just disclaim to people when I hand them anything handwritten that I'm very aware my writing is terrible, and I will not be offended at all if they have to ask what it says.
That obviously isn't to say that I don't think people should learn these subjects, nor that we should avoid presenting them at all to young minds. It's just that, as someone who failed math all through grade school and now does pure math research as an adult, I don't think "forcing them" in the sense of introducing yet another high stakes and high pressure set of evaluations to all the others is really the enlightened path here
But just having exposure to that material planted it in my mind and, in many cases, I revisited those topics later as an adult and learned them properly.
To be clear, I am not advocating for doing away with testing or schooling altogether. I am only pushing back on the idea that a 16 year old who doesn't care about cursive or calculus or Chaucer or whatever today won't ever care about those topics unless we mandate he pass an exam or course on it.
More broadly, my point is that there is a cost to "forcing them" as the grandparent advocates with regard to learning cursive. The moment you "force them" you also guarantee that some people won't graduate because of that extra requirement. There are other ways to enrich people's lives without force feeding, e.g., mention it and devote a question on an exam to it.
But I think you misinterpreted my comment -- I am not advocating for doing away with schooling or testing understanding in school in general, which is entirely my fault as my post was a bit unclear. But I agree with you that one should learn to read and write in school!
OP used the phrasing "force them", which I read as a stronger statement than "it'll be on the exam", and more "there will be an exam or entire course on this topic that must be passed or else". My point, more or less, was that there are many other more fundamental topics we already do this with other than cursive, and that adding another has a steep cost with uncertain payoff that is heavily dependent on the implementation.
I wish I could have a just spent "an afternoon" to magically make either my printing or cursive better, but it basically stalled out early on and never improved despite years of practice.
Much easier is to just have someone with authority in the room to hand out consequences to those who break the rules.
During an exam, it's an instant zero mark, on top of the after-school detention (the punishments escalate to suspension and expulsion for repeat offenses).
I (as part of a small team) run the IT side of exams for a UK University and as well as the many exams intended to be taken entirely with computers, we also deliberately have an exam that's always marked as "Available to take" regardless of where on campus you are and what machine you're using, that exam just launches a stripped down Microsoft Word, with no way to start other software or access your own data.
So instead of reading the instruction book and writing either in the book or on separate provided paper, you read the book and type your answers in Word, when you're done either your work is printed or these days often it's automatically sent to the markers electronically.
There's a spectrum of people using this, going from the profoundly blind who couldn't have attempted an actual hand written exam through to people who have dyslexia or similar problems and would be able to write but it might be very difficult to mark. It also becomes a "last ditch fallback" for a number of scenarios where plans went wrong or something was forgotten and so that's why it's always available - we do run exams specifically planned to be in Word, but those have distinct IDs so that you know you're taking the exam "HIST1234/C4 History of Clowns and Clowning. Essay on prepared topic" or whatever, as well as "Multiple choice" style exams, and a large number of exams which involve using computer tooling, e.g. R, Stats packages, programming.
It's fascinating to see all the people here who are arguing that it is impossible to do what we did all the time in my step-daughter's childhood, my childhood, my parents' childhoods, my grandparents' childhoods, etc.
What is practical changes over time. Insistence that "We always did X => That's enough" makes no sense in a universe where the only constant is change itself.
In 1926 it's not really practical to even have a global voice conversation, even world leaders must travel in person to properly negotiate or send trusted negotiators. By 1976 you can just pick up a phone and call from say Birmingham in England to Birmingham in Alabama, because why not. By 2026 you don't even think about which Birmingham the other party is in, they're both on the Network so you can talk to them, who cares?
Moreover, for math or math-heavy courses (assuming most people with degrees here have STEM degrees, and many with at least some math) I cannot imagine how to comfortably write math in a word processor. Or use latex and not spend half the time troubleshooting latex, esp without internet access. So for some kind of courses at least, imo pen and paper for a timed in-person exam is the only way.
Otherwise def doable, but knowing how some universities function, I think the main problem would be getting the agreement and initiative to set such a computer room up. Getting some kind of consensus between professors that this is how (some) exams should be held and including it in the Holy Curriculum. Getting bureaucrats understand what it is about eg why you need these wired connections when the uni has a campus-wide wifi. Getting IT security agree with using old computers with lubuntu instead of their bloated enterprise windows "secure" OS. And if they are not connected to the internet how will they get security updates? How do we conform to whatever IT security rules are in place?
Writing on paper is much simpler, everybody can understand it and has been standard for decades at least. It can start tomorrow and be used in the interim while waiting approval for such a computer setup.
A word processor allows you to edit, which is a major part of the writing process.
Forcing learners to plan everything “perfectly” before they write is a big ask. And you’re probably not teaching that skill.
I remember how much my technical writing skills improved once I started writing in a computer and editing. It was a huge difference.
Exams that only require paper, tables, and chairs can be done anywhere, and require minimal setup/set down, transport, or power, and no tech support.
Edit: I think we’re actually all agreeing. That a typed exam as an option is perfectly cromulent. I’m just saying the idea that it must be written is silly.
It was mandatory in an earlier age. Now it has become optional, but it could become mandatory again.
Presumably someone who is unable would get an accommodation of some kind.
Computer Science classes were all on paper for exams, and low level ones did the old "here's a Javadoc, write some code with a pencil."
The only online exams I had were for 100 level electives.
Without certifying someone’s abilities, the degree doesn’t mean much.
Those exams probably took the median student 30 hours and had a median score of 50% including partial credit.
Sad to think current students will never get that experience, because a chatbot could write something good enough to get a 50% in 10 minutes, for many of those classes.
Well, either that will happen, or the pace of advancement will increase faster than humans can adapt. That’s the singularity, right? I don’t think we’re that close to it.
It’s an old (and outdated IMO) tradition at some Ivy Leagues, as the article notes.
It may have worked in the era when students had a little more fear of repercussions and a little more sense that cheating on a test would only cause them problems when courses got harder later.
Now it seems there is little interest in dealing with cheating, as evidenced by how hard it was for this professor to even get attention to the matter within his department. Students also don’t believe cheating will cause them future harm because they assume they can cheat everything up through graduation the same way.
When the rug gets pulled and they have to demonstrate their knowledge in person without ChatGPT, the cheaters collapse. I fear that we’re delaying this reckoning so deep into academic careers now that by the time these students encounter the point where they can’t cheat their way to completion of a course they’re in for a world of hurt, if they continue at all. We really should be coming down hard on cheating earlier and more often.
This is the crux of it. The incentives don't support cracking down on AI cheating much. At least in the short term, for individual department administrators. Overall it will likely hurt the universities collectively to hand out degrees like candy this way, but grade inflation was already a problem and universities treat students like paying customers and they also want good statistics, fend off possible discrimination accusations etc.
So it's really not just about AI, the AI is simply exposing this underlying misaligned incentive. Professors complain about it all the time. It was similar during covid and cheating in online exams and that was solidly before ChatGPT.
My favorite: professor known for being VERY strict tells class repeatedly in weeks leading up to exam, "You will have exactly two hours to finish your exam, and no more. I will not accept exam booklets turned in even one minute past the two hour mark: they will score a zero. Be warned."
Exam day comes, and all but one student are in their seats with the blue exam booklet in front of them when the professor says "Begin." He arches an eyebrow at the empty desk, but says nothing.
Fifteen minutes later, the tardy student rushes in. "Family emergency," he says, "sorry." The professor tells him "You have one hour forty-five minutes." Student says nothing, but opens his book and starts writing furiously.
The two hour mark comes up, and everyone except the tardy student turns in their booklets and leaves the room. The professor reminds him "Time's up," but he just keeps writing in the booklet.
At the 2:12 mark, one hour fifty-seven minutes after he ran into the exam room, the student closes his blue book and walks up to the desk where the other books are stacked up in a messy pile. The professor is reclining in his chair with his feet up on the desk. "Nope," he says, "you're late. I won't accept your exam booklet, and you're going to get a zero on the final exam."
"That's not fair," said the student, "I had to drive my mother to the hospital. I shouldn't be punished for that. I took no more time than anyone else."
"Nope, you knew the rules," says the professor.
"Don't you know who I am?" says the student, raising his voice.
"Nope, don't know, don't care," says the professor.
"Good!" says the student. He slides his exam book into the middle of the messy stack, straightens it up neatly, and before the professor has gotten his feet off the desk, walks briskly out into the hallway.
And in the workplace as well.
And so do their parents.
With alarmingly good reason, at least some of the time.
—- Disclosure: I run a small start up which offers teachers a platform to create and conduct digital exams and interfaces with the safe exam browser precisely because cheating is the number one complaint teachers I have spoken to have when it comes to digital exams.
In person writing, etc. But also for anything take home students have to verbally discuss their work in some fashion. Seems to mostly work. Students still us ChatGPT as a search engine, which seems fine.
(Source: married to a professor. And my son is in high school)
Wouldn't a one-on-one interview be a better way to establish a broad evaluation of competency.
1) Possibly, depending on the material, but most classes aren't looking for a "broad evaluation of competency", they're looking for the specific material taught. It'd make more sense as a graduation requirement: the equivalent of a dissertation but for undergraduate work.
2) Even in small classes (e.g. 20-30 students) that wouldn't scale, let alone the massive courses earlier in a curriculum or that are shared by many degrees (e.g. hundreds of students).
(That said, any kind of subjective assessment has its own pile of hazards compared to objective assessments.)
[1] https://www.evenrealities.com/en-GB/blogs/buyers-guide/ai-mo...
So are jobs.
maybe we'll see specialty chatbots that give the equivalent of oral examinations and/or are willing to provide reference material but not make suggestions, connections, solutions or generate prose on behalf of the user?
I guess it has been 14 years since I graduated but I didnt expect it to change that much.
I went to a liberal arts college that prided itself on small class sizes and the honor code, and where professors typically had a good enough read on every student that they could tell when work wasn't your own or you'd collaborated with classmates. Our in person exams were not proctored except by your peers, the professors would sit outside the room. My understanding is that this sort of arrangement has been common at liberal arts colleges and similar universities for a long time - I know UVA has long prided itself on unproctored exams.
There's been a massive culture shift in the US where cheating is more and more accepted, beyond just the AI stuff, over the last few decades, and these sorts of peer based honor codes are no longer sustainable.
Back in my day we’d sit down ‘t muddy corpse pit having questions screamed at us by ‘t dragon who were also ‘t drill instructor. Get one wrong, and that was that, fricassé.
And students were ahead of the curve too, with chits, hidden iPads, phones and the classic psst-psst in the middle of exams.
I truly don't understand how people can sign up for a degree and then have to cheat? Must feel torturous to endure a class you're not even interested in.
When I was a student (and a student TA), what I heard or saw was that students were in CS for money, or their parents forced them to study it. Both of these things created some sort of extrinsic motivation that leads to cheating. In some cases (eg. in my high school) I heard parents would threaten to beat their children if they did not do well in their classes. So maybe that pressure continues in college. And for some, they just want an easy 6 figure job and are willing to take shortcuts. Students I know (some honest, some not) have mentioned they cheat on CS interviews or lie on their resume.
Additionally, I heard that multitides of parents would threaten to withhold tuition if their child failed a class. since the university is not well off, they acquiesce and make classes easier for students who aren't interested.
[1] https://www.brunel.ac.uk/news-and-events/news/articles/There...
LLMs has become good enough so whatever homework task I give, it can be solved easily by any LLM (and, in fact, this becomes unfair to those who are not able to pay for good model). I was asking to add comments & interpretation of results, this slightly helped, but LLMs are increasingly good in all this as well.
So, quite seriously, I am considering some on paper tests & quizes, because what else can be done?
Making people aware about importance of writing code is indeed a good hint to convince, at least part of the people to do this. Another thing is: they will leave university one day, they will search for employment and the employer might be much more hostile towards cheating during job interview and can easily make cheating with AI impossible... Question is, will it still matter?
> (I think older generations would do well to recognize this fact.)
That is so, so much better than the default mode of blaming kids for getting addicted to the systems their older generations developed.
https://ploum.net/2026-01-19-exam-with-chatbots.html is another interesting perspective
As a result, I've been able to challenge students to solve interactive software security challenges on the midterm and final with automatic grading - something that would have been impossible with pen and paper.
Scalability is really the major challenge. We're rolling out more CBTF rooms and rolling out access to other departments due to demand, but it's definitely more resource-intensive than pen-and-paper. One possibility is to treat CBTFs as computer labs when not actively administering exams (or maybe even vice-versa), something we're looking into doing.
If you find that onerous, I guess a paper exam would probably also feel pretty depressing to you.
... did LANs (Ethernet, with wifi disabled) stop being a thing?
Yes, said machine could have both the LAN connectivity and WAN access, and we could set up the whole exam website on it, but we would lose out on the flexibility to let profs choose the platform that works best for them.
I've got kind of messed up hands, but not enough that you'd look at me and say "that guy should get special treatment". From the comments here, there are others like me who similarly hated hand-written tests due to the physical pain of hand writing.
People perfectly smart, but with cryptic handwriting will have much worse results for the same exam on pen and paper vs with a keyboard.
Some profs also started requiring special software for doing the exams, which works fine but is pretty annoying to use since it requires Windows, but that application basically hijacks the OS, making it impossible to navigate to any different webpage/ application.
In 2024, we had some coding exams, and AI was explicitly NOT forbidden, which lead to them making the exam more difficult (because with AI, you can solve it so much more quickly, so gotta make it harder), which was really annoying, because it made it practically impossible to solve it yourself because of the time crunch. I hope they realized how wrong that was.
There should be at least one class, probably more, that work exactly like that.
That’s where the industry is moving. Yes, get the fundamentals too, but don’t omit teaching what graduates will actually be doing out of some misplaced sense of purity.
(So I fully agree with your point, but I think your example is not strong enough)
But the class I was talking about in my earlier comment was specifically on the fundamentals of programming (how to write a for loop, how to write and call functions... all that sorta stuff). If you need AI to complete this, why even get the degree. Maybe in 10-20 years we'll think differently about this, but as it stands right now, I think someone holding a degree in computer science should absolutely know the fundamentals of computer programming.
It's not easy, but I think that method is just as good (or bad) now as it was then.
Now, I require of students to submit written assignments done by hand. That way, I can at least be certain that there's some learning involved, even if they resort to AI to produce the relevant written part, because evidence points out that writing by hand reinforces learning.
I read that other professors resorted to requiring manual typewriters, which I also hate with a passion.
That is, AI is negating decades of enablement achieved by technology.
Honestly, between these circumstances and the fact that we are "enjoying" (?) brave new prices for RAM and SSDs, I'm finding AI increasingly harder to like.
Dad indulged me with many peripherals, including an Epson dot-matrix tractor-feed printer that printed 132cps! Among its features were configurable fonts and point-size. I discovered that this device could be set up for extremely small fonts that remained legible.
So, for Spanish and other classes, I decided to go into the "cheat sheet" production business. I worked out how to print a sheet full of little 2x2" sheets that contained Spanish vocab, or the study answers for our next quiz or whatever, and I used them myself, and then I began to sell them!!
And the most ironic thing happened, when I discovered that I never needed the cheat sheets: having created them by typing them in, repeatedly, correcting errors and checking against our textbooks, I had internalized, memorized and learned all the answers as any ethical student would!
I got caught, once or twice, and disciplined for distributing those cheat sheets, and I definitely did not continue cheating in classes, because I'm not that kind of guy, but the process was a revelation to me, and ever since the 1980s I have relied on hand-written notes to reinforce my memory of those words and facts that flow from ears, to mind, to hand muscles, to pen, and onto ye olde fashioned paper.
Now can we see a reason that "writing 100 times on the chalkboard" may have been an edifying exercise, beyond its punishment value?
That summarises pretty well the whole situation. AI is cool and all, but not worth it considering all the disadvantages
If you want to test for memorization, you need a live test to do so.
If you want to test for comprehension and understanding, require sessions with a tutoring AI and grade the level of comprehension exhibited.
And then please, please test for prompting with challenging assignments that would usually be beyond a student's skills and make sure they know how to drive the machines to do valued work.
I totally agree that there is utility in memorization and comprehension. I also know that by the time students graduate, AI will be the job and if they can't pair successfully with agentic workflows they will not be much use in the work force for many roles.
Does this mean you will requre all students to pay for a subscription to an llm provider?
Sounds like sarcasm, but the wheels appear to already be in motion on this.
There are few things more unenjoyable than parroting things I know to AI. I hope this is not the future we create for the next generation
What fresh hell is this?
they can use the AI all the want to help their workflow to develop teh content, but the better engage with the material enough that they can field some questions at the end.
the main downsides are:
- takes a long time to administer
- the Q&A section leads to unstandardized grading, but i think that's fine. just away from scores on precise 0-100 scale and just score in a couple coarse buckets (superb, acceptable, fail, etc)
We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.
This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.
My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.
In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.
It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.
Unrelated, but once you get to a more mature level, say grad school and above, I can say this is not always the case! Just like it's easier sometimes to Roll Your Own Damn X in programming, so too are some expositions of proof so dense (necessarily or unnecessarily so) that it is a less taxing affair to simply figure it out yourself, or at least figure out 90% of it, consulting a suggestive sentence or two in the proof in order to get at that last 10%.
Maybe this observation of mine is not so unrelated after all. I don't regret many of the times that I've thrown up my hands at the rococo explanations or solutions given to me by LLMs and simply did my own work. The Socratic method with AI is sometimes more effort than it's worth.
It's hard to verify that you understand someone else's work at 100%, but it's reasonable for an expert to tell once they've built it themselves, right?
lol, so true.
I think the main issue here is that LLMs don't have the ability to produce the terminal "I don't know".
There are dozens for every one who didn't get the opportunity to attend an Ivy. The penalty for cheating should be automatic expulsion. I'll note as a hiring manager the college degree won't be worth shit unless the school can show their students have any integrity.
I would say that tech largely rewards the behaviours people in academia call cheating.
What? The concept of cheating exists in marriages, sports, relationships, business agreements, accounting, and nearly every facet of human life.
cheating in schools is nothing like that. there is no direct harm to anyone, financial or emotional. the only real thing is if enough students do it thats bad for the schools reputation and others will treat degrees coming from there as more suspicious.
thats a very abstract and indirect harm, and it depends on employers caring about what school you came from more than a basic "reputable or total scam". to be fair thats pretty common in some industries (it shouldnt be) but i dont think the average student will get worse job offers because some of their classmates cheated.
When going to a doctor, you wouldn't want them to have cheated for their exams during training. You expect them to be well equipped to practice medicine and to ensure you get correct diagnoses and care. When hiring a programmer or software engineer, you should expect them to be able to provide you a proper, well architected application given the requirements.
Love the sinner, hate the sin. If you let yourself start blaming students, you open a pretty corrosive path of moral judgement. Students are clever, and they're making choices informed by what they see in the world. Don't blame them for being unwilling to fail a class that cost a down payment on a house. Don't blame them for seeing powerful people get away with cheating and trying to do the same.
Kallus is right: qualitative assessment is an adversarial problem. Build the assessment correctly, and all of those previous points become moot. By clearly establishing and enforcing the rules you make it more fair to boot.
Universities need to stay ahead of how people actually work. Framing the use of AI as 'cheating' gives a university a reputation that it isn't keeping up with industry, which has a wildly negative impact on admissions. You can't expel people for 'working like they do in the real world'.
- making a change to your own mind and/or body (studying math, lifting weights)
- making a change to the world (optimizing ad placement, operating a forklift truck)
Do you think gyms should allow people to lift heavy weights with forklift trucks, so that they can work like they do in the real world?
So, to continue the analogy, a degree is the equivalent of a forklift truck license, and people do want to drive their forklift truck to the gym. Because they're paying to be able to do that.
There's a pretty good reason for that - the base assumption is that training in fundamentals, methods and ways of thinking is something you won't get anywhere else.
You totally can, though. In the real world if I can't remember something, I might look it up in a textbook. Closed-book tests have historically been a totally accepted practice, though, and getting caught bringing notecards with textbook info secretly into a closed-book test would absolutely bring about disciplinary action.
But in real life I am allowed to look at any material available while solving a problem yet school exams dont allow it.
Historically, the penalty for cheating at Dartmouth was a 9-month suspension for a first offense (no matter how small, in theory), and permanent "separation from the college" for a second offense. Back in the 90s, there were multiple incidents where this wasn't properly applied to the CS department because the academic committees in charge of punishment were bad at evaluating plagiarism of source code.
Dartmouth certainly should blame the students. Their policy is historically clear on that point. It is the responsibility of professors to use their syllabus to clearly define "cheating" for a particular course, and it is the responsibility of the students not to cheat. The only case where this should be even slightly complicated is if the professor hasn't been sufficiently about what constitutes cheating (and there was one major historic scandal related to this).
I certainly agree that any degree which allows rampant cheating will quickly become a joke to employers.
Employers past a certain scale are not really using degrees as anything other than a low pass filter, and the people judging how qualified hires were in practice are not the ones deciding the minimum requirements, so there's no avenue for feedback on that point.
Our whole education system is setup as a skinner box for grades. People finding tricks for better grades are massively rewarded. Study methods are optimised for recall within a month. And educators are accountable only to the measured outcomes.
Very little of the designed system of education is aimed at teaching. Most of it is aimed at measuring and certifying. Treating students as adversarial grade maximisers will likely teach a lot of them they are expected to grade maximise adversarially.
Fortunately, there are a few alternatives, schools without grades, that don't focus on short-term recall but long-term understanding, intrinsic motivation, autonomy, and self-actualization, like the Ecole d'Humanité in the Swiss Alps: https://ecole.ch/
When further education wasn’t mandatory, things like the honor code made sense.
Our whole everything is setup as a skinner box for mulah. People finding tricks for more mulah are massively rewarded. Efforts are optimized for ROI within a fraction of a second. And management are accountable only to the measured outcomes.
Seeing as the article was about Brown and their PLME program is famous for med school acceptance, then I think we should expect that the culture there is more about grades than a generic undergrad university's culture. Which, yeah, is also very grade focused.
As an alumnus of Dartmouth College's CS program, I am sad to hear that my alma mater has sunken so low. Look, I know that the Committee on Students was historically bad at handling CS plagiarism cases back in the 90s (compared to ones in the humanities). But Dartmouth's historic solution to this sort of pernicious "optimization" was to reduce the expected value of cheating by imposing extreme negative consequences on anyone they caught, with a 3-term suspension and a permanent transcript notification for a first offense.
Allowing widespread cheating and LLM regurgitation will destroy a school's reputation with graduate schools and employers, and rightfully so.
I mean even before AI this should get tested, as people could just copy code from SO or other websites, from books, or from their classmates.
I've done a few projects where the other students didn't really do much (in hindsight this was probably because I was excited and productive instead of helping and delegating work, which was selfish of me), but they were caught out in 1:1 interviews.
Because you didnt know which type of integral you will.get, we had to learn to solve all of them. We made.hundreds of exercises with a small group of close friends (the teacher gave us a book that's infamous in Mexico from an author Schaumm).
Is anyone surprised that the academics are essentially being left on their own about is? University management has become increasingly disconnected from the academics doing the work on the ground at least during the last 20 years that I've been in academia. Covid was really an eye-opener in this respect, at our university academics were told with very short notice to make all lectures remote. There was essentially zero support from central management, there was not even budget to buy headsets or cameras and academics were expected to use their research funds.
At the end of the pandemic academics were thanked with two years of 0 and 1% salary increase, due to a tough economic situation. During the same time the president received a 8% raise. The tough economic situation was entirely due to mismanagement, i.e. the university had invested pension funds into fixed interest investments which did not keep up with pension growth (during a time when stock markets were going from strength to strength).
Let us know when they institute stack ranking and 10% mandatory cuts every spring. Then academia will have been fully corporatized!
I personally disagree with that very hard. Deontology begins at home.
Yes
Has that changed since I was in school? After I got my degree, not a single person or entity in the workforce ever asked for my grades. They just wanted to see the degree.
[Edit: typos]
https://en.wikipedia.org/wiki/Doping_at_the_Tour_de_France#D...
Note the quantity that actually got caught and with enough evidence. Lance won it 6 times without getting caught.
Looking at it systematically there's no way all of the top finishers were not taking drugs (how else could they compete with the world's best who were? The advantage isn't small.) And it had clearly been going on for many years before Armstrong entered the event for the first time.
I really don't care for Armstrong's yellow banded hypocrisy but blaming him for the "cheat or don't bother competing" reality lets rather a lot of people off the hook with a convenient scapegoat.
But I'm sure that's all in the past and it's not like that now. Just as was said when Armstrong won 6 times while correctly stating he was the most tested athlete on the planet.
Wrapping it up and tying all that to Armstrong, as has been done, stinks. He was clearly a bit player in that extensive fraud. Six titles with no meaningful positive drug test as the most tested athlete on the planet.
https://quotefancy.com/media/wallpaper/3840x2160/2503086-Lan...
> Lance won it 6 times without getting caught.
Yes, he was doping for years before being caught. He found novel ways how to do it.
> reality lets rather a lot of people off the hook with a convenient scapegoat.
No it does not. It just guves credit to where it is due. But we also have track record of Lance scapegoating others.
Bernard Thévenet won in 1975 and has said he was using steroids. Armstrong was 4 years old at the time. We don't know about all the drug cheating at the TdF, just what we do know is damning in ways we can't pin on the vile Armstrong. Do continue to loathe him as much as you like. He's earned that.
Ulrich, Riis, Fignon, Zoetelmelk, we haven't caught them all nor tried 1% as hard as those who got Armstrong did.
Look through the list and try and claim with a straight face that Armstrong was first, led the charge or that the Tour de France actually wanted to catch him or any other winner at all.
I don't care at all about Armstrong. The sport is clean now despite an ugly past. Just as it was when Armstrong won all those tours. Can I interest you in this bridge at a good price?
Who gets off scot free with the "It was all Armstrong" line.
If Armstrong stayed retired he probably would have gotten away with it, kept lying forcefully and be a secular saint by now. The TdF didn't catch him.
https://en.wikipedia.org/wiki/Doping_at_the_Tour_de_France#S...
Doping existed prior Armstrong, Armstrong revolutionized it and led worsening of the doping situation. His teammates were bullied by him personally to dope. That includes testimonies of people who left for that reason.
> or that the Tour de France actually wanted to catch him or any other winner at all.
Yes, plenty of them actually wanted to and Americans were acting butthurt. There were regular accusations of French just being jealous, like it is an international offense to suspect and American sportsman could cheat.
Look down that list of winners and the list of top finishers for two decades prior to Armstrong and the amount of doping among them that we actually found about and seriously try to make the case that Armstrong is the reason for "cheat or don't bother showing up."
You always have a choice. The right move, in this case, is to raise a stink to administration, donors and politicians. Hell, use AI to do it. Schools refusing to punish teaching is a problem that’s leaking into business and politics.
Earned credentials are a marker that you once demonstrated some sufficient combination of both.
The credential is a prerequisite but skills are a differentiator. Problem is, not all skills are equal.
Amazon isn’t going to ask you about your opinions on The Illiad, they are going to check if you can write an efficient algorithm to rob houses or merge sorted linked lists.
1. What is this education intended to optimise for?
2. What are the various participants (including students) optimising for?
3. What aligns 1 and 2?
But why else would you have a bunch of students join a class and cheat? Something like 22 out of the 27 perfect midterm students didn’t show up to the final? Obviously these people are not in it for the joy of learning.
So the answers to your questions are 1) learning 2) job-worthiness 3) measurements of 1 which accurately and plausibly reflect 2.
If a course does not plausibly reflect job readiness, or students’ grades do not reflect competence in the subject then the course must be fixed.
Instead they should optimize for keeping a job.
Since you mention Amazon, I work there as a senior software engineer (almost 10 years of tenure) and the quality of interns has massively degraded since LLMs are available. The result is that where the majority of interns used to receive return offers, now the vast majority doesn't.
Similarly, a lot of new grads get kicked out quickly, before the end of the trial period, because they suck. A lot more than in the past.
Also, your own example is an application of game theory; you've basically stated a 'prisoner's dilemma' problem. You state that in a high trust society, most people will choose cooperation, while in a low trust society, most people will defect.
"We economists understand reality as a set of people responding to optimization problems with restrictions."
It depends per country. Here we use a grading system that isn't relative to how other students perform. You basically get a percentage score on how many questions you got right. Perfect score is 100. 55 and up is pass, below that is fail.
But still, teachers can make exams easier or harder depending on last year's scores, so there's that. In short, nobody will hone your mind for you, it's up to you to want to.
Oxidation in steel structures is informative of where these things go and how to address them.
Perhaps this class isn't too hard for Stanford students, but I have to wonder about cheating when the averages on nearly every single assignment is that high. One clue was office-hours: Whenever I dropped by office hours, there was always a line (didn't matter the class). I quickly learned that TAs would often drop a hint that would make particularly hard problems easy to complete. It provided an unfair advantage to students who could attend office hours.
It also reminds me of the huge scam of cheating on the USMLE amongst Nepali medical students: https://www.medpagetoday.com/special-reports/features/113627
I've met many wonderful international medical graduates. Many have shockingly high USMLE scores. It is true that there's no time limit on their preparation (U.S. students have under 2 years to prepare during medical school, international students may wait years after graduation before taking the exam). Before that scandal, I never would've thought someone could cheat on the USMLE. Prometric test centers are crazy locked-down. But that's not how they did it. They did the long-game. Prior test takers would remember a handful of questions and just add them to a secret database. After many years, that database contained nearly every question on the exam. Test takers would work extremely hard - memorizing every single question. The reward of a U.S. residency is life-changing, I get why cheating was(is?) so rampant.
I'm not sure that exists though. There are challenges where that is indeed the entire "field", e.g. Pi digits competition, but otherwise for most if not all fields, it is about having enough knowledge to answer questions that have no yet encountered. They might be variations of questions you know but they are not verbatim the same.
Ideally learning about a field should even lead you to be able to consider, warranting you would have the resources to do so (e.g. time, experimental setup, etc), actual unanswered questions from the field.
So no, IMHO in most cases memorization is not learning.
What an oxymoron. I agree with the others here that AI isn't the problem.
Don't get me wrong - I'm sure people cheated even in my day. But this is the spirit - they're trying to give problems as challenging as they would for homework - and a lot of those classes have very challenging homework problems.
EDIT: Rate limited so: "smarter" here means how well the student will perform in a career in their chosen field. Fast + accurate is the ideal.
Lots of reasons a slow student can be just as smart or smarter than a fast one.
And then consider whether the point of the class is to test smarts, or something else.
I’d expect that’s not the intent of most undergraduate degrees.
Memorization is already hacking the rules of the game that's supposed to be gauging understanding. An ideal test is resistant to rote memorization as well as outright cheating.
Assuming this is true (others have already addressed a few of the many reasons it might not be), wouldn't that imply that practically all existing tests are already flawed? If you want to grade on time, every exam should be graded with a formula taking both the time and the correctness into account. A binary "fast enough" vs "not fast enough" is about as useful as a pass/fail class grade.
The exams that felt like the fairest reflections of my own knowledge were proctored in-person, closed book, and time unlimited. Of course, being time unlimited works better for quantitative/engineering exams. I haven't put much thought into more qualitative/liberal arts type exams.
It always struck me as an ingenious way to get a feel for a new group of students, since it's quick to administer and equally quick to grade, while revealing quite a lot of information.
Certainly we hold speed to some regard, since a lot of academic accommodations involve granting extra time. If we aren't testing on speed, then surely we should give that to anyone that asks?
Dubious assertion.
> "smarter" here means how well the student will perform in a career in their chosen field. Fast + accurate is the ideal.
Still dubious.
Also, I don't know where you work, but in most of my jobs, career growth is not limited by speed with which you do the work. It's one factor among, say, 10. Most of the people who got ahead were not the fastest.
They're not trying to gauge who is the fastest. Or even the smartest. Just those who have the skills. In the real world, you'll rarely (as in, never) have to solve those same problems with the same speed you will in the exam.
In a lot (all?) of the jobs I worked, taking a day to solve a Medium level Leetcode problem was quite OK.
In addition I had to defend the report (120 pages of typescript and charts) of my final year project to my supervisor and another senior academic. And it was clear that they had actually read it.
All those exams were open note; anything in your own hand or a copy of a lecture handout was permitted. Again the weaker students would not have been helped by more time because they hadn't understood that you have to have enough familiarity with your notes to be able find the right information. Some brought in 50 litre rucksacks stuffed with ring binders and the noise of them furiously leafing through was enough for the invigilators to warn them to make less noise or risk being ejected.
In Norway it is typical that an exam of similar standard allows five hours.
It's not a problem with homework assignments as they have multiple days to finish.
So the professor has to decide between poor data with high variance, or good data with lower variance.
> I'm pretty sure that having more time would not have helped the weaker students get through the Quantum Mechanics exam.
Funny you mention that. When I did my (undergrad) QM II exam, I likely got the highest score, and I'm sure my score was below 40%.
There just wasn't enough time.
Such scores were expected when I was studying. No one got 100% in the kind of exams we had, not even those who got firsts.
The problem is that the professor has little clue on who can and can't solve the problems. Everyone doing poorly on a tightly constrained problem is mostly noise.
Pretty much everyone still took three hours. But you felt so much calmer, knowing you weren't racing against the clock.
Some people get up in their jimmies about this but if they don't want me doing something they should make it impossible to do it
</bait>
We had lots of time, and a fair idea of the range of questions. It rewards actuality mastering the material vs memorizing it.
For the CS exam some people brought more books than they could physically carry, I don’t think it helped them much.
I do agree that the idea of giving take-home exams and expecting students not to cheat has long since passed its prime. There may have been a time when it was reasonable to expect most students to behave honestly, but that does not hold at all in today’s climate. Especially post-COVID, for whatever combination of reasons, students just don’t seem to care at all about anything other than min-maxing their effort to grade ratio.
Which leads to a lot of outcomes like this one where students start using ChatGPT early, think it will continue forever, and then get completely crushed when they encounter the first assignment where they can’t cheat.
One particularly striking example was a timed, closed-book math exam where the exam paper just listed four problem numbers from the book. We were expected to open the book, copy down those problems without looking at anything else, and then put the book away. Honestly, that one felt like just asking for trouble and now that I'm a professor myself I think that the prof was unreasonably lazy, but it certainly captured the spirit of the thing. (Technically, that wasn't a Harvey Mudd professor but rather a professor at the adjacent graduate university. Maybe that had something to do with it.)
If you're the type that applies (and is accepted) to those schools, you are likely very informed on careers out there. So you also know what is at stake.
Good grades, or at least above a certain cut-off point, will open doors to prestigious jobs, as well as further studies. Finance, Law, Tech, you name it.
For these students, the stakes are high - and more akin to professional sports and draft season. I'm not saying this as an excuse to their cheating, but rather what the reality is for them. Again, not only is your competition smart and hardworking to begin with, but this sort of cheating is basically equivalent to academic and intellectual PEDs.
You could be studying English at Brown, with the intent to land a job at some management consulting firm or bulge bracket investment bank (1 out of 4 students at Brown end up in finance and consulting). Work that is miles away from your major, and where you're being provided the necessary training when you join the firms.
It is stupid, but understandable. And if you know others are doing it, it really only impacts you negatively if you don't. When so many of your opportunities come down to a two decimal number, people start taking risks.
[1] https://en.wikipedia.org/wiki/G._K._Chesterton#Chesterton's_...
Presumably you need some way to gauge the quality of your graduates
Between this and a decline in junior hiring, this is sorting itself out in the form of sharply declining CS enrollment. Which is fine, except for anyone with an interest in keeping enrollment high.
But the number of students applying for CS is actually up slightly.
Here is a whole article about it: https://alumni.berkeley.edu/california-magazine/2026-summer/...
the best way to do that would be making it school policy to issue transcripts with any grades you want any time after you finish your degree. set it up so theres no difference between one you got at graduation and years later. even if you never request one the fact that its possible makes it so treating everyone from that school equally is the only rational option.
but for that to work all the schools in a state would have to use the system so employers cant be like "you graduated XY university? thats the one that lets you fake grades right?" and treat you like you got a 1.0 GPA. we need to get the government involved.
or if you want a less radical/more realistic solution let people retake classes after they graduate to retroactively get better grades. the point is some number on a piece of paper you get in your early 20s shouldnt be visible and affect the rest of your life.
Another issue is with the curriculum and course structure, which should long been updated. But that's another rabbit hole on its own, especially in public universities with a big hierarchy of system. The sad truth is professors have no passion in teaching outdated curriculum, and students have no desire to learn.
At least where I live, universities used to run entrance exams. They didn't care about high school grades, or even if you'd ever been to high school: if you passed the entrance exam, you could enroll.
In class room stack ranking is just meaningless. Either student passes demonstrating various levels of knowledge or not.
I worked briefly in post-secondary CS education a long time ago, before academia turned into the ideological warzone it is today, and if I said such a thing, I would've probably lost my job.
Also, grades have long been inflated
Then stop inflating them. This is also what standardised testing is good for --- but no surprise, so many are against it because it would just show how terrible they actually are.
"The fish rots from the head."
It's not just because "it would just show how terrible they actually are"
Interestingly, at Reed, there is a low emphasis (or even anti-emphasis) on grades — a student has to go out of their way to obtain them. Instead, emphasis is on written feedback and discussion, to understand one's performance on assignments.
All this to say: de-emphasizing grades in school is not necessarily a bad thing, and does not necessarily harm the reputation of the university. It can be a sign of good priorities (eg: learning, rather than numbers-gaming).
That's odd, I would expect any employer for whom the class of degree mattered to know the reputation of the university. For instance second from Oxford is probably going to be as desirable as a first from Oxford Brooks (formerly Oxford Poly).
> before academia turned into the ideological warzone it is today
with this:
> and if I said such a thing, I would've probably lost my job
Weren't universities supposed to be exactly the kind of place where unorthodox ideas could be freely said out loud? To fire an educator for saying the wrong opinion about education doesn't make your university sound like the great place you suggest it was.
Education reform, including changing or ending grading, should totally be the kind of thing that people can safely discuss.
That bowled me over when I was young and still sort of working out effort/reward sort of stuff. I had put a lot of work into a lot of subjects where I wasnt very naturally talented and got a lot of mediocre results, but seeing that if I put the effort in continually I could make stuff thats worthy of recognition was amazing.
Meanwhile, my (now) wife was completing a diploma subject at the same institution and they were handing out pass/fail only. You could see a lot of people really confused about that. The quality of work that fit into "pass" ran a very large gamut.
Corporations were able to convince future employees to pay for their job training (i.e school). Getting professors to do the screening is not much.
You are not doing it for free. Grading is part of what UC gets for the $250k+ they pay a professor in salary and benefits.
HR departments will use whatever signals exist. If smart people tend to have college degrees, they'll use that as a filter. If smart people tend to have gone to a certain set if universities, they'll use that list as a filter. If colleges hand out transcripts with grades, and smart people tend to have better grades, they'll ask for transcripts.
HR departments didn't invent grades or transcripts.
I agree with your final sentence. The signal in grades (and even graduation) has been greatly diminished (even at brand name universities).
If you want to improve that situation a good step you can take right now is add your name to this open letter from UC STEM faculty: https://docs.google.com/forms/d/e/1FAIpQLSdwvDywR-CAt3t_U3Aw...
Are you suggesting a pass/fail method, or something else?
It was certainly not difficult to cheat at a "closed book" take home exam before.
Edit: typo
a) I got bullied into sharing my math homework so people could copy it, just like high school and college... but this was math grad school!
b) In 2011 I TAed a 4000-level course where the instructor left the solutions to the homework online (he wrote the book). I estimate 95% of students copied the solutions. It was only 5% of the grade and they paid for it on the exams. Still. Kind of stunning to see at U Waterloo - it was a continuous optimization course and most of them wanted to work in finance, yikes.
Im going to guess that it is a safe bet that the 50 cheaters are all legacy enrollments.
1/2 of the graduating class is there for the education, the other half is there because the parents are keeping up the network.
It's a sad state of affairs.
Surf the chaos, bro.
Source: am student @ Brown
A time when a take-home, closed-book test, relying on the honesty and integrity of the students, was viable seems completely unrealistic now. I assume there were always some who cheated, but I can imagine that it was sufficiently socially unacceptable that many didn't.
That seems like a very different world.
Back in my day, you could also just Google the problems and find the solutions. What mitigated cheating at UVA was the honor code and each professor's faith and trust in our integrity. That culture was enough to not cheat.
Imo, the fix should be to work on culture. Cheating should always be a tempting choice, so that the student may challenge their integrity, which is a muscle that can atrophy.
Yes it was. This was the pitch for as long as Google has existed. The only relevant change from the early days of Google is that now you also need to go to a T20 school and GPA inflation has gone completely insane.
Assuming the person you're talking to is still alive, this isn't true.
> The honor system is idyllic and requires idyllic circumstances: people who pursue education for no reason other than curiosity or self improvement.
This isn't true either; if your student body consists of only those people, you don't need any system at all. There is no point in even notionally punishing students who don't want anything from you.
However, some students will cheat, and for that reason, I am very much against curves. If you learn the material and demonstrate that you have, you should get the A. But it’s more work for professors to calibrate their curriculums, and there seems to be no real accountability for the inverse of learning objectives—teaching objectives—so curves are likely here to stay.
We can’t even agree on what’s wrong with it. We aren’t going to be able to fix it.
The decline of actual consequences for cheating has played a big role.
When I was in college, people who got caught cheating found themselves in a world of trouble. Repeat offenders faced severe consequences like failing courses, which could delay their graduation date if it was a critical-path course. This has a real dollar amount attached to it because you start working later.
Now it seems universities avoid addressing cheating problems at all costs. The professor in this article complains about how hard it was to draw attention to the cheating problem, with no response within his own department.
Students know this. As cheating gains critical mass and you see that nothing bad is happening to the cheaters, you start feeling like you're at risk of falling behind if you aren't cheating. The cheaters are getting higher grades (100% for many in this case) and they get to go out partying while you're still working through the material. You're really screwed if grades are distributed on a curve.
So temptation spreads. Anecdotally, I've seen a few young people lie to themselves and think that they're just going to use ChatGPT to check their answers and learn from it, but they don't realize how superficial it is to have ChatGPT fix your problem and then skim the correct answer. They put less effort into checking their work because they know they have a button to push to check it for them. When they get put into a situation where they can't rely on that button, it all falls apart
https://jamesgmartin.center/2025/11/harvard-admits-that-grad...
Interestingly, it seems like you weren't joking about the decline:
> Finally in the spring of 2022, a sanction reform referendum succeeded with more than 80% of the vote, changing the penalty for an Honor violation from expulsion to a two semester suspension. [1]
[1] https://en.wikipedia.org/wiki/Honor_system_at_the_University...
We had a very real threat of rustication. People still cheated. I think culture does play a big role. Of course, there need to be consequences too.
One place to start is to question seriously who actually needs a university degree and why, how this credentialism has been stealing years of productivity from young people. Though at least the deal was that in exchange you could expect some middle class job, even if gated behind a totally unrelated bachelor degree. But then this deal also got soured.
Couple that with increased awareness that classroom instruction (delivered by tenured research facility who seem annoyed at the idea of teaching) often has little relevance to workplace skills, I think kids have correctly surmised that the smart thing to do is say fuck the code and focus on checking boxes.
This creates some problems. Let's say that you go to university and major in X to get a job in Y. So you assume that the companies are requiring you to get the degree to prove that you are competent in X which is necessary to do job Y. But when you get to university you realize that most of your classes required to get a degree in X actually have nothing to do with X. Furthermore you also notice that students who just cram for the test and have no functional knowledge of X are still getting As and graduating.
Then you get a summer internship in a job doing Y. You notice that learning to do job Y really has nothing to do with what you learned about X in school. You notice that your mangers an the company who are really good at job Y (and all majored in X) have basically forgotten everything they know about X and know much less about it than you, but are excellent at job Y. You finish your internship and now know that you are perfectly capable of doing job Y. But, of course you can't get job Y yet because you don't have your degree in X. You have to go back to school and learn more about X before anyone will hire you even though you already know more about X than the people who do job Y and who you can see really don't need knowledge in X to do the job.
So you are being forced to get a degree in X to get job Y, but job Y doesn't really require knowledge of X. And on top of that, a degree in X doesn't even really mean you have any functional knowledge of X! Everybody just learns Y on the job anyway!
So now you have an exam in X, so you decide fuck it, why would I spend time on this? This whole system is retarded, I'm just going to use ChatGPT.
This was basically my experience in college. I never cheated, both because it was harder before LLMs and because I didn't really need to study much to get As, but I find it hard to fault people who do. I really struggle to see how integrity plays into this system. The system itself has no integrity, so having integrity inside it almost seems like being a sucker. If you want to change the culture to promote integrity, the education and the degree have to mean something besides an arbitrary bureaucratic gatekeeping device.
I never understood this behavior from undergrads though, you're paying so much for an education and then you just skip the education part? Why bother?
One need only look at the resume filtering process, a once manual bias that has now been codified into algorithmic bias with AI. A degree from a good school boosts your chances immensely, and other facets such as coursework don't matter much.
If you have ever seen someone filter applicant resumes, you will understand instantly. There are too many, you have to filter them somehow and the allure is irresistible.
Because you are viewing the motivation of college wrong for most people. For most people, the purpose of college is to get piece of paper that will open up higher salary opportunities. Ergo, they are just doing whatever required to get said piece of paper with least amount of effort.
Until degrees, in particular, degrees from well-regarded universities stop being that method, this behavior will continue.
It's why we're seeing the death of the liberal arts majors. It's sad, because usually the smartest and most creative people I've worked with in the field of engineering and software have been liberal arts majors. But corporations don't want intelligent people. They want people who have been molded to whatever the soup du jour is.
I don't think so.
The problem rather is that corporations very often want some very different knowledge of employees than what universities teach to the students.
If what the universities teach was very important for the job, applicants who have not invested serious effort into getting a deep understanding of the topics of the courses would nearly all fail in the job interviews.
The problem rather is that for many jobs the knowledge that you could have gotten from the university typically does not matter, and thus investing minimal effort into the courses does not get you rejected in a job interview.
Corporation A "Hi University, here is what we hire highly paid people to do, and what we need to improve as a company."
University B, "Hi Corp A, here is our educational mandate to create well-rounded, highly educated people, we can probably fit your needs into the curriculum in the last couple of semesters, let's work together to make sure you have good employees and we have people who aren't struggling to pay back loans because they have an engineering degree but can't make more than 50K at a dead end job."
I'm surprised Corporation A doesn't say "FU, we're just going to hire HS grads with high SAT scores and train them ourselves," but for whatever reason they typically don't.
More true at an Ivy than anywhere else.
> This year, the economist decided that both the midterm and the final exams for his course would be of the take-home, closed-book type (there is a certain tradition of this at Ivy League schools).
If this guy thinks AI is motivating his previously guiless student body to start cheating on these tests, rather than simply changing the way they are cheating, he's been sniffing too many of his own farts.
This is already starting to happen, at least for software engineering positions. There have been plenty of stories of candidates with degrees from prestigious institutions failing to answer the simplest of questions correctly. FizzBuzz is a famous example, but there are many others.
so you have a huge population of people bullshitting their way through to the piece of paper.
My first job out of college, I worked with veterans at the company who all got in with a HS diploma. Now you realistically need a masters degree to be competitive, for no other reason than that where I live (Norway) most applicants have a 5-year masters degree. It is basically academic inflation.
Here we have a tongue-in-cheek word "Mastersyken" which translates to "Master's illness/disease", a word for the phenomena that too many people are pursuing a master's degree for the sake of the diploma alone, trying to become more attractive in the search for a job, but with the side effect that suddenly "everyone" has a master's degree, and in the end everyone is stuck at the same place as before, but with extra student loans.
The worst part is when you start working, and indeed discover that this is a job you could have done just fine straight out of HS.
A master's in science helps you understand how the physical world works and how to reason quantitatively as well as qualitatively. A master's in humanities gives you knowledge and understanding of human culture, such as literature and the arts, and history - subjects that can be deeply enriching and can provide insights that transcend disciplines. A master's in social science will teach you about how humans behave in groups and how they interact with their environment, and about statistical analysis.
Writing a master's thesis will also teach you a lot and make you a better writer - if you actually write it yourself and don't rely on AI.
Any of these degrees will certainly qualify you to be a more interesting, knowledgeable, and insightful barista or Uber driver.
Business majors should take physical science courses with a lab component! How else are they going to learn anything about reality?
But there is no excuse for bad teaching, anywhere (especially given how insanely competitive faculty positions are - even crummy adjunct and lecturer positions.)
Unfortunately research universities prioritize fundraising > research > teaching. And sometimes grad students are selected to teach based on financial need or departmental requirements rather than interest or ability.
It's also dangerous to assume that higher education is for everyone. (Although I agree the opportunity needs to be there for anyone who wants to try it.) Some people just want to get on with their life after high school. (I raise my children assuming that they will go to college, and if they want to seek an alternate way through life, I will support it.)
If students don't want to be there in the first place and/or don't see any value in learning, it is unsurprising that they'd take the easy way out. Or maybe they cheated their way into Brown and are just continuing.
But I was always interested in learning, and understood that cheating was a method of learning avoidance. Why waste the amazing learning resources - faculty, teaching assistants, courses, labs, libraries, studios, rehearsal spaces, interesting speakers, arts and culture events, computing facilities, maker spaces, etc. - that are available at a place like Brown?
An expensive education comes with higher temptation to cheat.
You've never put anything off in your life or taken the easy short term route? Come on this is not difficult to understand. You aren't different either.
* This is brought to my attention by an exam question
* I have an oracle in the form of a textbook, an LLM, the internet, or all of the above
Which action is skipping the education: looking up the answer, or not looking up the answer?
Teachers really need to stop doing it, it's so destructive to create a metric that tracks how well you can cheat and lie, and basically forces you to cheat and lie (because everyone else is) if you want to get a job.
IMHO to solve many problems we should go with Ivan Illich's ideas: https://en.wikipedia.org/wiki/Deschooling_Society and make education about education, not testing and certification.
We tried this. The whole grades-are-racist nonsense movement (in part a reaction to NCLB).
It doesn’t work. Learning requires confronting difficulty and evaluation, even if you’re learning alone. If you want to see what attempting to learn without tests looks like, see people who think they’ve taught themselves a topic by chatting with AI. Nine times out of ten they think they know something but can’t solve actual problems on their own.
> The whole grades-are-racist nonsense.
Wow, swerve off topic. I never mentioned anything like that and I do not want to.
I like learning from textbooks and others. I would say self testing myself is the difference between reading nonfiction and trying to learn something.
> never mentioned anything like that and I do not want to
It’s the rhetoric that was used to push back against examination in California and New York. Explicitly. The rhetoric was new, but it was essentially Illich’s ideas.
The result was kids in wealthy districts with tutors or with educated parents who valued learning for its own purpose (and had the time and skill to convey that to their kids) did well while those in poorer districts got left behind because absent measurement you have no accountability.
I don't mind grading exams (though I think grading is somewhat futile - see my link elsewhere). Sometimes an examinee will come up with a new idea, but we don't need formal examinations to check that kind of thing out. And you can do it in a comfortable chair, while listening to Miles Davis, unlike invigilation.
Based on my own experiences and those of people I’ve watched, I have three points. Sample size of 1 and all that; but it’s a lowkey passion topic for me.
- The argument that AI is “the next calculator” and education testing and overall methods need to adapt, is true.
- The distance between learning concepts to passing memorization tests(or worse, non-reality test specific logic) has grown significantly in my lifetime, and the AI education problems are really a mix of the cascade impacts of this issue + the AI impacts on conventional education measures.
- The problem with students who use AI to get past (often arbitrary) difficult courses or testing scenarios may look like one problem compensating aka solving for another (and I love AI for it!) but the parallel problem of AI (and our education system baseline) enabling students to normalize non-cognitive drone like approaches to any problem is super problematic.
The last point is an admittedly recent eye opening moment for me. Working with younger students recently, they are shocked at anything less than a fully clearly defined problem (education system training) and anytime thinking is required, they go to AI (which is fine!) and they can’t think beyond the AI output (which is super not fine!) The latter has been astonishing for myself and coworkers and really has us reconsidering our young talent programs.
But i also did a semester abroad at a German university and i found there was a different philosophy there where the assignments don't teach you much, and everyone skips class and then studies really hard for the final exam which is often 100% of your grade
I’ve talked to a bunch of teachers and school leaders, and see three main ways schools are handling AI use in assessments:
1. Punish it: Detect AI use on homework and take home exams; treat it as cheating.
2. Prevent it: Move to live assessments – oral or offline – that are hard to cheat on.
3. Embrace it: Assess the process, not the output.
The second one seems to be the only real answer for foundational subjects. And the third one can also work for more creative or project-based work.
if ai doesn't help, then it won't help. if it does help, then you should use it. the metric is your output of whatever is being tested. writing an essay well and clearly understanding the material. solving a pset. whatever.
if you give access all the time for that, and then you test on a hard problem that could be done with or without ai, then it's fair. e.g. "clearly explain these four sentences of Y." obviously ai researching loosely and blathering isn't useful. won't be high signal / dense and correct and worthy of an 'a'. but someone who can harness ai and someone who knows the material well in the end will be rewarded the same by society. what you are testing is correctness and information density in a response. so you have to start now in accepting the reality that those who use ai to get there should be rewarded just the same as those who don't.
the burden is on educators to be as good as they can with ai if it is relevant or not if it is not relevant (and schools to fund them and ai companies to fund them if they have excess capital and are humanitarian).
and note the hard part even for us engineers at tech companies is in the correctness. it is very hard. but the sooner we start teaching how to do things correctly with ai, the more prepared the next generations will be.
Serrano did not void the midterm exam, but warned students that the final one, which counted for 50% of the final grade, would be held in-person. He also said that if the grade distribution was not similar to the midterm, only the final exam would be taken into account. The average score dropped to 48 out of 100. Of the 89 students who did the midterm exam, only 59 showed up for the final one. And of the 27 who did not show up, 22 had scored a perfect 100 in the midterm exam."
Running all your students exams through an AI checker is one of those Temptations of AI I am sure.
So just mentioning my own anecdata here; I've had reasonable success converting a "code + written report" assignment into a "code + structured video report", where the code is assessed "exclusively" through the video report, and where the student needs to demonstrate ownership and understanding of the code, explain and defend design decisions, etc.
Yes, in principle it is possible for the student to generate a "script" and narrate it during the video presentation, but this is fairly easy to detect, and in general, given enough flexibility / subjectivity in the marking scheme, you can always downmark such students on the basis that their presentation demonstrated lack of understanding and ownership of the codebase, without having to go into too much specifics.
Obviously one implication of the above is that you no longer care if their code was "manually" written vs AI "assisted"; only if they have a good grasp / ownership of it, and can defend the broader architecture and design decisions. But I feel that this is realistic in the age of AI, since this is largely what will be expected from their future employers too.
Even the idea that MIT is somehow better than some other universities is itself a fiction. People are conflating financial success with academic ability. The former is mostly the result of social connections which are formed within the academic institutions and have very little to do with actual capabilities.
Universities should just sell degrees for a high price without requiring the students to attend. If they're rich enough, their skills aren't going to matter anyway; they'll succeed in their careers regardless so the university will still look good. 'Academic integrity' will be intact. Especially true for business, economic degrees or other humanities.
I seem to recall they submit the exam via usb drive.
now they may have a closed exam mode where they can finish the exam without network and the. Submit over the network.
Of course this type of exam must be conducted in person or else the examinee could simply use a phone and type the ai provided answer.
Also is the professor living under a rock? Everyone is going to use AI if they can. So if you want them to not, asking them nicely isn’t going to cut it.
Exams are definitely in person. Homework is not graded, except that once a week, one problem would be randomly selected from the homework for an in class quiz. The numbers would be changed up.
From my perspective, you didn't really have a choice but to be able to solve all of the problems on the homework unless you wanted to gamble failing that week's quiz. The answer key and AI are always available, but the learning got to happen on my terms. It was a good balance. And grading for the professor was relatively light since it was only one problem per student.
I have no doubt that AI was used for this, but I am shocked that this is even a thing. Before AI, any rich or connected student (which I assume is essentially all ivy league students) could just get someone to take the test for them?
This is a trap. I understand they've done this in the past, but profs are paranoid now.
I don't believe he's 100% correct on each incident of fraud and he's going to ruin students [academic] lives because of it.
...
"But it also hurts him that the one time in 34 years that he decided to offer a take-home exam, for highly justified reasons, the response was wide-scale fraud."
-----------------
Not to in any way defend or condone academic misconduct, the fact that this was his teaching-career-first take-home exam is probably relevant. Take home exams can be fiendish. I remember having one in grad school where we were given a very insufficient 36 hours to complete it, and many people just didn't sleep. That was from a prof who knew what he was doing. This guy may have accidentally made his exam absolutely sadistic.
Couple this with the fact that students often have other exams they need to be studying for in the same time window. The pressure can be immense. The temptation to use AI to help is going to be hard for many to resist unless the penalties are severe and strictly enforced.
AI cheating is probably going to be a problem going forward in all situations, but open-book, take-home tests are going to bring it out more strongly than other test formats.
A lot of the cheating students would have concluded that other students are going to cheat and get a much better result, so they might as well do the same.
The fault lies squarely with the professor.
Of course 70% or so usually crashed out in particular in Calculus and I suspect given that US education is paid for daycare that's exactly the thing that can't happen which is why they're never fixing it
I think the tricky part here is essays, but maybe they should be replaced with an oral exam + essays? A lot of the humanities and classics courses I took were pretty small.
Overall I don’t think these problems are that hard, I just don’t know if the incentives of universities align with fixing them. If their reputation is all that it’s for, grades don’t really matter as long as the $$$ is coming in.
But being upset that people use available technology to solve problems is quite an exaggeration and makes the guy close to being a luddite. He can just say "Hey, we do exams on paper and in class next time" and be done with it if he does not like technology.
All I know is I wouldn't want one of these cheaters performing open heart surgery on me, designing the airplane I'm flying in, or managing my financial affairs. Or marrying my daughter.
I would have thought the defense of cheating to be a much more marginal position.
Hand written exams are either very labor intensive to grade or are confined to multiple choice, so either inflationary to cost of education or inauthentic / inaccurate representation of most knowledge and skills.
The best answer, which enables authentic meaningful high integrity assessment that is also unit cost efficient is to have testing center facilities with institution supplied devices and well trained proctors.
This way instructors can assess students in ways that are relevant and authentic to the subject matter while ensuring the assessments are accurate, consistent, fair and actually reflect the students abilities.
I'd like a citation for this being the "biggest known scandal" in the "entire Ivy League". Frequently such situations are kept somewhat quiet, for a variety of reasons. But fifty students is not a large number in courses that can enroll hundreds or up to a thousand students.
You can crunch the numbers on this to verify for yourself – most of these have a population of 6000 undergrads or so. 1/6 of all of the undergraduates need to take a single course for your "up to a thousand" to be true.
I agree that a citation would be nice, but the number of students is indeed large for Ivy League courses (as per the article, in this econ course, he had 86 students for a class that usually only has 30).
I’ve studied in classes of 40 to 50 students to one teacher. The challenge of grading students at that ratio, means you need automation. It also reduces the kind of teaching possible. You really can’t provide one on one attention when you have multiple classes of 40+ students to teach.
And med schools are quite focused on grades.
Same goes for Law Schools, which, again, Brown is an outlier for JDs.
Look, the professor here is right. But he's living in a different time. The students are under the gun here and are responding to their incentives. The prof's gripe should not be with the students but with the AMA or the Bar associations, ultimately.
Essentially, the system is sick, the kids and prof are caught in the middle of it. Don't fight each other, fight the power.
He understands the stakes here. If a university degree becomes useless, then what?
Maybe the students cheated, maybe they didn’t. But the article does not seem to provide actual proof that they used AI. It mostly presents circumstantial evidence: unusually high scores, similar answers, and a later drop in performance on an in-person exam.
https://web.hedc.shizuoka.ac.jp/msg-from-center/creating-mea...
Edit: Didn't realize the original URL was paywalled. Sorry. This newer URL is open. Apologies.
FORCED SKILL-BUILDING GOALS
Where my original purpose of giving homework assignments was to help students build their skills through active engagement, I can no longer trust that all my students will have done it by hand. But I still want my students to feel the need to improve their skills. So, I start by considering which skills I think are most important for that class (writing, for example) and build a forced need for my students to improve that skill. For example, in my writing-focused classes, I now give students in-class hand-written exams. I let them know about the essay-based tests (mid-term and final) on the first day of class and encourage them to practice their writing each week, in-class and at home, thus giving them a very clear reason to practice — to get a good grade. Additionally, I take away the need to get a “good grade” on written homework by giving them full points if the homework is turned in on time, regardless of contents or mistakes. Thus, with the need to practice their writing skills for exams and no need to worry about making a good grade on homework assignments, students are encouraged to try the homework themselves — they cannot fail and they can only benefit by doing it themselves. In fact, using an LLM to complete an assignment will likely hurt them as other students who improved their skills will do better on the exams.
Homework once again can be used as a tool for them to improve their abilities through engagement, not just a pointless activity that an LLM can do to help them “get a good grade”.
FORCED LLM FRICTION
Another method I have found for encouraging students to complete assignments on their own is to purposely create friction to using an LLM. I try to plan assignments that it would be MORE work to use an LLM than to simply do it themselves. Humans being what we are, we often will choose the path of least resistance, especially if a grade is on the line!
For example, in one of my classes, I have students write a short story for homework (of about 250 words) about a picture I show them in class. Spelling and grammar mistakes are OK, they are guaranteed to get full marks if they turn it in on time, but the story has to be written BY HAND. To use an LLM, a student would first have to write a lengthy prompt describing the picture in detail and explain the type of output they need (a lot of work), and then they would still have to rewrite the entire story by hand (forcing them to focus on reading, spelling, and handwriting anyway). Thus, by introducing forced friction to using an LLM, especially where there is no danger (no friction) of getting a bad score because of mistakes, encourages students to take the easier path — just do it themselves.
The same idea also applies to shorter listening assignments. As long as the homework receives full marks for turning it in on time (low friction against the fear of poor ability) and the exams including a listening element (clear purpose for trying), the hassle of downloading the audio, sending it to an AI to transcribe it, then upload pictures of the homework questions just isn’t worth it. It’s just easier and more beneficial for students to do it themselves.
This is what my university (EPFL) was doing ~40 years ago. And yes, some students didn't bother with the homework, and mostly paid the price at the final exam...
Unfortunately, if I give homework that will not be graded, almost no one will do it. But I WANT them to do it for the practice it gives them.
The only reasonable answer I could find is to award them full points for simply doing the homework on time, even if it has errors.
That said - requiring students to hand-write answers is reasonably effective. It's a lot more boring to hand-copy text out of an LLM answer than write it yourself, and it makes the "cheating" significantly more visceral.
"Of the 89 students who did the midterm exam, only 59 showed up for the final one. And of the 27 who did not show up"
59+27=86
Maybe the article was written with AI that doesn't check its math.
> Turkstra said he’ll still use the AI detection tool going forward.
That's crazy! AI detection tools are notably unreliable, and even if they had only 1% false positive rate (I am sure it's actually much higher) that'd still be multiple innocent people failing the class for no reason.
Imagine enrolling in the class, and the professor says: "oh, and btw I am going to randomly pick three students and accuse them of cheating". Would you want to stay in this class
I think the future is going to be proctored exams on paper or on locked-down devices. If there will be projects, they should be accompanied by secondary evidence, like interviews about them.
I would be truly ashamed to assist a top notch course only to turn to a computer for answers.
I would prefer to fail forever and learn the hard way before doing some cheap copy/pasta.
I was very fortunate to attend Brown University for grad school, and consider it a great place.
Why would many people who were also fortunate to attend there not honor that opportunity?
A single person can easily do that using glasses with a micro-camera & rice-sized earbuds, and almost impossible to be caught.
This equipment is not very common today, but if the smart glasses become popular, then universities (or proctoring centers) will get it.
And if they don't become common, then some fraction of cheating is acceptable, as long it's not too high. After all, one could do micro-camera + earbuds even before AI, with human conspirator.
Another point is that finding and using a capable human accomplice costs much more, with much more risk than asking AI.
I went an ungrad school that was top-5 in engineering. But my experience - and in the experience of other people I've talked to - formal undergrad education was, and always has been, a farce. At best, you learn through working on projects that are meaningful to you and learn "how to be an adult" (and later, you learn how to manage the enormous financial debt you acquired). But more typically, it's pure credentialism - no one cares what your grades were, only what school you graduated from.
The amount of actual learning that goes on from classes is minimal, but somehow we can't shift the overton window away from this silly game of grades that don't measure anything meaningful.
After graduating, I've was asked about my grades exactly twice in my life -- once when I applied to a master's program, and at one job interview (the company had a policy of asking about GPA for anyone who graduated less than 10 years ago).
I'm pro-education but anti-school, and all this nonsense makes me this way even more.
Accurate and high-quality exams are a solved problem. The issue is that universities aren't necessarily judged on teaching quality and opt for examination methods that scale well.
The obvious problem with that is it is terribly expensive. You need Masters or Doctoral level people for long periods alone with students and you need to trust that these proctors won't be some form of -ist towards the students and also that their grades will be fair.
It is by design not something that scales.
But it seems that is where the path lies at this point.
Essentially, the aristocracy gets education again and the plebs get to fight/cheat it out amongst each other and paying to do nothing in the end.
Damn it
I hope it does.
The problem is that kids, even (or especially?) in the elite schools, are treating college as a box to tick. At the Ivies, for many students, it's nothing more than the requisite means to get their ticket punched so they can become/remain part of the elite class. This has been the conventional wisdom for half a century.
Now, this paragraph doesn't apply to an economics class like his, which is actually a useful degree, but many students will never use what they learn in college in a professional capacity, so it barely even matters if some (maybe even most?) students cheat because outside of the useful degrees, it's mainly a sorting hat to determine who works at Starbucks to pay back student loans, while rolling their eyes because of their 'valuable degree,' and who works there unironically, to pay their bills.
What needs to happen though is that students at all levels need to either believe that they need to learn what they're choosing to take courses in, or even better, actually innately want to learn it to satisfy their own interest. Either will do. If you have neither intention toward the material, of course you'll cheat your way through it. No student who actually wants to learn would waste their time and money taking a class only to not learn, and cheat their way through it.
Since students are notorious for being cheap
Still cheaper then being jobless or without the bachelor
These news articles are just tiresome at this point. Obviously folks cheated previously, obviously it's easier now, obviously the answer has been to not have take homes all along.
Ivy league is such a scam, in so many different ways.
It was a closed-book exam. The professor shouldn’t have to hold students’ hands for them to act with integrity, they are all adults.
In this particular class, the professor made the final exam in-person, and didn’t count the take-home midterm because the score distribution wasn’t consistent between the two exams. I think that’s a reasonable approach, but it’s kind of sad that it was necessary
Whether that really matters if your goal is to climb the social ladder and have power and influence, I don’t know.
It's become very difficult to have even a middle class lifestyle without a college degree. Obviously a huge percentage of people there don't want to be.
I've noticed more and more that people lie to me and I call them out immediately (aggressively, as if they've just spat in my face) and they just don't care.
Does "my word is my bond" not have meaning any more?
Pretty funny - cheating so bad that even a blind man can see it.
If it's a math for finance course then some kind of model building for the midterm and being marked on the quality of the model or something. If AI becomes so good that it always chooses the best fitting model and requires no numerical optimization then they will have to change the courses to be more like UChicago where it's primarily undergrad directed research but AI assisted.
You can only lead a horse to water, you can't make it drink. Maybe a student's sincerity should play a larger role in the admission process, maybe with a sharp expense curve such that students judged to be more sincere have to pay less tuition. It is an inherently subjective evaluation though.
Edit: I completely misread your comment. Asking students to build a model is not a finance class anymore.
For example the old and new Berkeley model to study rent control effect on market prices