I’m 30 and “we can’t do tests in paper” seems _insane_. Just how metastatic has ed tech been in what, 9 years since my undergrad?
It's fine (tho annoying when you lose points to "typos"), but it limits what kinds of problems you can reasonably put on the exam. You'll definitely lean a bit more into theoretical stuff than practicals. Which is fine for some courses, I think a bit less interesting in other courses.
Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example
Having said all that... "we've booked the computer room, you don't have internet, go type up all your stuff in this VM we have set up" feels fine if you don't like this constraint IMO
How practical is this?
1. Is your institution able to provide this support?
2. Do you believe you are able to supervise the room well enough that students will be caught if they cheat? (Eg, bring a phone and look up answers.)
I don’t know about 1s practicality. In my schooling it would always be doable but I have the impression US schools are a different scale
If you can escape the vm on the fly and manage to use an llm to cheat, you deserve that A.
This is basically how ICPC works (https://icpc.global/)
Contestant computers boot a Linux image that contains all the interpreters/compilers and documentation for the supported languages, some editors/IDEs(vim, Jetbrains, Eclipse, etc) and a web browser.
Network access is limited to the judging system only.
My experience in Europe is that most universities can support this.
If you're doing a good job, you have to do that anyway, or at least have enough of a spidey sense for broken code to know when to investigate and add an extra test case.
Something like 30% of the time at $WORK, interviewers report the candidate as having solved the problem when a closer inspection reveals UB, memory corruption, and other bullshit. The test cases pass, and I think that's part of the problem. You can't tune out and avoid deeply understanding the submission.
I think the problem is that the grader has to run your code in their head. That's a whole different problem.
I'd say compared to just typing the code on a disconnected computer: Not worth it.
The grader has to grade N exams.
How do you think English, philosophy, or history tests get graded?
Or were you just assuming your experiences (large university, I'm guessing) are universal?
There are many, many colleges that have no grad students. Period. They are undergraduate colleges.
Expecting any college or university to just be able to scale up the amount of manpower required to grade their classes/exams by a large multiple is absurdly unrealistic. I assure you that most existing colleges do have plenty of resources to effectively evaluate individual students' outputs, if they don't have to assume that anything submitted digitally is written by LLMs.
That new requirement is a massive change to how they operate, and it's incredibly rude and short-sighted to be dismissive and contemptuous of institutions that have trouble adjusting to it.
That Computer Science is somehow a privileged subject where the exam grading must be automated is elitism of the highest degree.
So in some theoretical hiring example, if I have to choose between two candidates: one which is really good at keeping the code in the head and the other who is less good at that, but who is very skilled at using programming tools like advanced type systems or formal verification tools, then I'd prefer the latter.
Of course in practice, I would of course take a student that I knew had passed their paper test with their own knowledge over a student that likely ran their digital test through an LLM.
> or whatever is the tech of the day.
Rote memorisation has been the standard technique probably for way longer that anything else so I’m not sure that every innovation is inherently inferior to what we had before
No wonder some people are so easy to replace with an LLM.
this would also end up valorizing particular ways of thinking or developing, so those whose development methodology is more exploratory until you get a certain point where you get an overarching view of how everything should interact and then go into a quicker iterative mode would be penalized.
I guess it's ok it does this, some things after all must be valorized whatever is chosen, although I've known many people who are quite good using this way of working who would then be downgraded.
Interesting. This reminds me of COBOL and FORTRAN "code sheets" that were used back in the day...
We had 3 hours to make the test. The papers the teacher delivered had differences between them (I think it was four different versions of the test?). The Teacher didn’t stay in the class also, you had to deliver the test to them later in their office.
IME, that kind of nitpicky grading mostly happens when the exam was designed too easy and does not depend on the medium.
> Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example
Which you really need to do anyway because code that is broken due to a brainfart and code that is broken because of a fundamental misunderstanding should not receive the same grade.
Additionally expecting whoever is reading the paper to comprehend everyone's (likely very sloppy, in this era) handwriting is an exercise in frustration for the person who would be evaluating the papers.
Not that tests/exams can't be given on paper, ever (multiple choice still works), but for something where people are expected to provide multiple pages of coherently written essay output, I would struggle to do it by hand. And I'm old enough that we did do a lot purely on paper when I was in school.
There will likely be a period where those who went through high school with computers struggle with hand writing stuff, but the next generation will have done it all their lives.
If you did it before then, barring physical limitations which have occurred since then, you would struggle for a short while and then you would be fine. I also did coursework purely on paper, got out of practice, and then once my kids were in school got back into the habit of handwriting. I can even write cursive again. Cursive makes it much easier on the hand and wrist. There's a reason it was invented. :)
For righties.
I'm right-handed, but I had to learn left-handed cursive at school when I spent 5 months with a cast on my right hand.
I think you're missing the point. This is a physical limitation. People don't write as much as they used to. I used to be able to write for hours, taking notes, writing papers, etc... Now, I try to take notes during a meeting and my hand cramps. And as a bonus, my handwriting never was great, but now it's illegible for me.
Yes, we all "could" re-learn this skill, but how many people will? If you asked me to type a paper on a locked-down computer, I could easily. If you asked me to write a 2000 word paper by hand in an hour, I doubt it would happen.
If we expect students to take in-person tests on paper, then you should also do the rest of the classwork on paper. I am completely in the camp that we learn more when we write something. The physical act is part of the learning process. But you can't expect students to write an in-depth exam without having the practice of doing it often.
Are you using a fountain pen, writing cursive and do you take care to hold it properly? Those three things fixed all my cramps
Say what? Writing by hand is not only useful to write. It helps practice fine motor control. Something you need in many places in life, and most people have no practice of other than by writing.
Doing it now I really don't think I could deliver my intellectual best while worrying about if anyone can read my handwriting and whether I'm gonna cramp up by the end of the exam.
Pen and paper is just not a very good way to produce text.
Pen and paper are literally designed to produce text.
Can you say more? It's worked well for 4,500 years. Probably the most important invention ever.
So it’s like saying that clay tablets worked just fine for 1000+ years why would we switch to papyrus? Or who needs character for vowels? Or spaces between words, it’s not like anyone reads texts silently?
Then we got paper and pens and that was a pretty decent interim for a short period. Then about 100 years ago we got typing. Then about 20 years ago we reached a world where almost everyone is better at typing than they are at writing.
Obviously it's still important that people can write by hand, but expecting people to do it for more than a few hundred words at a time is idiotic. Would you like your clothes to be hand sewn too? That also worked for thousands of years (much longer than writing) but we stopped doing it for a very good reason.
Initial cost is almost certainly not a factor; the components could be so old as to be free.
Playing devil's advocate: one reason I'm good at typing on computers is that I learned on typewriters, which need much more force for each key press, especially the little fingers. Someone who learned on computer keyboards won't have trained the muscles on their fingers as much, and will have a hard time pressing the outer keys strongly enough (and if you press the keys too lightly, the mechanism won't hit the ink ribbon with enough force to transfer the ink to the paper).
I also challenge that "hand writing is harder to revise"; again, why is the speed it takes to write it at all relevant?
What? Suppose you want to fix the opening of your essay. Best case it's pencil and you can erase some, but worst case you have a longer sentence you want to put in there so you can't do it without scribbling all over and making a mess of the page. Word processors let you edit. How is this controversial?
> why is the speed it takes to write it at all relevant?
Okay, so from first principles:
1. Time is finite, we will all perish
2. Unless you are doing open book, needs to be supervised (proctored / or prof/TA is there)
3. That person is paid for a shift
4. That shift must end
5. Therefore, anything that enables people to write faster is good for the students, who can get more paper written, or the paper better revised, during the finite time available for writing.
You don’t write the test to fill the 60 minute slot. You time it so students are able to finish early if they’re really good. Slow ones need the whole time but can still do well if they understand the material being tested.
- someone with a good understanding will often come to a concise, clear answer while someone struggling will produce a convoluted paragraph.
- the way to get to the result will vary depending on your understanding (e.g., are you blindly applying some method or understanding what's going on). For instance, "hey, this is a vector field, I don't need to evaluate this complex integral, I just need to compute the difference between the start and endpoint of the curve!". Both answers will be correct, but one denotes a much better actual understanding (and will take way less time).
But then I'd use that to write the actual text answer, and consequently, it was mostly clean. I'd still have to cross stuff out or add a phrase between the lines every so often, but once the basic structure was done, that was mostly unnecessary.
And if I was ever pressed for time, I'd usually get part marks for the outline anyway. Depending on the subject, I'd sometimes race through the questions doing just the outlines, and then come back to write the full text as a complete second pass.
To be fair, this wasn't creative writing. I think it'd be harder when the expression is the content.
I think you're missing some logic in there somewhere. If a student fills in as many pages as they can, each with the number 1 written on it as large as possible, was this a good conclusion? The quantity of writing is not a good metric.
EDIT: to give a closer to reality example, an essay that's 4 times longer than a competing essay does not make it better.
Most curricula should probably feature both forms of assessment, demonstrating your knowledge of the basics in a closed book assessment and your ability to produce high quality final products using all resources available to you in take home assignments.
You're thinking of the output as the goal rather than as the means to an end.
Honestly, school essays, as a text genre, suck to read. They are not meant to be enjoyable, beautiful texts! Instead, they are meant to help gauge the student's ability at text planning, the quality of their text project, their grammatical knowledge, and orthography.
I must say I'd be open to seeing schools attempting other models, though, and measuring the results thereafter to see in practice what happens when students type their essays instead. In terms of text projects and planning, that could be interesting and train different skills.
But this is missing the point. The issue is not that people in general can't write by hand for hours, but that these particular people may have difficulty.
Define “worked” there are reasons we have abandoned quite a few archaic practices which turned to be highly ineffective in hindsight (and often quite misery inducing). I’m not implying that handwriting is one of them just that it’s a terrible argument.
> burden of proof is on the side saying that kids
Smacking children who fail to give a correct answer with a stick worker for thousands of years as well..
That's a strawman. I said bottlenecking testing by requiring hand writing is stupid. Put another way - you're meant to be using the test to be thinking and re-thinking the problem and articulating approaches and solutions, not use your time, effort and energy on managing quill and paper.
Testing for understanding requires the fastest WPM possible. Regardless of the method used, students don't all write at the same speed. To minimize the influence of this variance, they need to use the fastest writing technique possible. If a fast writer spends 30 minutes writing in a two hour test, and a slow writer needs a full hour, the fast writer gets 50% more thinking time. But if you double both their writing speeds, the fast writer's advantage drops to 17%.
The faster the writing technique people are allowed to use, the better your test can identify poor understanding.