Hacker News Reader

Oral exams are still common in Germany and many European countries in case of smaller courses taken by 20-30 students in a semester, and sometimes even in larger courses, and the task of examining is then split among multiple professors and lecturers.

In Hungary it's a pipeline. You go in a room, you pick a piece of paper out of a hat, which gives you some task or topic to prepare for. Then you sit down at a desk (the room is filled with about 15 students working already), then you solve the problem or sketch out your notes to present the topic you picked for like 20 minutes, then you go to the professor and you have to, in low voice, explain your solution, the professor asks back and pokes at it to make sure you understand. The professor then gives a grade immediately and you go home.

In Germany, it's usually no random pick of a topic, you just go through the entire semester's material over 30-45 minutes and the professor stops to spend more or less time on each topic to explore your knowledge. You can explain things by making sketches on paper. A PhD student sits there quietly taking notes of what was asked and what the student answers approximately. You leave the room, the prof and the PhD student discuss the performance, decide the grade and you come back, and the professor explains the grade and what you did well and not well.

I'm not familiar with the German universiry system, but I think a 1 hour, 2 professors (or one professor and one PhD student) would be impossible to scale. A class of about 100 students could take an entire month! (or better, 2-months of professor-hours!)

What is the point of this reply? GP already specified oral exams are mostly reserved to courses with smaller numbers

In order for it to be a solution against AI it needs to also work for courses with bigger numbers.

At low student counts, like 20, (think specialist niche elective master's courses, not the huge bachelor's courses) it's perceived by profs as more efficient in fact than the ordeal of organizing a written exam.

With an oral exam, the prof blocks off 3 days or so, with ~8 students per day and it's done. It's not a huge additional effort compared to all the effort of giving the course over the full semester anyway.

An added, though minor benefit is that the professor might decide to offer a student assistant job to someone who performs well, because it's much easier to judge who is actually good in an oral exam vs in written. There's a reason that job interviews aren't just tests filled out on paper but involve personal interactions.

But that is how certain exams are done right now. Though it's usually 30 minutes including overhead. It's not something you can do in classes with multiple hundred students, but it's certainly done at scale.

I was the PhD student in the room in dozens of these exams, and I toke some of them myself. But that was for classes with a bit less than 40 to maybe 60-80 students. The load for the larger ones was shared among professors, if I remember correctly. But it was certainly the case that a single professor spent 1-2 days doing nothing else but these exams.

And it was 1 professor and 1 PhD students, and those don't count. They essentially grow on trees and are always available, if you need some.

> would be impossible to scale

I think education is something better scaled horizontally than vertically. More schools, with more professors instead increasing the student per professor throughput. The classes where I learned the absolute most, and had the best experience, were closer in size to 25 than 100.

The US has proved it can do horizontal scaling, as the majority of adults gets a college degree.

The horizontal capacity is ineffectively used by having many degree mills and low-value degrees, but if we re-tooled the knowledge sets we could raise the bar.

I actually think of degree mills as being vertically scaled. Online courses, a lot of course work graded by computers, large classrooms, etc.

As for the quality, I think small classrooms with good teachers probably produces a better outcome than large classrooms with great teachers.

Only about a third of US adults have a bachelor's degree or higher. Even if you count associates degrees it's still less than half.

If you filter for younger, 25-34, and for employed, it's closer to 50%. For employed young women, it's the majority (for them, if you count associate degrees, it shoots up close to 75%).

One big difference I observed between the US and $HOMECOUNTRY is that US colleges require a lot of courses with little credit, so a normal college year is N courses x K credits per courses, N large, K small. In $HOMECOUNTRY it is typically N small, K large - think instead of having Calc IV, Trig whatever, Algebra 2 and Introduction to Proofs, you have a single "Mathematics" course that last a whole year, not just a semester.

Since each course needs its eval, assessment is a function of N, not N*K or K. Much easier to scale if there is ~5 evals per student per year than ~20.

it doesn't. in Italy we have been doing it for ages.

Oral exams can also compensate a lot for different types of exam anxiety. Blanking on individual points doesn't matter as much, the examiner will help you along or ask along a different path. Doesn't help for someone that completely shuts down, but it's much easier to determine if a student was just stuck on some minor point and actually knows the topic in an oral exam.

They are likely prone to more biases than written exams, and the variability among examiners is likely high. My experience is both in taking these exams myself, and also sitting in on them as the note taker many times. Though that was only with one specific professor, and almost always on the same course. But that was in my experience a very fair exam, even for those students that were very noticeably nervous.

I think you have the right observation but the wrong conclusion.

The appeal of digital systems, at a social level, is that they can transmit information accurately for almost no marginal cost. They come at the risk of fraud.

With AI the levels of risk have increased. Anyone can now be a fraudster, and it gets harder and harder to prove anything online. Either of us two, in this conversation, could be a well tuned LLM.

So society, collectively and at an individual level, is forced to reassess reliance on purely digital information. Which means that the advances in the technology, actually end up reducing the utility of digital systems more generally.

You can argue that this makes education costly and less accessible. It does. It absolutely does. There is also no alternative except to see higher education get wiped out, which would have devastating economic and social effects.

So we are going back to blue books and vivas.

"So we are going back to blue books..." exactly. And, it is also efficient. You just need a proctor to ID the test taker and make sure they aren't using AI to cheat in order to verify that the student learned something which seems like a good idea.

> Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective?

Universities are entirely elective, why would they need to cater specifically to disabilities? Driving schools don't make exceptions for the blind. There are skills that people expect you to have with a certain kind of diploma, and it reflects poorly on the school if they don't have a standard to hold people to. Not being able to present your work in a basic way is one of those mandatory skills that imo makes you unqualified to graduate if you don't grasp it.

I'm not saying you can't make an exception for a person or two who is clearly as good or better than the rest otherwise, but people will always try to misuse as many exceptions as they can, en masse, as an excuse to just avoid doing required work.

I feel this was already a severe problem pre-AI. From what I hear from my partner (in university administration), nearly half the students demand accomodations for disabilities. Many of those conditions are very difficult to falsify

It doesn't feel good, but at some point we have to ask, can we support every single case, at the expense of the whole? I'd like to be convinced that's not a necessary tradeoff

I don't think it's that bad yet, but it's certainly on its way there. We're like at this intersection of lots of conditions that previously weren't documented becoming actual diagnoses, that pulling along a lot of peer-pressure kind of trendiness to have some kind of fault that makes you more unique, and the practicality of a system that was designed to give massive benefits to rare exceptions, for which now so many easily qualify on paper.

The list of conditions that qualify for what will most likely have to be redone completely from scratch.

What I don't get is why we disadvantage everyone who's honest. Just give everyone extra time to level the playing field.

If you give it to everyone, it's not "extra", is it? It's just a longer exam.

That's precisely the point. Stop giving people unfair advantages.

Well, the prevailing wisdom is that these accommodations improve fairness. To confront that way of seeing things is an uphill battle in the current intellectual environment.

I know it's a losing battle. The way I see it is if you're not going to test performance under pressure, give everyone as much time as they want (within reason). If you want a fair stress test, don't discriminate and give special privileges.

Well, they don't see it as privileges, but accommodations for a health condition. The measurement method needs to be adjusted when the health condition is present, to make the same kind of determination about the acquired knowledge. Experts have likely written expert publications about this being like this and hence it is The Science.

Even in fairly recent times viva voce was common. I had to defend my final year report on a long piece of practical work as part of the final for my BSc in Applied Physics in 1975.

You don't need to do everything orally. A return to handwritten final exams done without electronic aids would help too. They can be open note as mine were in the mid 70s or even open book.

As for scalability, I really don't see the problem; if we are producing so many qualified people then we surely can use some of them to examine the next generation. It sounds like the argument that often appears on the web where someone, usually an American, claims that a solution that works in, say the UK, can't work in their country simply because their population is larger.

Also, we really should consider if shoe horning every young person into an academic institution is useful either to society or to the person concerned. It certainly has not been an unmixed success in the UK.

Oral exams are --- at least partial -- a thing since "ever" in the german school system. Details matter, over here schooling belongs to culture, and the culture sector is in the hands of the 16 federal german states, so things vary a bit from e.g. Hessen to Bayern to Sachsen.

* you are in school and your marks are between two marks, e.g. between 1 and 2. You can voluntarily take the oral exam to swing the tide towards the 1 ... this is even a case for apprenticeship exams, not just normal school. * you are at the risk of not getting into the next class year. Often here the oral exam is mandatory.

Personally, I think this is great. Some people are better orally. But some have trouble, e.g. social anxieties. And some people are better theoretical, with a piece of paper. But other struggle with an "empty white paper". And since we don't want to test the difficulties of people, but their domain knowledge of e.g. mathematics, politics or electronic scheme design (again: think that this is also for apprenticeships), that we do both is IMHO a good thing.

I graduated in 2000 and remember doing the oral defence. It wasn't a huge deal. You already have a personal relationship with your tutors and they are familiar with you and your communication style. They are not the Gestapo.

It’s a good answer for keeping online education honest. It doesn’t need to be applied to all in-person schools, where teachers distribute spot checks across the course relatively evenly, but it makes perfect sense in conjunction with online testing. You only have to challenge one essay per student per term to catch cheaters with a near-perfect success rate. Assuming a 12 week course with 36 students, that’s qty.3-4 of teacher:student one on one meetings per week for the term; I think that’s doable. If you’ve oversold your teachers and have more than 36 students per, guess you’ll have to increase your labor expenses :D

This is basically my thought- my school is considering all sorts of things to combat AI cheating, and most of them scale poorly. You can implement them and increase your reputation, but your tuition will have to increase to pay for a lower student/teacher ratio.

So we’re probably going to see a split- mass production schools that still act as job training for Industrial Revolution era jobs will keep doing what they are doing. Schools that want to keep their academic prestige will implement these inefficient safeguards and raise prices accordingly. Employers will distinguish between these two types of diplomas when there are enough options out there that are combatting cheating- much like in-person schools are generally valued higher than online educations, or how certain schools (MIT, Stanford) are valued higher then others. This might lead us to the transformation of higher education many have been anticipating, because the current system cannot remain as-is. We all thought technology was the thing causing the upheaval but AI has really accelerated things.

I'm quite skeptical that industrial-age measures of efficiency are the right way to think about education. Feinman's story about teaching in Brazil comes to mind: https://enlightenedidiot.net/random/feynman-on-brazilian-edu...

And that's before we get to technological changes making that sort of knowledge and those skills much less valuable.

As an aside, accessibility cuts both ways. As a person with ADHD I'd much rather do oral and practical exams.

Thanks for sharing that story. A lot of that resonated with me, having come from a similar background in education.

> A written paper could be handed in and graded without lining everyone up in front of a panel

It can also be graded blindly. That allows for removing–or at least mitigating–biases in evaluation.

And what a better time to set optimization aside when schools and universities are getting less and less students each semester due to population decline.

Interestingly, you dont consider your 'inverse' point: that now the manual part of teaching the stuff can (and definitely will) be automated with LLMs.

It will become similar to coding: humans evaluate, LLMs just provide what has been mechanical for a couple hundred years

It shouldn't take long for educators suss out whether students know the subject they are talking about. You don't have to question them exhaustively to determine that they likely did or didn't do the work they are claiming. TFA is about high school projects, not PhD defenses.

But even with the written exam, wouldn't the bottleneck still be there where the professors still need to read through all the papers? I guess its definitely more efficient and asynchronous, but I can't imagine by much assuming the same and equal rigor is applied.

Grading written exams is delegated to teaching assistants / PhD students.

Not everywhere.

You can be certain many instructors are having AI do a lot of their grading.

Complete bullshit.

Universities in the US charge fortunes and pay instructors a pittance while plowing ever more money into “administrators”. Fire half the administrators and hire more instructors to both proctor exams and give more personalized instruction.

I don't think the world really works like that. Hearing an elevator pitch has a very different impact from reading a short-form business proposal. And of course the efficiency of the written word is not absolute; in lively discussions and debates, there is no time to put thoughts down into the written word; they had to be uttered. I am personally of the opinion that a successful university graduate should not only be able to write down their ideas in words but also be able to speak to defend their ideas.

"The oral system scales poorly. One examiner can only hear so many students in a day."

I'm not joking: you could have an AI quiz the students and test their understanding. That way the tests go deeper into testing the understanding by following through on the answers, and are standardized and fair for everyone. I don't think we are there yet at all but I actually think its not insane for the future. I expect downvotes though/I don't think our generation would agree with this.

What's old is new :)