Hacker News Reader

Comment by nonrandomstring | Hacker News Reader

We are in an arms race now. Turnitin. Proctoring. Next? Airport style security scanners at the exam hall. I wrote about this arms race recently [1] and where it will lead.

The problem is really that, under conditions of self-commodification (reification), intrinsic motivation to learn and be a better person is replaced by extrinsic motivation to appear to be a better person. The experience (simulation) means more than the reality - which is a general trend in Western society now proved by the very existence of the company Meta.

https://www.timeshighereducation.com/features/we-cant-teach-...

View on HN

Some courses seem to put far too much emphasis on the memorization aspect of the courses, which has always seemed silly to me. For so many jobs you are not required to memorize a ton of information and instead can refer to manuals, textbooks, the internet, etc... for those kinds of things. These are also typically the things people forget shortly after the exam, which begs the question of how useful it even is in the first place. For example, I tried out an "Intro to sociology" class. It was a horrible class, and over half the exams questions were things like "Which person came up with xyz theory". Or "What year was xyz theory proposed". So many questions were purely memorization and did little to actually test your understanding of those theories.

What I remember out of most courses is not the things I had to memorize, but instead it is the practical components. Those should be the takeaways and things that get tested. Many people also just sucked at memorizing things like that, and I can totally see why cheating is such a problem on these kinds of tests.

Some of the bests tests I've taken where I've remembered the most material have been open book tests that allowed me to use all my textbooks and notes that I had taken. They were hard tests which really tested the understanding of the course material.

Intro classes (i.e. 101 or just 100-level classes) are designed to weed out people or fill cross-class requirements for majors. My degree required some "aesthetic" classes so I took Dance 101; the class was comprised of a diversity of majors + athletes looking for an easy course. It's also a -lot- of work to grade 150-200 students' tests, and they might change their major anyways.

Nearly every course I took at a 300/400+ level was about demonstrating a fundamental understanding instead of strict memorization. Open-book tests were more common. And it went down from 150+ students to 20 or 4. Considering my degree is a b.a. in media arts, some senior courses had minimal or no testing at all and instead had large projects.

That said, perhaps one of the hardest classes I took was a 300 or 400 level one on Tarantino films, probably to weed out people who were just looking for an easy class. On top of demonstrating understanding through analysis via storyboarding or papers, the quizzes/tests had multiple choice of 5 answers, which always included "all of the above" or "none of the above", and short/long answers/prompts. You really had to have studied each lecture and actually have watched the relevant films. You wouldn't believe how many people didn't watch the films... in a film analysis class. But all of that aside, it was thoroughly interesting and memorable.

Memorization of "raw" facts has become quite easy with spaced-repetition tools like Anki. The best performance on the job always comes from putting together raw memory and deep conceptual understanding. Neither of these is useful without the other.

I think there needs to be acknowledgement that there are two different learning tracks in higher education - one that is facts & figures, and one that is experiential. For better or worse, most institutions treat success in facts & figures as a high quality proxy indicating future success in the experiential areas, which includes everything from how an individual knows how to learn & how to teach themselves, to how they effectively build relationships and collaborate with others. Speaking as someone both liberal arts & technical degrees, and technical & business leadership experience over 20 years in manufacturing & big tech, my assessment is that this is ridiculous. Of course there will always be a need for "human computers" -- people who specialize in functions requiring programmatic, fact-based work -- but most jobs are not like this, and most individuals who prefer this type of work would generally benefit from expanding their comfort and capacity in human-centric skills.

To this end, I agree vigorously that the current state of higher education is both unsustainable and insane, but I think the end game will be more private enterprise & public sector employers deciding to expand intern & apprenticeship programs as an alternative to being constrained in hiring to the pipeline of candidates emerging from top universities. We'll see.

I agree, I think University in it's current form has been more about being able to efficiently cram as many students into programs as possible and keep pumping out the degrees. The goal no longer seems to be too focused on the education itself. I so often am seeing my local universities take people by the boatload in areas that have very little jobs. 100's will graduate with degrees that barely help them and have almost no jobs available to them. What they were taught is so fact/figure based that there is little practical skills they can apply to real life. So many people I find will go and take courses at the local colleges after to make themselves actually hirable. In Canada I believe our "colleges" are equivalent to community colleges/trade schools in America just for context.

I did a minor in psychology and so many classmates went into things like a 2 year program in HR administration at the local college. Because their degree simply led nowhere. Unless they wanted to continue and get into some highly competitive masters programs there was little in the career prospects.

And then I find Universities in general have been suffering from a lot of admin bloat.

I got off on a bit of a tangent here, but overall I think Universities have been ending up going in the wrong direction. Online learning has made the problem of raw memorization for testing purposes an even more prevalent issue. And instead choosing to test using different/better methods, educational institutions would rather opt to keep using raw memorization while adding privacy invasive programs to monitor students.

Students cheated with phones and hidden small speakers at exams that are not about reproducing stuff from memory, the student would tell the friend on the other side the question and somehow the friend would find or know the correct answer.

I can understand one motivation to cheat: exam grading is very unfair at the individual level because your performance has a high variation depending on factors outside of your control. Those factors are random from the perspective of a test taker: the exact questions, whether you had a flu, family circumstances, and hundreds of other factors that are independent of your ability. Apart from the fact that exams strongly measure the skill of passing exams, yet often poorly measure actual ability for the subject.

The unfairness depends on how steep the cutoff is: does 89.9% mean you miss out on an important life goal, whereas 90.1% means you win?

I did a quick google to find facts on expected individual exam mark volatility, but couldn’t find anything - what keywords do I need?

One pattern to the results I did read is that individual volatility is not even acknowledged - the unwritten assumption is that exams are completely fair and volatility has other dominant causes.

Edit: I am an engineer type with some spectrum attributes and I loath cheats, but over time I have seen how important some “dark” skills are such as: deception, judging when to ignore rules, and meta-games. In some roles or countries, perhaps cheating is a good quality? There is a reason the best card games are about deception. Cheaters are also risk takers, and taking appropriate risks is associated with entrepreneurship. Risks with extreme downsides are interesting.

Based on my experience teaching I don't think individual variability is that large. In a semester-long course with 3 or 4 exams the top students and bottom students are mostly consistent from exam to exam.

That said, of course individual variability does exist. In one graduate-level math class I misremembered one theorem which caused me to immediately lose 40/100 points--there were only 5 questions on the exam, and that theorem was central to two of them. Had those two questions been different I probably would have scored much better. I consider that a poorly-written exam, though, since so much of the score was dependent on recalling one theorem correctly. I somehow still ended up with an A in that course, I can only assume through either creative accounting or a generous grading curve.

In any case, I don't know any other method of evaluation which is more fair than exams. Every method of evaluation is subject to similar sources of individual variability and some have other issues in addition. "Fair" in such cases is a mirage. It's not like there's some other objective evaluation that we can use instead of exams, just ones with different biases.

It seems to me that TurnItIn and proctoring take care of 99% of problems in exams if the exams are well-constructed such that consulting hidden notes wouldn't help much.

The 1% of exams that remain problematic are the memorization-style exams that maybe shouldn't still exist in the modern age.

Right. there are plenty of exams in tech that require plain old practice and on-the-spot ingenuity. However, for medical and other fields, memorization and recall is key, so it's rife for this kind of cheating.

But honestly, if someone were to tattoo the same information onto their arm, there's an argument to be made it was permanently accessible just as much as memory.

I mean it is kind of stupid even for medical. You are never in the middle of the desert having the memorize the mechanical properties of the inner ear. Every doctor there is has reference books in reality. Why not let people just bring those books to their exams if that's how the job actually works in reality? Learning how to consult a reference for information is just another mental offshoring tool like a calculator, so its a little silly when exams force you to work without it.

To an extent, yes. As a programmer who can't remember the syntax for declaring a two dimensional array and populating it with hard-coded strings, I feel somewhat hypocritical when I say that Medicine seems to be valid domain for rote memorization.

When was the last time someone’s life depended on you recalling the syntax for a two dimensional array within the next few seconds?

I think it’s OK to hold medical professionals to a higher standard.

How often do doctors even need to do this? Everything they do probably has a standard operating procedure printed out in a binder that can be dictated by a nurse. The training should be in learning how to take those instructions and turn it into action, not that as well as having to memorize all the instructions that are going to be referenceable anyhow.

Maybe if the training were more like the former than the latter, fewer med students and residents would be pulling all nighters on some amphetamine. Personally, going into an ER I'd rather be met with well slept, focused hands that consult the relevant information, rather than a sleep deprived zombie barking out protocol they hypnotized themselves into remembering.

From what I know, not often. But more often than a developer.

Sometimes memorization itself is important to a role. How would you test for that, if not an exam that tests memorized facts?

From recent personal experience: pilots need to know certain information so that it can be employed on a whim to help solve time-critical problems. The written exam (which precedes a practical exam) definitely requires memorization.

It's also worth thinking about why we use tests and what they actually measure.

Malcolm Gladwell (yes, I know) did a podcast[1] on the LSAT that's pretty good. In that podcast, he makes a pretty compelling argument that what the test measures isn't very useful.

[1]: https://www.pushkin.fm/episode/the-tortoise-and-the-hare/

The LSAT is probably one of the few tests where what it measures is directly useful to a prospective lawyer. I find it hard to believe that he can show that reading comprehension, logical ability and analytical ability aren't useful to law school admissions.

His criticism was mostly about the timed nature of it. The time restriction is a very artificial constraint and they end up filtering a lot of potentially great people.

> We are in an arms race now. Turnitin. Proctoring. Next? Airport style security scanners at the exam hall. I wrote about this arms race recently [1] and where it will lead.

This is completely self-inflicted. When the map becomes the territory and exams are lazily written to reward rote, you are no longer testing for reasoning and understanding.

See Caltech as the exact counterpoint with its culture of take home exams and honesty.

Not the full body scanner used by the America's TSA, but the TOEIC testing centers in my country already require candidates to pass through a metal detector when entering the testing room.

Reply on news.ycombinator.com