Hacker News Reader

Comment by ttonelli | Hacker News Reader

Maybe making questions that you have to think about instead of just looking up the answer would help?

You need to give a range of questions from challenging like that to easy regurgitation to discern their level of understanding and be able to assign them an appropriate grade.

Why do you need a grade at all? Fail, pass, distinction. You either know the answers because you know and understand all the material, or you have no business functioning in that field period.

More granularity is still valuable though, since in topics where deep understanding is needed, the shallower levels of understanding are required to master the deeper. E.g., you need to be able recognize an integral before you can solve them. Which in turn you need to be able to do in canned rote mechanical cases before you can handle general word problems that involve assembling and solving integrals. And not all subsequent classes or careers will necessarily require the full mastery of the topics in a class. Plus it's good to have such detailed feedback for both the student and the professor to focus on shortcomings.

Another issue with the purportedly "optimal" testing situation of creating unique challenging and deep questions for each test (and even for each student). It's difficult to make good questions. Sometimes they are too hard, are not as clear as they seem since students might misunderstand and go off in the wrong direction giving you some trivial answer to a similar-looking question. The safest questions are the most shallow. And deeper problem-solving questions may take a few iterations of "testing" (as in trying them out on exams and seeing how students do). So what seems like the best kind of question will actually be the riskiest and least-tested.

Isn't that how good tests should be written anyway?

It's like having unit tests and integration tests in software testing.

Depends on the subject. In a long form essay where you need to cite sources or technical details you could make a big difference to your score by quickly looking these up. Nobody can go into the exam blind but they can improve their answers significantly by looking up the details that others have to spend a long time trying to remember. For example in an English exam, direct quotations from literature. Or in a law exam, case names.

But why should you not be allowed to look them up?

Isn't that just testing for memory rather than comprehension?

Beats me. A well designed exam asks one to apply knowledge not to regurgitate it. You should be able to look up any of the supporting information but you also need to be able to organize that information.

All of my BSc Applied Physics exams including the final (Exeter Uni. 1977) were open note and this really helped sort the competent students from those who thought that they were at university to learn facts. The latter brought fifty litre rucksacks full of notes in to the finals and without exception either failed entirely or scraped through to third class honours.

They seemed to have not realized that the faculty would not set questions that could be directly answered by looking up the question in one's notes. The best example was the final quantum mechanics paper which did not contain a single question that had a direct answer in the notes we had taken or the textbooks we had used.

My most recent exam was a C# course and that exam was open book. It wasn't as extreme as my QM final but had basically the same idea: apply what you have learnt.

> Or in a law exam, case names.

I want my lawyer to double check the sources. Even if he has the case name memorized, joe vs sally is easy to get confused with joe vs sue (or sally vs joe...), and those details are important to get right, lest I lose the case on a technicality.

There is a time to memorize details, but for the most part I want people who verify before making a statement.

It sounds like the root problem is that your exam grade is boosted more by a direct quote or case name more than by your comprehension of the message from the quote or by the case name rather than the precedent established by the case.

It should not matter if you have the exact line "To be or not to be" verbatim if you can explain what Shakespeare tried to convey about suffering and the meaning of life. More importantly, you should be allowed to have the book! You can't expect to jump from confusion to comprehension by reading the whole thing during the exam.

It shouldn't matter if you can regurgitate "Miranda v. Arizona, 1966" or not if you can explain when a person is giving voluntary statements or is under custodial interrogation. Access to the full PACER database wouldn't enable you to understand those distinctions.

And as comes up here all the time, it shouldn't matter in a whiteboard interview whether you have the arguments reversed in your strcmp(), an editor, compiler, or reference (or just running the code) would show you which way it was supposed to be. The important thing is the planning, algorithm tradeoffs, data structures, and debugging process that you used to write the thing. Heck, it shouldn't even matter if you can remember that the function is called 'strcmp'.

Reliance on these factoids usually indicates that the grader is taking the easy way out - scanning for expected quotes, for case names, or compiling the code and seeing if it works or not - rather than gauging comprehension. I'll grant that it is far easier to do the former, but that's not what the goal is supposed to be.

Edit: What can't be allowed is regurgitating someone else's comprehension of the question while having none of your own. If that's a possibility, you need to either ask different questions or have some way to prevent people from accessing that explanation. Remote exams need more of the former and less of the latter.

Reply on news.ycombinator.com