Hacker News Reader

Comment by Aperocky | Hacker News Reader

The traditional methods of measuring competence via a closed-book, fixed time test is being rapidly dated.

Practical knowledge and rote memorization have held mostly equal status, since out in the field both used to be inaccessible. Our tests are still structured around this, however the reality has moved on and rote memorization is now easily a search away.

I like to see more tests moving to open-book model where test takers are allowed to use all resources, as in work. Of course, this will be harder to administer because schools can't just reuse tests over and over again since they're bound to show up in the internet. Overall, more and more of evaluations will move from testing altogether into some project based evaluation, which I believe is superior in terms of accessing skill.

Let's see how long the old format hangs on.

View on HN

> I like to see more tests moving to open-book model where test takers are allowed to use all resources, as in work.

Open-book model does not mean you're entitled to all resources, including paying someone to solve all the questions for you, or mooching answers off your peers. Cheating isn't limited to closed book exams, and to that extent, it's still an unsolved problem.

Cheating is a separate problem in this case.

Moving to open book or project model isn't to eliminate cheating (and can't), it's to adapt tests and grading to modern reality.

Bonus if it makes cheating harder.

Being able to search for an answer is not the same thing as knowing the answer, and it does not contribute to being able to synthesize ideas from observation combined with knowledge.

Otherwise I don't disagree with you that this is where things are heading.

In a few more years we probably won't even be giving traditional grades in school anymore. What would they mean, other than that the student has enough Google prowess to look up the answers?

New degree: B.Goo.

Well, typically open book tests are designed with that ability to search in mind, so the problems are much harder and designed to test understanding rather than memorization. They often are also much longer, so you cannot just search for everything and still get a full score.

Agreed. The value is in being able to know _what_ to search in conjunction with knowing _how_ to compose the amalgamated resources together to derive a solution to the question at hand -- all in a timely manner.

Simply knowing a generic fact (e.g. "the Civil War began in 1861" or "the atomic mass of Oxygen is 15.999u") is not sufficient. One needs to know how to use that information.

I graduated from a liberal arts college in 1986, math + physics major. In virtually all of the courses that I took, rote memorization by itself was of limited usefulness on the exams -- certainly not sufficient to get a good grade. Many years later I taught a couple of college courses, and noticed that the students who focused on memorization, also had the weakest grasp of the subject matter, and often did the worst on the exams.

My own grades had their ups and downs for various reasons, but when I finally hit my stride, it was with the realization that sufficient memorization would occur as a natural consequence of the work needed to understand the subject matter.

Being able to search for information only gets you so far. Imagine reviewing a design or proposal, or engaging in a technical discussion, if you have no domain knowledge. Have you ever seen a non technical manager try to bluff their way through a technical meeting?

You mention "as in work", but let's be clear here: "looking up how to do it" is not really a profession. Professionals are the ones who could write the stuff the former are searching for, explain why it works, and have the good judgment to choose a good solution over a poor one.

I am in full agreement with you, the tests as conducted in favor of rote memorization would be rendered useless in this regard as all the answer would be behind a simple search and the student would have learned nothing.

Therefore the test must change to be more engaging and more in depth, or be replaced with a final project.

Even exceptionally smart people will often write guides and explanations at their peak, only to come back to reference them later.

I've encountered cases of people looking up questions on StackOverflow for which they themselves wrote the answer many years ago.

Of course, and nothing I wrote above presumes or implies otherwise. True professionals make frequent and effective searches, but not from a position deficient in knowledge and understanding of the domain.

Law exams are open book; Engineering exams are not.

I'm not sure exactly what the distinction is - apart from law being "soft" and requiring argument - but it's deeper than inability to reuse past papers.

Lots of engineering exams are open book. The only closed book ones are the freshman & some sophomore weeder classes.

Thanks - looks like this has already changed in line with GGP since my time. https://peer.asee.org/open-book-problem-solving-in-engineeri... [pdf]

I'm glad they were actually, it allowed us to focus on learning the material instead of copying all the formulas from the textbook onto a notes sheet.

This depends very much on the job. Memorization is of course needed when learning a foreign language, and for any job with a real-time performance aspect, you can't be looking stuff up all the time.

In classes where I had open book tests, the value of the notes was always mitigated by the time allowed for the test. It was always made clear to us that while you could use any resources you wanted, you definitely didn’t want to do that. You just didn’t have time. But if you needed to look up a particular chemical structure (example), it was possible. But you had to really know the material in the first place.

In some intro classes like accounting textbook access makes exams WAY too easy. Open book could work in a hard physics or math class though.

I don't know... I can't look up the circle of fifths if I'm in the middle of a gig and the singer wants to change key.

Reply on news.ycombinator.com