Comment by advael | Hacker News Reader

Comment by advael | Hacker News Reader

Parent Full thread

My position on this has been pretty controversial when I've shared it before, but I still think it's correct:

Measuring knowledge at scale is futile, harmful, and pointless. The fact that a lot of society has been arranged around the fiction that this is a feasible endeavor does not mean it has borne out in practice, and prioritizing assessment in this way has been gradually hollowing out most forms of pedagogy of their value while building an ever-expanding series of increasingly meaningless hoops for people to jump through to get what they actually need. We have deemed it necessary to create assessments to prop up the idea that education can be easily measured and should gate meaningful life outcomes for most people. Most if not all "cheating" behavior is either just a rational, strategic response to this situation, or a disconnect between how people actually solve problems (e.g. often collaborative and laser-focused on the part of the problem that drives the outcome, in this case the assessment) and some weird cultist notion of what it means for an individual to do it "correctly".

Effective pedagogy will never scale unless we get some really AGI-like technologies (I loved The Diamond Age as a kid, but A Young Lady's Illustrated Primer is from the perspective of extant tech a total pie-in-the-sky fantasy, illustrative of how meaningful teaching requires individualized approaches), and we see time and time again that teacher-to-student ratios as well as particularly good specific teachers are overwhelmingly the drivers of even the stupid metrics we are optimizing for

In short, this whole system is broken because its fundamental premise is flawed

What you are saying is not at all controversial, but it is incomplete, which is probably why you have received pushback in the past. Criticising the existing system is easy. Giving an alternative is harder. Implementing that alternative and showing that it's actually better on some metric is MUCH harder than that. But you have not even given an alternative!

The alternative is to treat higher learning like any other experience in life or on your CV/resume: you do it, you tell people you did it, then you either convince them that doing it imparted something useful on you or you don’t.

As someone who’s hired plenty of people, exams and grades do not help one bit with the process and you shouldn’t pay any attention to them.

The only good use of exams I see see is as a potential entry gate, administered by the place you’re trying to impress, to get onto a course, be considered for a job, or be given a license to do something. As exit gates they’re just noise.

Let us consider the proposal of keeping higher learning the same except that we don't do exams. What would happen?

For better or worse, whether somebody completed their degree does often factor into hiring decisions, so exam grades do indirectly factor into hiring decisions. Having completed a degree signals some level of domain knowledge, conscientiousness, and intelligence. Without exams you would have a 100% success rate (unless you introduced some other assessment mechanism), so the signal would be gone; having completed a degree would only signal whether or not you were able to afford it financially.

Secondly, a large fraction of the students would lose motivation and not do anything by the second year. Many would stop doing homework, stop doing any real studying except maybe superficial reading, and many would hardly come to class. In fact, many students currently already do this until the first midterm, even though they know the midterm is coming. A lot of students need the existence of exams to motivate themselves. Not all students, but a lot. These students don't want to lose motivation and waste time; many would probably regret not learning anything for several years. Our monkey brains are not suited to motivating ourselves to do things with a >3 year time horizon. Exams are a mechanism to motivate our monkey brains to put effort into studying.

I think if we consider professions that are important to our own lives, we do recognise the necessity of assessment. Would you prefer your doctor or nurse, accountant, electrician, or for that matter, teacher of your kids, to have come from a school that doesn't do exams or from a school that does? If they are experienced, maybe it doesn't matter, but how many people are going to take a chance on a fresh graduate if they are from a program without any assessment, where the philosophy is "you do it, you tell people you did it, then you either convince them that doing it imparted something useful on you or you don’t"?

There are other ways to solve that problem, and some of them may even be better than the current system, but I'm not convinced that taking the current system and simply removing exams would work.

The evaluation isn’t necessarily the problem, but I think assigning grades may be. It’s gamifies education and I think generally makes things worse.

I think it might be worth spreading some of the standards from medical schools to other programs. Don’t assign a grade to a student, make everything pass/fail. Either you know the material or you do not. There’s no honor roll or deans list and no class rankings.

> Either you know the material or you do not.

This is simply not the case. How well the person knows the material is more than yes/no. You'd be rounding the exam result to 1 bit and throwing away the extra bits of information. Maybe it's a good thing not to show that information to the student, but I'd like to see an argument for why the advantages of hiding that information from the students outweigh the disadvantages.

As for competitiveness in education...there are advantages and disadvantages to it. I'm not convinced that class rankings are a good idea, but I'm also not convinced that the optimal target for competitiveness is zero, to the point of not showing students their grades beyond pass/fail. Anecdotally I've seen the aim for zero competitiveness have perverse effects, where students instead start to compete on how lazy they can (appear to) be while still getting a pass, to show how smart they are.

Some of the best medical schools in the world have adopted pass / fail. Either you are good enough to be a doctor or you are not. That makes a lot of sense to me.

Harvard is one example in the US and McMaster is an example in Canada. Neither place has students competing to be most lazy.

Are you aware that there are grade-free and exam-free schools out there and that they have been operating for decades?

Measuring the effectiveness of school systems is difficult because of selection bias, but I'm sure you could find some attempts (e.g. PISA) if you went looking.

You don't need more of an alternative to stop digging the hole deeper (subsidizing the problem, hard-requiring degrees awarded by the problem, etc.).

What part of "its fundamental premise is flawed" is unclear? I don't propose an alternative because I don't believe the stated goals of the system are achievable or desirable. Also, if one believes something does more harm than good, an argument to stop doing it does not require an alternative.

Exams are a bad way of learning… but a good way for a university to filter out slow down people that would devalue their degrees.

Reply on news.ycombinator.com