Hacker News Reader

Comment by rdlecler1 | Hacker News Reader

I was a TA at Yale when I did my PhD there and I contrast that experience to when I was a TA at University of Calgary. Whether it’s a higher sense of entitlement or just grinding it out to maximize your official grade the contrast between these two schools was striking. At Yale you’d better think twice about honest grading because if you did there would be an army at your door with pitchforks ready. You then have to spend hours explaining why they didn’t deserve a higher grade. At the end of the day you’re worn down and there’s very little incentive to grade honestly as you get pain nearly nothing and you’re really there to finish your PhD.

View on HN

The problem is when the grades given on assignments influence what goes on your final exam papers.

At a university, only the final exam should count. This way you can give honest feedback, and the student doesn't feel obligated to point out every little mistake to try to make you change your decision. It also allows them to misunderstand things at first, but then 'get' it later.

Of course universities don't like this, since they don't consider students mature enough that they do the homework if it doesn't "count".

Really? Spreading grades out among various assignments and tests is how I managed to pass university (I coincidentally also attended the University of Calgary, hi rdlecler1!). I would have imploded if it all came down to a single giant exam. Why the heck would you want academic achievement to come down to a handful of hours of actual performance? What goal does that serve? University isn't the Olympics.

The true answer is: No grades should matter except the final exam and you should be able to take the final exam as many times as you want. (With a reasonable minimum period between retakes, possibly increasing each time.)

If you take AC motors, and fail, and study your ass off and six months later can pass it with 100%, shouldn't that really be... your actual grade? Isn't a grade supposed to represent your ability and understanding of a topic?

And then almost magically: health problems, study problems, (most) test anxiety, and other problems no longer actually matter.

>Isn't a grade supposed to represent your ability and understanding of a topic?

University education largely isn't about certifying that certain students know certain things. It's about showing that people who possess the degree meet a minimum IQ + trait conscientiousness cutoff, so that employers can preferentially hire those people.

Grades have several purposes in this framework. First is to drop low-employee-quality students so that the signal for the remainder is stronger. Second is to allow honors and GPA to provide multiple cutoff-levels to fine-tune the signaling. Third is to force re-taking of classes to extract more money out of folks who want to substitute it for IQ + effort.

As it stands, your proposed change would simply increase the amount of work that is allowed to be used as a bid for class position. That's not particularly helpful, IMO.

A grade is supposed to represent your ability and understanding of a topic, but that's only half the equation. Are exams an adequate proxy for ability and understanding of a topic? They aren't for me. When I want to understand something, I create something which requires and demonstrates that understanding - this could be a Wikipedia article, a piece of software, or a lecture. Nothing of value is demonstrated by sitting down for a few hours under extreme pressure and regurgitating my "understanding".

I've never understood how someone could master a topic yet could not answer questions about it. I know from my own personal experience that when I bombed an exam, it's because I simply did not understand the material. When I did well on an exam, I had corresponding confidence that I understood it.

It is the difference between knowing "given a task in this knowledge area, I can accomplish it" and uselessly loading up on all that information to perform in a purely artificial situation the likes of which is never seen again.

So, good preparation for how interviews work, then?

Conversely, I was able to pass (with As) some physics exams without really understanding. That became painfully obvious in my solid-state physics lab class.

Memorizing a bunch of facts doesn’t demonstrate understanding of a topic.

That's a good point. Sometimes you need large, ambitious projects to both learn and demonstrate that learning. People learn to code well by working on big programs, and learn to write well by writing in depth on some subject.

I guess you could sum it up as: the best test of whether a person is capable of creating something of value is to ask them to create something of value and see what happens.

Taking the exam as many times as you want is a bit unfair to the professor, who has to concoct a new exam each time. If it is the same exam taken multiple times, then it is what is known as an "infinite time" exam which has a deleterious effect on students, as they will be more or less forced to spend infinite time on them.

Besides, taking a month to solve a single calculus problem that should take 15 minutes is a pretty good demonstration that you have not mastered the material.

How about periodic testing, once every 6 months? Let previous classes retake a final. That's basically the same test load as before. (Another variation could be only giving partial credit. But it's going to be fairly clear who is trying to succeed and who is trying to game the system. Hence my suggestion of increasing lengths between subsequent testing.)

Also, students are rampantly cheating by leaking previous tests. I saw it every day on campus. TA's leak tests to students and they're printed out before every exam (as the risk is too high to NOT look at a test in advance because your grades = your future in college). I've literally been in classes where everyone had last semesters final printed out... in the room before we took the current final. So the impetus is already currently on you, as a teacher, to have tests with permutations to prevent cheating.

Additionally--while I'm sure many teachers would object--tests shouldn't necessarily be written by the teacher. The teacher isn't teaching their own class, but a class meeting the departments requirements. So if students are being prepared (and not learning very specific permutations of general topics) they should be able to solve problems written by other teachers / test makers. So (on the bright side), teachers shouldn't actually be burdened at all by the testing. That could be someone else's job.

Now, of course, all of this has plenty of practical considerations that would need addressed. How do you permutate a QUALITATIVE question? And how do you easily grade highly permuted (per-person) tests? But just because there are unsolved issues, doesn't mean the underlying (god forgive me for saying using this term:) "paradigm shift" is unsound. It's just different and requires some new solutions to new issues.

But as programmers, we of all people, should be able to figure out how to further automate testing. But I think we really should decouple the idea of "testing" from "occurs at the end of class." I mean, why do we test all finals at the same time during the super-stressful "finals week"? Simply because it's practical. Except it was designed as "practical" before computers and automation were within the hands of every living person. Surely, students would test higher if they could study for a single class's test at a time and without worrying about it occurring during a family or health episode.

And what's a test for except hopefully proving that a student understands the material--but we all know thanks to "teaching to the test" culture that many students can cram for a test and then forget it within a month. But now I'm getting way off the original topic.

My thesis statement is simply: Re-testing finals is practical in modern environments and beneficial and incurs (almost) no additional flaws that the current setup doesn't also exhibit--like cheating.

Professors do need to permute the tests so that people won't just look up last year's final. But permuting them once a year is a very different proposition than permuting them an endless number of times as students demand to retake the test.

There will always be cheaters. That's why companies, as part of the interview process, ask basic knowledge questions. It weeds out a lot of the cheaters and frauds. I hear people all the time desperately try to justify cheating. I don't care if people cheat, but I wouldn't hire any of them, and I doubt many employers would.

Sure, there'll always be bad tests and bad teachers. If one could be "taught to the test" then it's a bad test. It can certainly be true that one could pass the test and yet fail to understand the material, but I'm not convinced that people can master the material yet flunk the test.

But testing, in general, works. For me, I know that the pressure of final exams would motivate me to learn the material. No pressure, no learning.

40 years after I graduated, I'd surely flunk every final if I retook them cold today. I just finished taking the MIT 18.01 calculus class online. I was surprised at how easily the stuff came flooding back into my head. It's still there, I just had to chip the rust off :-)

That's a strong counterargument. But I strongly believe homework shouldn't be graded. Incentives are completely misaligned; you're penalized for getting things wrong, when that's a natural result during the learning process. No surprise, then, when cheating becomes a major problem.

I personally found doing problem sets and being able to check my answers immediately to be the best way to learn. There's no point to just copy the answers if homework is ungraded. If students feel no motivation to do homework to learn and practice, that's on them, especially at the college level.

Except when it comes to writing programs, the homework is the only thing that even remotely demonstrates your utility to an employer. I would go as far as to say that exams should not count, only homework should.

What? What employer has ever looked at a potential hire's homework?

If you don't grade exams but grade homework, then homework just becomes a take-home exam...

I'm teaching an intro Web Development course at the University level right now. No exams, only projects and homework. All submissions in github, so it's likely some will be looked at!

I didn't suggest that an employer looked at homework. But now that you mention it, I have personally done that when it was made available on github.

Why, preparing for an interview.

At Caltech, the only thing that counted for your grade was the midterm and the final. The homework only mattered if a grade was on the knife edge of passing or failing.

Students were free to do the homework or not, collaborate on it or not, as they saw fit. It was regarded as a learning tool that students could decide how best to make use of for themselves.

I thought it worked rather well, although the high stakes final exams were rather stressful :-)

(Attending lectures was also optional. Institute policy was that attendance would not be taken, and would not be a factor in the grade. You were free to attend lectures or not, do the homework or not, and if you got an A on the final, you got an A for the course. Some students could do that, such as the legendary Hal Finney, but I learned the hard way that I could not afford to miss any lectures and must diligently do and understand all the homework on time.)

Another point here that was true during my time: almost all exams were take home, open book. Some were even unlimited time, although that was much rarer. Generally though, time was not the constraint keeping you from completing the exam.

The problem is that some students are terrible at exams, and exams aren't necessarily a good test of knowledge or ability.

Engineering relies on group projects and assignments a lot for your grade, because engineering isn't just about remembering some facts and writing them on paper. Then there's the problem that people have good days and bad days.

Different people are more capable in different areas of assessment, so I think that the most fair way to assess students is to give them a range of different types of assessment, so that even if they struggle with one type, they can pull through the other.

For instance, I always did really well at large assignments and tasks, especially group assignments (not because I rode on coattails, but because being in a group motivates you to do well), I was mediocre at exams, and I was terrible at small, individual assignments (<5% of final grade). You can see a direct correlation in my transcript between papers with lots of graded homework and small tasks, and poor grades. I did 200-level Algorithms the semester before I took 100-level Algorithms, and managed to get a better grade in the 200 level paper, because it had less homework assignments, despite being the second year version of the same paper.

My experience was that the top students (now PhD students at MIT, Stanford, etc) were very diligent about homework. In the honors classes, you often needed to go far above and beyond the textbook for some questions. The average students (still smart kids, many working at Google, Amazon, etc) were much more lax about it, and wouldn't work that hard to solve the hardest problems in a set.

What worked for me (Caltech) was to make sure I understood 100% of all the homework problems front to back, before going in to the exam.

Living in the dorms helped enormously with this, because I could usually find someone to help me with particular problems, and I sorely needed that help.

Anything less meant I was in deep doodoo on the exam :-) because that's the wrong time to try to figure out how to apply what was learned in lecture.

This was good enough to reliably get a B. To get an A, I had to work much harder and go beyond the homework.

The school where I studied CS as an undergrad didn't even have grades. It was all about the narrative evaluation, which considered only things the student actually accomplished or learned. So a typical narrative evaluation in the sciences would have a sentence for each of the course topics that a student demonstrated some knowledge of somehow at any point in the course, like a discrete math course might have "He showed an excellent understanding of number theory proofs in the coursework and final exam."

If you really screwed it up, your evaluation wouldn't say anything bad, but it would be damned with faint praise and they could reduce your academic credits.

Overall it's a much higher resolution picture of a student's education. I transferred from a school with traditional grades and I prefer the narrative system. It really stressed out a lot of students who were too indoctrinated in the traditional game process of getting the right number of points for their target grade; some of them never quite figured out that they should just focus on actually learning the subject.

This is how it works in Germany. One exam at the end of the semester. Nothing else counts. Studying and finding motivation to study is the students responsibility.

It's not perfect either, a bad performance on exam day might mean a bad grade even if you're good, but whatever, nobody cares about grades, anyway.

Not universally. Midterm exams, graded (regular) homework, graded projects, graded written reports, take-home exams, ... all exist.

How does this solve the problem?

Do you honestly think students will not complain about the grade on a final exam and that the university will not continue to put pressure on students?

Not everyone performs the same way. Some students do badly under the pressure of testing, so having a variety of grades from a variety of types of assignments does help everyone do well.

That's strange, all my mathematics classes had ungraded homework. Anecdata of 1.

To add to that anecdata, most of my mathematics classes would at least have 1% homework assignments weekly. One of my papers had 25% as homework.

They weren't super hard, if you turned up to the tutorials, you could usually get the tutor to basically hand hold you through it, and if you had any friends doing the same paper, you could just copy theirs. In fact, in first year, we had a rota for homework, so only one person would do it each week.

That's right! You're paid nearly nothing so you represent cheap labor for the Yale. I guarantee you that somebody is getting paid a lot!

The problem with these so called "Higher" institutions is that the people fulfilling the actual function of the University don't actually get treated all that well. Everyone just walks all over them, including the people in charge. It has become a crazy lopsided system. These schools are basically becoming feel-good daycare centers for adult children of rich families

Research suggests 'honest grading' is a pipe dream anyway. The book 'A Drunkard's Walk: How Randomness Rules Our Lives' dedicates most of its final chapter to grading. Even people specifically trained to grade to a very strict standard were incapable of doing so when they knew almost anything at all about the student.

Did you get any different instructions for your grading at Yale? And doesn't such thing lead to large varying amounts of average grads per class?

No, but students will go around you to the prof and they get away with things that wouldn’t have worked at UofC. Mostly you end up asking yourself how much you’re willing to go to battle. No upside, lots of downside. Grades become more binary for anything subjective. Obviously multiple choice is much easier to deal with, but even then students will get exam re-writes from the profs.

I've actually seen similar things happen at public schools; I think students may also intuit that very little incentivizes profs to grade honestly: https://jakeseliger.com/2015/01/13/what-incentivizes-profess...

I believe average GPAs are much lower at Berkeley and UCLA than they are at ivies though, is that correct?

Yes when I went there but dependent on whether it was graduate or undergraduate and department. Undergraduate lower division CS classes were graded assuming a normal distribution (standard deviations from average). In my opinion, it was mostly used as a weeding out process. Graduate classes had much higher grades, mostly As, some A-s... few B+s. I heard that undergraduate Chemical Engineering had especially hard grading. It was supposedly hard to maintain a B average in that major and that major had students coming in with much higher average SAT scores and GPAs.

That's correct.

What a scam it all is. How tragic that you TAs are apparently complicit.

Edit: Thanks for the downvote but giving higher grades because it's more convenient than doing the right thing is... the wrong thing.

Painful that at some point it felt like your (and many others') best choice but it was still wrong flat out.

Hey maybe now you can "change the system from the inside!"

The system of elite schools is designed to perpetuate itself -- just like any beauracracy. The elite schools do this fairly well, imho.

Note that grading hard is basically friction in the system that is fairly efficient to avoid.

The elite schools have plenty of smart and hard-working students who don't need hard grading as a motivator to do great work. Those students are the backbone of your school.

Some additional questions after your edit:

1. How does one define the "wrong thing"? Genuine question.

2. What about soft grading makes it a scam? Since letter grades are distinctly not criterion referenced, I struggle to see a strong justification for this claim. Please help me understand.

3. What makes you think that anyone in the system wants it changed? I humbly suggest that very few people actually do -- the current system is super-low friction and allows professors to work on the things that they are actually evaluated on.

I'd say giving one a grade they didn't earn, because it is more convenient for you, is the wrong thing.

It's surely a breach of ethics to evaluate someone's work and report that they did better than you concluded they did for convenience. I'd call that ethical breach "the wrong thing."

OP said to think twice about "honest grading" so I think dishonest grading is the problem with what you are calling "soft grading."

I don't think anyone inside wants it changed, I don't think anyone ever does when a broken system benefits them more than those outside. That's why it was in quotes.

As you feel so strongly about this issue, I assume that you will not sit around and be complicit now that you are aware of the realities but instead will put in the hard work to marshall a campaign of change.

Should I have to be ready and willing to start a campaign every time I learn of something is unethical?

Am I not allowed to call out obviously unethical behavior without offering myself as tribute to correct it?

If I call you out for littering do I have to clean the whole planet?

Sorry this feels bad man, I don't think you are a dirtbag but I think you did a bad thing. Done plenty myself, my crosses to bear, but always turned out better when called out for it.

Cheating is unethical. Choosing to not aggressively grade because the costs (time away from your PhD work) exceeds the benefits is not. I put in what was expected I just didn’t put in more. My moral obligation was to my PhD work, not some grading crusade. No crosses to bear here.

Why are you obligated to "explain"?

Because one's job is not just to evaluate, but also to teach.

Can you go into more detail about the positives/negatives of your experiences at the University of Calgary?

Reply on news.ycombinator.com