Hacker News Reader

I used to teach in academia. Student reviews are REALLY easy to game. Give everyone As for doing nothing, and most will give you As for just being entertaining. It doesn’t matter if they learn anything, or worse, if they learn the wrong things.

It is 2018, these patterns are _easy_ to detect. Honestly at a fundamental level bailing on optimizing teach performance and student outcomes through feedback loops seems nuts to me. This is how all high functioning dynamic systems work (ok maybe not all).

You just can't throw out the baby with the bathwater here just because there are things to figure out. Most of the reasons people have cited here seem like lame excuses, all easily addressable if you could get sane people to get aligned behind a well designed system. Unfortunately not everyone involved in these conversations is acting sane (or objectively in the best interests of the community) and therefore getting everyone aligned in practically impossible.

This whole conversation, BTW, reminds me a the similar debate around public access to doctor and surgeon outcome data where some doctors are amazing and others have horrendously bad success rates for specific surgeries but if you go to that doctor for that surgery you almost never are given access to that history. A large portion of the medical community is very hostile to the idea of things being otherwise which, IMO, is indefensible from a public health POV and only makes sense if you are a crappy, unscrupulous doctor seeking to avoid accountability.

the same thing happens with doctors when you make satisfaction/results stats public: surgeons, for example, are more likely to refuse to take on difficult cases, and prefer easy ones which will increase their patient satisfaction; or even the best surgeons get stuck with the hardest cases, which tanks their relative outcomes.

But why do you let the students grade the course after they've already gotten a grade? That's insane. In my university, you'd get a paper for the teacher and course review after handing in the final exam. This means you can comment on the quality of the exam, but you don't know your grade yet.

Theoretically, if you grade students several times throughout the course, you can correct those who are on course for a poor grade (due to misunderstanding the content, or misunderstanding how much studying they need to do) while there's still time for feedback to improve their grade.

(At my school many teachers were slow to mark and return assignments, so this benefit wasn't really achieved - but it might have been had feedback been more timely)

Then make your exams very easy so the students will be in a good mood when they fill the teacher and course reviews.

> Regarding professors wanting to keep review data tightly sealed: in my view, if you can't by public disclosure of your evaluation, then you either don't feel you're meeting expectations, or have no desire to improve in areas where students feel improvement could be made.

There is also the third option, which the author is arguing for: that the professors do not think the ratings are an accurate reflection of their teaching skills.

I'm sure many professors would agree.

Perhaps the engineering statistics professor I met who bragged that no one ever got an A on his final, because teaching wasa competition between him and the students. And the government professor I had who was inordinately proud of the fact that his course was required because three soldiers from Texas stayed in China after the Korean war and spent the rest of the first class going around the room having students introduce themselves and then mocking them. (I dropped the class the next day.) And a number who were just disorganized and incompetent, but protected by their relationship with other faculty. And the professor who was given tenure for political reasons, after threatening to fail an entire class (of a required course) of computer science undergrads because they weren't electrical engineering great students.

I bet the adjunct whose english was so bad that she would just answer a question kind of close to what you asked, as quickly as possible and try and move on would too.

She also answered any questions asked in Mandarin, in Mandarin. When asked to translate an exchange for the rest of the class, she blushed and said 'it was complicated'. At the end of the semester, the class was so far behind, 1/3 of the materials for the course exam were delivered during an optional study session.

And so would the electrophysiology professor who spent over an hour of a grad seminar explaining how to use a floppy disk. Because he had trouble with computers.

I've seen a lot of cases where the professors thought exactly that. Unfortunately they were in denial because their teaching skills were abysmal.

I've seen courses where the students were memorising MATLAB scripts by rote for exams because almost none of them had sufficient understanding to have any chance of recreating it in the exam. Why? Because rather than teaching this stuff, the lecturer spent several lectures explaining the details of the representation of floating point number (this was a first-year class for math majors, not people doing CS).

But it was hard as student representatives for us to prove any of this, because we didn't have access to feedback responses.

I don't understand how that's a third option because either the review data is open and publicly available, or it isn't.

That 'third option' is just an opinion about the data.

Like with healthcare recipients, students don't necessarily know what they need and therefore aren't able to accurately rate providers.

In some aspects yes, in others no.

I accidentally cut myself a little too deep while doing dishes a couple years ago (opaque, soapy water and sharp surfaces don't mix well). I couldn't get the bleeding to stop (without constant pressure held for about an hour), so I went to urgent care. I knew that I needed stitches or that "glue" they use to seal wounds. I didn't care which solution the Doctor picked, but I knew I needed something.

Students have some sense of what a generally good instructional experience looks like. It's better to collect this feedback, and have people that are experts in instruction examine the reviews, synthesize with their own knowledge of what makes instruction great, and use that information to improve.

But saying we should totally close out students since "they don't know what they need" removes a fruitful data source in determining where professors can improve.

If the doctor chose one of the sealing methods over the other which you preferred, you gave a lower score of because of it, it still is not an indication of the quality of the treatment you received. You wouldn't have been qualified to determine the best method.

When you review feedback collected (in any situation analogous to what you mention), you take that into account.

What if the feedback also said the doctor wasn't personable? Maybe the doc could be a bit warmer with his patients... and a patient would be absolutely qualified in determining whether or not that bedside manner is present.

Sadly, the only thing many students feel they need from a university education is a degree. This is not surprising, given our society's constant refrain of telling everyone, "You need a college degree to get a good job".

Many students are there not because they want to learn but because they want to get that piece of paper that is the ticket to a successful career. A teacher forcing them to learn, to spend time and effort studying and working, is not seen as a positive if your goal is simply to get a degree. The ideal teacher for them is someone who just give you an A no matter what. That would be the quickest and easiest way to guarantee achieving that goal.

This is the true issue here - the disconnect between what many students want from a university education and what that education is actually supposed to be.

Studies show that the pay differential for "all but completed degree" and "completed degree" are stark. Researchers have said that this indicates that a large fraction of the "value" of college is just signalling (intelligence, aptitude, diligence), not education.

If that's the case, many of the students in your post are right -- just giving them the degree could save them (and the school) a lot of time and money. Maybe your art history class is a cost-effective way to get a world class education in late Italian Romantic oil paintings, but as a way of proving you're smart enough to work in a law firm (or even "broaden your horizons and become a better citizen through rounded education") I can't imagine it's terribly efficient.

Well, there's a collective action problem there.

It's in every student's individual interests for their school's standards for admission and grading to be low, so they can obtain the credential easily.

But it's in the student body's _collective_ interests for the school's standards to be high, so the credential retains and improves its signalling value.

There's a reason a C- student at Harvard doesn't transfer to a deprived community college where they'd be at the top of every class :)

The thing is that most university grading systems have a very high rate of false negatives. For example, in the UK it is rarely possible to retake a course. And this means that messing up a single exam can cause grade problems, even if the material is well understood.

In my experience, exams are often full of ambiguous questions, questions testing knowledge that is not part of the course syllabus, etc.

If the grading was fair, I would agree with you, but it rarely is. And IMO this is what students really hate: they put in a hell of a lot of work. They actually get to grips with the material, and for stupid reasons out of their control they end up with no credit for it.

Like all end user feedback, what they claim they need isn't always going to be what they actually need, but the pain points they reveal can inform where improvement is necessary and valuable.

Sure, but that feedback which is directed at someone who is also a private person does not need to be made public.

Yeah, that's another angle I didn't touch on. You don't have to publish all the feedback data. You can choose just to goes with the quantitative results.

Comments are tricky since they're both qualitative and might need to be scrubbed for anything personally-identifying.

When they are paying the bills, they have a right to providing feedback on the product they are receiving. Students go into deep debt to pay schools, they are the customers and their feedback ought to matter.

I was a student representative on my course at one point. I strongly considered running my own evaluation (with the same questions as the official one), and organising a boycott of the official one because of this kind of bullshit.

How the university thinks they can claim ownership of data that come's from students I will never understand.

If anything you would think professors would welcome reviews. They already get reviewed anyways. It's just word of mouth. At least a student could see a much wider range of opinions vs one person's experience.

We made our own review system, to compete with the contracted-out system they used for their siloed data we didn't have access to.

We agreed to not launch the system, in exchange for limited access to the data, as I described above.

Professors en masse opposed opening up this data. That opposition alone made me feel we were doing good work. If they had to stand by their public reviews, hopefully they could stand by the quality of the instructional experience provided.

I ended up making a startup to deal with issues like this, to sell SaaS solutions directly to student governance groups, rather than to institutions. Both control fairly significant budgets (student governance groups at mid-to-large sized institutions have 6/7 figure budgets to easily afford enterprise pricing). My platforms are student-first. It's more a passion project than trying to get me Zuckerberg levels of wealth.

The things on word of mouth aren't things that the professor would consider anyways.

Personality, difficulty, ability to teach (relatively), understanding the student's pressures etc.

When I say difficulty, I mean easiness (people take the class to improve the gpa) or difficulty as in: the professor wants to make an example.

Student's pressures: Does the professor demand more than a reasonable amount of time from the student?

----

All of this being said: I haven't seen a pragmatically taught course before. (As in the professor looks at the course as a means and is the person that helps facilitate the student through it) I think that's what students really look for.

We know we get reviewed anyway.

What we're pushing back against is these reviews getting turned into yet another semi-useless, noisy and biased metric we're evaluated by.

You want to recommend for/against a class to your friends? Don't care. What I'm not interested in is my dean pretending there's deep insight there.

I can understand that perspective about being used as a metric to the dean, but as a student I would have loved access to that info. It's kind like any other review system, a couple of bad reviews among many high reviews, you take that into account. but a lot of bad reviews, I'm probably going to avoid that professor, restaurant, product, service etc.

It should not be used as precise metric of performance, but as a general trend overall. Instead of one friend giving a bad review for whatever reason, now I have access to the hundreds of bad experiences or the 99/100 good experiences.

As a student I really don't care about the dean professor relationship. I am a paying customer of the institution. I want to know if that professor is worth taking their class.

Also if you don't care about students giving praise/negative reviews to a friend, that kind of says something.

> Also if you don't care about students giving praise/negative reviews to a friend, that kind of says something.

Note I didn’t say I didn’t care what those reviews were. Honestly, they’re probably more nuanced than student Evans and thus more useful in choosing classes.

I misunderstood what you meant, my apologies.

Also, the biases pointed out in these reviews aren't unique to academia. Gender and age biases exist everywhere.

But alternative isn't bias someplace else. The alternative is engaging in biased activity or not doing to.

American colleges more resemble four-year all-inclusive resorts than they do rigorous institutions of higher learning.

Serving someone well doesn’t necessarily mean making them happy. Students — people who by enrolling at an institution in a course of study admit their ignorance of that domain — do not seem to me good judges of an instructor qua instructor.

> American colleges more resemble four-year all-inclusive resorts than they do rigorous institutions of higher learning.

I've been to both American colleges and all-inclusive resorts and, no, they don't.

I’ll take your word for it, but the ads for Sandals resort I saw on tv twenty years ago sure seem a lot like what I observe walking through NYU’s campus every day.

Answering emails from students while at an all-inclusive resort made this comment particularly funny.

And yeah, they really don't.

There is a universal baseline expectation that everything the instructor says will be correct, so the student's lack of knowledge isn't holding them back. What they have to evaluate is how good the professor was at teaching, and they are the foremost experts on that subject.

Students aren't reviewing the quality of their campus on-site gym when they review their professor.

There's genuine concerns that should be reasonably heard. One example is slowness in grading homework/exams/etc. What happens is professors get backlogged, and you have multiple homework/exams and the student missed on understanding some concept they were later graded on. If the student had regular grade updates, they would know where they needed to focus more on the course material to master it. Instead, those misunderstandings snowball and you end up doing worse on a final exam since you didn't know what course concepts you correctly understood and missed.

I agree broadly with the wholistic assessment of what American colleges have become. But the review process is still germane-- classes are the core offering of college.

I remember with fondness a professor of mine who drove mad some of my classmates by basically only grading papers on demand, due undoubtedly to some of what you’be mentioned undoubtedly.

Classes may be what students take when they go to college, but I always thought of them more as ice-breakers for the forming of relationships with instructors and fellow students and a way to foster a community of intellectual enquiry.