Students Evaluating Teachers Doesn’t Just Hurt Teachers, It Hurts Students
chronicle.com
chronicle.com
It’s too bad, too, because there certainly are criticisms to be made, and this article hints at some of them. But a well designed evaluation instrument along with critical interpretation of the results and comments can be an invaluable tool for instructors who want to improve their teaching.
But unfortunately this article also spreads untruths about the weight given to student evaluations, and dismisses common-sense security protocols like having a student deliver the results instead of the instructor (not perfect of course, but the student has much less incentive to tamper) as demeaning rather than a sensible precaution.
If the author had presented any evidence whatsoever to support her claims linking student evals to grade inflation (possibly related, and I’m sure studies have been done, but evals are very far from the only obvious contributing factor), or even made a passing attempt to explain what percentage of weight is given to raw eval scores for actual tenure considerations (very little in the broad scheme of things), and cut the junk out about how universities seeing students as customers is a new thing (it’s not new at all as any skim of the history of universities as an institution will make clear, but someone at a US state university should realize that it’s skyrocketing tuition rates, and not whiny students, that give students a bigger sense of entitlement) the article would have been much better.
Solution: stop the evaluations. Even most entitled students won't make unsolicited complaints. And of the ones who do? As long as student complaints aren't used to rank every professor relative to each other, but rather only to weed out the most egregious instructors, there's significantly less pressure for grade inflation. It's when you provide the administration a regular, systematized, "objective" stream of data points that the temptation to assess professors this way becomes irresistible for the administration.
You just made that up.
> So a professor that systematically inflates grades will be called out.
Yyyyyyyeaaaahh...where?
From experience, the average mark given to Harvard humanities undergraduates is an A regardless of quality. Students literally cry in the middle of class if they get an A- because they can't string together a coherent argument from evidence.
In any case, consider that this system is in fact a solution to the proposed problem.
Believe me, I'll be first in line to shame their academic standards. However...
> In any case, consider that this system is in fact a solution to the proposed problem.
It might. I think that it creates other sinister problems in the process. You have to ask whether you think that the purpose of going to school is to better yourself versus competing against your neighbors. If a person does poorly in class because of someone else's performance, that's pretty fucked up. Likewise, if someone does well in class because of someone else's performance, that's also fucked up. The system you appear to be describing perversely encourages sabotage and cheating, because learning is secondary to "winning", because you're fucked by other people succeeding. My intuition is that systems that treat grades like a competition produce a combination of accidental failures and assholes who treat other people poorly.
Even without this historical evidence, however, it does seem unfair to grade this way for exactly the reasons in BugsJustFindMe's response. It might make sense in larger first-year classes, but in the more advanced smaller classes (say, 30 students) it's likely that you're already down to a set of students who try really hard and potentially all deserve an A. There's a lot of correlation between (perceived) difficulty of the course material and the students who take the class, so there's no reason to expect that you have the same distribution of students from those beginning first-year courses in your "Advanced Stochastic Processes" course.
It must be both. In order to better yourself, you must challenge yourself. Your peers have similar capabilities to you, because you both met roughly the same standard in order to get into the school you got into, as opposed to a better or worse one. Competing with them will therefore be challenging, but not too challenging.
Note that I don't propose the average is set at precisely 75%, nor that professors change the weights after the fact to make it so. Just that when they are setting the course - choosing what material to cover, what to leave out, setting the exam - they bear in mind the aptitudes of their students and choose appropriately. If the grades come in too high, meaning the students found it too easy, they might in future terms increase the pace or the difficulty of questions to compensate. This means that as a student you aren't competing with your class, but rather with the body of students that came before. In turn, you gain nothing from doing poorly, because you will still be awarded a proportionately low grade, and only make the course easier for future students.
Why this isn't implemented at the schools you attended I could only guess, but perhaps these schools historically determined standard based on more objective criteria, and have ceased doing so more recently without replacing their system for determining standards.
Some of this probably has to do with internal requirements. For example, if you say that a student needs a 75 or better to count this course as a prereq for the next course, then do you really want half the students to have to retake the course every time? Some of this also has to do with societal expectations of getting As and Bs.
I can say, however, that the idea that a 75 is the class average is definitely not true at most US colleges.
This is admittedly a more likely explanation for harvard's higher grades then the one I offered.
<60 is failing. Very few students actually fail courses.
In general, in my experience, grades for the courses weren't a symmetric normal distribution with a mean of 75, like it is thought. The mean is more like 83, and it was skewed toward higher grades.
To answer your final question, no, I don't think the administration would care if your class average was 95%, quite the opposite. I had one professor in my undergrad who was more "old-school" - challenging coursework, no spoon-feeding, your grade was your grade and that was that. He'd regularly complain about visits from the "Center for Student Success" who complained that he was grading too harshly and fought (on behalf of specific students who went to them with complaints) for higher grades. According to my classmates, they were actually able to get their grades overriden at some level higher than he could view from his system.
An instructor who actually wants to improve their teaching solicits and engages with feedback from the very first day of the term and doesn't wait until end of term evaluations roll in. Why? Because end of term evaluations, by definition, cannot help the students who wrote them.
(Unless you define "want" as "what the behavior of this system would lead to" and thereby pretend that no human act is an error)
In particular, a very close friend of mine is a tenured lecturer. He is a really good guy, but good lord is he a terrible instructor. The student reviews for him agree.
Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female professor, and their qualitative answers were also very different: https://www.insidehighered.com/news/2018/03/14/study-says-st...
I'm grateful to be a computer scientist, where the consolation prize for not getting tenure is a cushy industry job, but I feel deeply for my pre-tenure colleagues in the life sciences who have to work twice as hard for their evaluations while still maintaining their research and service requirements.
students rated identical teachers
[1] claims to be comparing "identical teachers" but [1]'s sole claim to courses taught by male and female instructors is "neither students’ grades nor self-study hours are affected by the instructor’s gender". Clearly those are hardly the only factors relevant to teacher quality. Moreover [1] claims to use "objective measure of the instructors’ performance", and which includes -- I kid you not -- "self-reported number of hours students spent studying for the course".
[2] is similarly vague, and claims that "the courses were identical: all lectures, assignments, and content were exactly the same in all sections" only to to state in the next sentence that the "only aspects of the course that varied between Dr. Mitchell’s and Dr. Martin’s sections were the course grader and contact with the instructor". Well, isn't "contact with the instructor" significant?
Both [1, 2] use p-values [3], which doesn't increase confidence in the results.
As an aside, neither paper discusses potential bias the authors might have, in particular their own social desirability bias [4].
[1] https://academic.oup.com/jeea/advance-article-abstract/doi/1...
[2] https://www.insidehighered.com/news/2018/03/14/study-says-st...
If it was possible to objectively determine teacher quality, student evaluation probably would not exist in the first place. The reason we ask people's opinion is in order to quantify the subjective. Double edged sword, because we tend to conflate quantified & objective.
That's half the theme here. erm.. people's opinions are subjective..
Part of the problem with "student-as-consumer" is that students aren't always the real consumer.
[2] is similarly vague, [...] isn't "contact
with the instructor" significant?
The article doesn't detail how their online courses are structured, but when I've done free udacity/coursera/edx courses, contact with the instructor has been nonexistent.
Seems to me it'd be very difficult to design a test to investigate this that wouldn't have some valid methodological criticisms. Even if you had lecturers, delivering online lectures with fake names, using voice-changing software and online-only office hours, you could still criticise that as being unrepresentative of real college lecturing, and having confounding factors if the courses ran at different schools, years, or times.
Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses.
Though the worst professor I had at RPI held our grades hostage until the evaluations were in, so they're not perfect.
Just set a threshold - such as “scores an average of < 4/10”. If the score is below that threshold, invest a little more time and effort into getting more detailed evaluations and figuring out why the score is low and whether it’s a genuine problem with the professor vs something like their having high standards and lazy students. Then train or fire as appropriate.
And by "you", you mean "no one ever". Teacher evaluations have immediate and universal impact. There is essentially no filter and no interpretation on this. It's like tweets. Once a social signal is out-there, public, everyone is acting on the implications regardless of hypothetical mitigating factors.
A lot of people talk about this stuff in the abstract, as some hypothetical, like we'll do this and we'll have safeguards in place. The news is, this is how things have been for college teachers for a while. You get an evaluation and it has an impact and there's no mitigation and no contextualizing.
Also if you look at literally any study of them TAs for harder courses get harsher ratings, they get harsher ratings from weaker students.
I also spent a number of years as a TA I can say anecdotally that is my exact first hand experience - good reviews in easy courses like graphics, atrocious scores in design of concurrent systems. In fact for concurrent systems it was well known that the TA for the course would literally always get the lowest review rating in the department, and that was independent of their rating in any other courses that they TAd.
TAs also get punished if the actual instructor is bad - because students blame everyone in the course.
I mean the American system has its own slew of problems - lecturers run the tests themselves, they by design don’t provide prior exams (hint: if knowing the prior years exam questions tells you enough to be considered your exams are bad and you should feel bad)
Adjunct professors are not tenure track, so there is really no expectation that they would get tenure. Though I have seen the numbers that say the number of adjuncts is increasing.
One of them just plays videos of himself in his lectures. He doesn't actually go to them. He is actually around during less than half of his scheduled office hours. He complains to (and at least once has shouted at) the TAs about how students keep asking him questions. He has tenure.
You have to attend every lecture (I did anyway), redirect / fix during discussion / lab, help them learn what they need to know to pass the exams, and if you have time, what they should remember when the term is over for their future academic / work careers.
Only bad ratings I recall were from students who only attended the last discussion section; it's hard to teach 10 weeks of theoretical CS in 50 minutes.
You think that's bad, try being instructor of record for a remedial math course as a TA.
Do they reuse past questions forever? That is a bad practice. Each term's exams need to be equivalent but different.
My interpretation of the way US lecturers work is that they may reuse entire exams. Especially as a lot just get exams from book publishers.
In NZ (at Canterbury at least) all exams are archived and available in the libraries, and most also as PDFs. All of them. Shelf after shelf stretching back decades. At least back when I was there - maybe it’s different now? Publishers selling exam material as part of their text books seems to be moderately recent
Schools by and large reward research, not teaching (https://jakeseliger.com/2010/09/26/how-universities-work-or-...), and, even if they do reward teaching, it's not clear that students on evals do more than reward grades.
From the perspective of most students, the ideal professor has low grading standards, “good enough” teaching, and most importantly, a “fun” class (primarily by means of humor). Most students are there to get a degree, to get a job. The best teacher according to students is the one that best fits that model (which is generally at odds with the goals of classical academia).
Undergrad feedback I wouldn’t trust at all regardless of major, Masters is major-dependent (ce, cs are particularly screwed) and phd candidates are probably fine. Simply by looking at the why the general population is even in the major (eg ask an undergrad and the reason is probably not actual interest in the subject: parents, jobs, money, dropout-major, couldn’t decide, best grades in hs, etc). Their feedback will naturally reflect the misaligned incentives
If your interest in feedback is to make sure the teachers are actually good at teaching, anyways.
My girlfriend is studying in a distance university, so she basically never meets any of her teachers. This year she has to do practices at a chosen company, and she got a teacher assigned to her who is supposed to "help" her. In her study guides it is written that the teacher has to give her classes every week. After first contact the teacher said that she will give every information by email (or other kind of online communication), and won't waste the students' precious time with classes. At first this sounded awesome, because my girlfriend has a lot to do in her last year. BUT... Since then 2 months have passed, the teacher has been repeatedly asked to keep at least a few classes, because things are not moving forward. She ignores her messages for days, answers in very short sentences, and although it is mandatory for her to keep the classes, she is always "busy" at those times and doesn't offer any other dates, so it is impossible to meet her. My girlfriend spends crazy hours combing through PDFs hoping to find information on how to do her practices, what she should prepare, how to write her work diary, etc.
And the other day, when she tried to contact the teacher's higher-ups to do something about this, one guy basically shouted at her, on the phone, saying that she is making her teacher look bad, even though the said theacher works really hard, and she should just listen to the teacher and let her be. After some of this preaching he just put the phone down.
It has been a really frustrating experience for us.
Well, to be honest, I really wish everyone who wants to one more hurdle in front of teachers should understand the correlation between bad teaching and teachers having a low-paid, unrewarding and effectively abusive job.
Lousy teachers keep their job because no one highly competent wants these jobs. Lousy teachers keep their job because a huge array of bureaucratic song-and-dance exists and people good at that can be bad at teaching.
I'm sorry you had that experience but you might consider looking at the system. Corporate customer support is terrible, let's all call for more test-of-competence for these low paid flunkies too. That will address the problem.
Unions? Really? Not administrators and profit institutions cutting college teachers' salaries to the bone, creating a situation adjunct professors literally starve to death in the process. No, unions. And tenure is your other complaint? The job guarantee that no longer applies to most college teachers today. Why not complain that the average teach still is able to sleep at night and has roof over their head?
Okay then.
Unions have their flaws but it was that way well before the current disastrous regime.
And sure, teaching is competitive market because even with all the horrors, lots of people want to teach because it's something to believe in, in this horrible and the bureaucracy, that unions are at best a junior partner in, do make it hard.
As far as that goes, look at the condition of teachers in Kentucky if you think unions are the problem.
Unions have a fairly inflexible response to most "reform" plans but the problem is most reform plans, as can be seen here, aim to make teacher more insecure, more constrained, more "flexible" but without compensation for that flexibility. Sure, LA, Oakland or New Jersey might look bad but compared to strongholds of non-union teaching, they are paradise.
It would be great to come up with a reform plan that empowered teachers. The unions might fight that but individual teachers probably wouldn't. But the kinds of reform that come often from people lips - more testing and less job security, are rightly resisted by both unions and individual teachers.
It’s a very cyclical market as cohorts of graduates coming out of school wax and wane due to birth rates and the economy. All government or teaching jobs are that way — their fiscal woes lag the economy and they tend to pick up better employees when companies blow up.
Blaming unions like citing the boogeyman. It’s almost always the lazy answer that doesn’t pan out.
If a teacher grades a student poorly, there can be repercussions. They have an incentive to grade fairly and accurately. The same isn't true of students, who can grade on feelings and grudges.
Teachers are also subject matter experts in the material being graded. Students aren't trained as educators, so why would they be any good at evaluating their teachers?
Student evaluations lack even the smallest part of the rigor that they should contain. They are practically worthless.
Yes, but the above poster was talking about TEACHING poorly. It is easy enough to expect students to know things at the end of class, but harder to be of use in them learning that material.
> Teachers are also subject matter experts in the material being graded.
Some are. Maybe even most. But definitely not a universally true statement, particularly in areas where teacher pay is dramatically out of line with industry wages.
> Students aren't trained as educators, so why would they be any good at evaluating their teachers?
Fair point. But as pointed out by others, there are few other forms of evaluation/accountability. And while a student may not be able to identify WHY they struggled to learn, they can certainly evaluate IF they did. If many students agree, that's a problem best addressed quickly.
I teach a university class (just one, my day job is coding) and I've been utterly amazed at the lack of accountability applied to me and my peers. Meanwhile I have friends that teach teens, and the bureaucracy and policies they must follow seem just as bad, but in the opposite direction.
If you have a good alternative to student evaluations, please share, otherwise I'll agree with the flaws you listed but still find them better to have than not.
> Some are. Maybe even most. But definitely not a universally true statement, particularly in areas where teacher pay is dramatically out of line with industry wages.
So you say that if we find out which teachers are incompetent and actually kick them out, we would need to raise teacher wages to the point of being competitive with the industry?
Sounds like a win-win to me.
A few negative evaluations based on personal disputes can be expected. If there are grudges or strong feelings among a large enough proportion of students to meaningfully influence the aggregates, then something really is wrong.
I have seen this work well for the end of highschool test (prepared every half year), participated on the mailing list purely as an observer (where teachers discussed edge cases - and there was/is a process to get official guidance from the test writer group, but not directly, so that provides a bit of blinding against biases).
As long as we try to let teachers cope by themselves (and maybe give them a few TAs in higher ed), and don't make their work testable, we're bound to have wild theories fueled by anecdata.
The story on our professor. She could not communicate ideas well and proceedures worse. This was a second or third year accounting class. She would present some procedure on calculating some complex set of ratios. Students would be confused and ask for clarification on where different values came from (they seemed to originate from nowhere). She would pause. Look sideways at us. And ask I'd we new how to add and subtract, shake her head, ignore the questions, and move on. Evaluations we're the lever that students had to help correct the system.
They are, or at least used to be. At least at high schools in Poland, teachers would get occasional visits from higher ups who observed their lessons and would give them feedback.
Everything got recorded in an online system - the training teachers did their reflective writing &c and I read their blog posts (private blogs!) and tried to support. Managers at the University (relevant ones) could access the blogs and help out with any more serious issues. I had direct contact with the mentors as well in case they had issues to discuss.
Probably the hardest teaching job I did in a 30 year career. But almost all of the training teachers are working in local colleges and schools.
The value of most coursework and any learnings gleaned from it, has dropped significantly.
The best value from entering education institutions is instead from job opportunities after. Unfortunately, these opportunities are often based on grades.
If the grades are more important than the coursework, then of course students will optimize for it.
Students should choose institutions & teachers based on both quality of opportunities and ease of coursework.
Going a step further, teachers that don't understand the changing interests of their students should be reprimanded.
At my university, most major recruiting for internships and such happened in one semester. Now, I specifically remember the dichotomy of two of my professors' approach to this.
Prof John - If you miss a class/deadline/exam due to recruiting, you get a 0.
Prof George - I understand recruiting is happening, reach out to me ASAP if you feel overwhelmed or need a date changed.
Now, the coursework has largely been forgotten. But, I will never forget John's idiocy.
As the total number of college graduates grows, they're competing in job markets that are smaller than the total number of college graduates. This ratio is unbalanced and becoming more unbalanced with every year. The people hiring college graduates are developing stricter and stricter filters in order to decide what body of people they're even going to allow to compete in the job market; the biggest, easiest filter to sort out who's truly competitive for positions is grades, and so it makes sense that students compete to get the best grades. rather than competing to be the most competent.
Most grads are leaving college in tremendous debt with the understanding that the longer they go without employment, the less likely it is they'll end up employed. This means that while they're in college, rather than focusing on the material they're learning, they're focusing on whatever will allow them to have the largest advantage when they enter the job market. In order for this trend to reverse, the cost of college needs to be dramatically lessened; I think this could be accomplished by reducing the total percentage of high school graduates that attend college immediately, increasing the prevalence of online schools, and reducing university overhead by cutting services to students and administrative positions.
Likewise, an open secret is that college work is a simulation of the workplace. Those college students who develop through practice the ability to turn in work on time that makes their teachers happy, will also be able to do the same in the workplace. If you give up the chance to learn it while it's available to you in college, where your only risk is a Zero on an assignment, you will learn it later in the school of hard knocks. Your performance review will be a grade. There are people who come to work knowing how to get good grades, and others who don't.
This is coming from one who struggled to get good grades in college. A goose egg on an assignment is rarely an isolated occurrance.
Knowing educators, and having kids in high school, I know that the kids who miss a deadline during recruiting tend to have already missed many deadlines over the years.
I strongly believe that while credentialism and college brand recognition are real and probably regrettable, if those are your only reasons for attending college, then you are wasting your money and your life, whereas someone who has other reasons for attending the same college is getting a lot more value out of it. But colleges assume they get the same money either way, so it's really your choice.
I don't think this is the case, and hasn't been for a long time. Assuming you're not looking at graduate classes college is largely a game of keeping your head down and doing the work you need to when you need to and the minimizing effort other than that. Cramming is totally a viable option as the metrics for success in college are totally different than the workplace. The skills one learns to survive in college are extremely maladaptive to a long term functional career.
If the criteria for success are orthogonal, how can one be a simulation of the other?
Here's my stupid parable. Give two people shovels and send them into a mine. One of them comes back with a bag of gold. The other comes back with an intense hangover. What's broken, the shovel or the mine?
Colleges, despite the image of paternalism, are actually designed to let you fail. People will leak clues on how to survive, but are less likely to tell you explicitly how to succeed. But everybody has the same chance of getting a degree in something. As a result, the people who benefit from the signaling and branding of the college outnumber the people who got a good education out of it, creating the impression that college is largely about signaling and branding. But college is really about deciding what you want to get out of it.
There are doubtlessly many things wrong with college education today, but in spite of that, taking college at face value and pursuing it in a relatively straightforward way may still be a better strategy than trying to figure out what game to play in order to survive.
If college is really a simulation of work, does that apply?
The target audience is a crucial part of the feedback loop. Removing the students from the equation sounds counter intuitive. What are the proposed alternative metrics? Who delivers the feedback? And what is the feedback based on, if not on the direct opinions of the people on the receiving end of the service?
(In a sense, I guess the whole thing is optional - after all, it is anonymous, and we don't check that every student has filled the whole thing out. It also switched from in-class to an online form during the time I was teaching, and the response rate went down quite a bit as a result, but this was after the situation I mentioned above.)
It must've been a frustrating experience.
At the same time - you're making a very broad statement here based on a rather personal experience. You went from a certain regimen yielding certain results, to a different regimen yielding different results. There are way too many parameters here to draw conclusions.
In speaking with other grad students this seemed to be a well-known phenomenon, to the point that most other grad students intentionally didn't put much time into their teaching and basked in the positive reviews as a result. It was suggested many times to me that I was spending too much time thinking about my teaching. In my case, the lax teaching was not intentional, I simply was overcommitted that semester and had less time to prepare.
Not only do such institutions seemed to be concerned about cancerously growing credentialism, they’ve embraced it as a money money making scheme. See e.g. the explosion of terminal masters programs.
You reap what you sow.
I'm not sure how this statement addresses my claims.
How do we measure/quantify the quality of education? I claim that the way students feel about the staff and the institution should be taken into account, rather than dismissed.
If colleges, and those that work at them, want to be in the education business rather than the credential selling business they ought to take a good long look at where and how they went wrong instead of constantly trying to push the blame onto to people that, again, are just buying what they are selling.
Often times, time and experience give a different perspective on how valuable something was.
Indeed. I'm a professor in a STEM field.
One of the most valuable courses I ever took was in medieval feminist art history.
Doctor 1 comes in on-time, she's smart, capable, friendly, and spends a good amount of time concerned about you and your situation.
Doctor 2 comes in late, she's irascible, pedantic, hurried, and only listens long enough to tell you what to do. She also insults you and your family in various ways and doesn't seem to care.
After the visit, you go to Yelp. Which one was the better doctor?
You have no idea! But do you think that's going to stop somebody from forming and posting an opinion anyway?
You can make the argument that these other qualities are good to have in a doctor. I agree. I want a doctor like the first one. But people go to doctors first and foremost to get better, not to make new social acquaintances. If the first doc is incompetent and hurting people, and the second doc a genius, and her attitude is actually causing more people to comply with what's best for them out of a sense anger at her style, perhaps? That's the thing. Who the hell cares if the outside looks cool/sexy/friendly or not?
So when we ask for reviews from students who have never worked in the field they're studying for, they're like that Yelp doctor reviewer. I don't see how this situation is good for anybody. In fact, it'll probably lead to a bunch friendly, good-looking morons with great people skills entertaining bunch of kids who aren't learning anything. That's the only natural consequence here.
Pre-register patient problems, and see how fast doctors solve problems on average/median. Then it doesn't matter how jovial/amicable the doc is.
With enough data you can fish out potential signals (and then test them properly), maybe bedside manner really doesn't matter, maybe being on time is more important. (Because then people spend less time in the clinic among other sick people.)
And as others mentioned the teacher eval problem can be solved by splitting the lecturer, exam writer and grader roles. (Grading should be done by a book written as edge cases accumulate, edge cases should be handled without names to prevent bias, as much as possible.) And then aggregate stats can be published about each class and school.
Doctors, like professors, are evaluated by former patients based on a mix of factors.
In your doctor example, doctors don't "solve problems". They see patients. Many times these patients present with conflicting and hard-to-resolve symptoms. I imagine with the internet this has gotten worse. They use a mix of Occam's Razor, deductive reasoning, and social psychology for the patient's long-term health.
You don't want a doctor that finds the answers to complex problems and gives out morphine. You want a doctor that over decades has patients with the best outcomes. In my example, the one doctor may have ticked you off so much you decide never to go back -- and quit smoking just to prove her wrong. That's a win for both you and the doc, although there's no problem being solved.
You can speculate various ways to go at this. Perhaps the doctor is evaluated by other doctors working at that location. Or perhaps it's a matrix of various things. As long as you're just pulling stuff out of the air, you can invent any kind of system you'd like. And tech will deliver it for you.
Human interventions are messy affairs, and thank goodness for that. We are not robots. Your proposal might do a great job of sorting out professors who teach students well enough to take the test, but heck if I see where that is the end of the analysis. That's just the beginning. We're just getting started talking about professors who can lead a class to mostly pass a test. That should be a baseline for any professor.
This is the problem we see in software development. You can come up with all sorts of metrics to stand in place of evaluating whether a team/person is good or not, but whatever you come up with mostly seem to work -- and has very bad effects on those being measured. (With the possible exception of business-value-loop feedback time)
I don't think this is tractable in the way you seem to want it. If I were picking a college or professor, I might look at five or six variables and weigh them in various ways depending on my personal needs and goals at the time -- which might change tomorrow. I might want to go interview them. Meet some current and former students.
There is a severe and onerous "platform danger" in tech. It starts with the conceit that we can build platforms for anything. It ends up with people not-as-smart-as-they-used-to-be having Google tell them where to go for dinner. Or Wally in Dilbert coding himself up a minivan. [0]
I'm not saying the metric definitions and accumulations aren't interesting. They deserve to be considered. I'm saying that by reducing human judgment to single vector, it makes us as a species dumber. That's one of those intuitive things that sound good but are in reality quite bad.
The vast majority of healtcare problems are dependent on using the right known technique (medication, surgery, etc.) and of course the problem is finding that, so diagnosis. Handing out morphine is not a solution in the genral case.
Long-term health is sadly largely independent of acute healthcare providers' work. It depends on genetics, environmental factors, behavior (exercise, diet, lifestyle) and a stochastic factor of acute problem solving (as the body breaks down more and more depends on doctors, and the fast and correct identification and application of the best healthcare technique).
So healthcare has two categories, the first is the common flu, I have some gastro issues, refer to specialist, manage chronic illnesses (diabetes, pain, low/high blood pressure) and the other is usually the problem for specialists, when they have no or wrong idea what's going on and thus an acute problem becomes a year(s) long trail of tears.
Now, that said, I'm not advocating for ignoring the edge cases and just use this one simple magic formula to decide who can play doctor. But we are still stuck in the dark, because we focus on the complexity, the long term, the human component, the expertise, yet fail to do the most basic thing to get at least somewhat reliable data.
And I'm aware of the problem of fishing for signals in data, but without it we can't even start the trial and error process, we can't differentiate between the two cases, and on average let's say we end up doing nothing.
Furthermore, we know that the obvious solution is to refactor the system. But that rarely presents a real possibility for education and human long term health. (You know, the small stuff :)) It'd be easy to ban smoking and alcohol and give less damaging stuff for people instead and spend more to treat addicts, and introduce negative income tax as a form of basic income to help poverty to reduce stress and help mental health to reduce number of smokers. Similarly it'd be easy to reform schools, and do all of the above to help parents to have more time for their kids so they get a better nurturing youth overall, which makes them ambitious and motivated to learn, study, explore and understand. And it'd be easy to do all this, but that's not likely. Hence the proposal for simple but maybe vastly effective changes.
Hmm? There has always been private schools. Nothing is new about them
This is probably what is being referred to.
Which the student is required to repay, but in any case are not from the chartering government.
> There really isn't any private education in the US past high school
There are plenty of private beyond-high-school educational institutions that don't qualify for government financial aid and thus rely on purely private financing, so even if you consider “accepting government issued loans” as making an institution not-private, there is plenty of private education.
1) I would have used "corporatizations" of schools. Running universities as if they were businesses, framing students as customers, etc.
2) Tuition has become a bigger share of the budget for public universities as federal funding has been stagnant and state funding has been cut, which feeds the same problem.
Certainly feeling entitled to a good grade is a possible consequence of having to pay for it yourself, and there will always be those who react that way, but I think it is a socialized response that depends on other factors.
That something that they should be entitled to is a good education.
The purpose of the review should be purely to provide feedback to the instructor and TA. It should not be available to the administration - the purpose is to help the TA and instructors improve their teaching. Especially if the instructors have tenure and so can’t be fired for failing to teach.
Ratings should also be scaled to control for biases - what are the average grades given by each student, how do they compare to other students in their classes, and to their course grade.
Furthermore it forces teachers to communicate the requurements, the curriculum and sillabus well to the exam writers and graders.
If another professor wrote the exams for a class they
didn't sit in, you'd have the students screaming in
unison, "this wasn't discussed in class!"
For example, in electronic engineering I would expect a second-year class in switch mode power supply design at any university to look extremely similar.
Admittedly there may be classes in cutting-edge research, opinion-based subjects or extremely niche topics, where you can't find two academics in the country who could agree on a year's syllabus and set of answers. In my degree I'd say less than 25% of courses were that way, though.
"They are no doubt imprecise, but for a majority of the faculty the scores form a tight bunch around a mean. Newer faculty are likely to have lower scores, so it is often times useful to sit down with them and review the feedback and talk about ways to improve. A few faculty members continually score higher than average, and I want to know why--perhaps they can help do some mentoring with the lower scoring faculty."
Of course this comment assumes that higher scores means better teaching.
FWIW, my experience both as a prof and as an evaluator of profs is that when students are reasonable good their comments, in general, are reasonably useful. After all, they are in the room and they see what is happening.
The problem is that when students diverge from reasonably well-prepared and reasonably hard-working, their responses diverge from valuable. It is a kind of Dunning–Kruger, where you get evaluations that are hard to reconcile with any kind of learning goals standard.
All this would be mitigated if there were multiple measures. But there are not, at least in practice that I've seen.
It is a problem. As an evaluator you want to give people credit for doing a good job. But it can be hard to tell.
As a prof for many years, I know how to increase evaluations. (I don't agree that it is grades; I recently had a fifth year review and calculated the correlation between my grades and class evaluations for those years and r^2 was basically zero.) But the things that increase evaluations are not very tied to increasing learning and are certainly not tied to increasing the amount of material covered.
Lots of students under the impression that commenting on their dress is appropriate. A large chunk of those being suggesting they dress in ways more visually interesting. Some direct references to their appearance (usually "positive").
One particularly egregious one speculating on who she slept with to have her position.
Tons of gendered expectations in language. Expectations to "be more nurturing" and things like that. Far more comments about them being "young", "inexperienced" and "new" than my own evaluations, despite being considerably more experienced than I am (and no, there was no way my class was just more polished - it was thrown together last minute).
That would make for a rather interesting study if they look at gender expectations overall for teachers. From Swedish studies it has been clear that male teachers leave the professions significant more and earlier than female teachers, and same with male students for the teacher master program in higher education. The numbers is very similar if not almost identical to master programs in STEM except for the genders being reversed, a fact that is rather unexplored in gender studies but noted in a somewhat recent government study.
Just looking at evaluations, I wonder how height, build and wealth symbols (expensive car, clothing, jewelry) impact the rating for male teachers. Do a non-typical male traits contribute more negatively to the score for male teachers than non-typical female traits do for female teachers? Same question for typical male traits and female traits. Is it correlated to leaving the profession, and is there a difference in abuse tolerance?
Generally speaking, in my opinion, teaching is a very complex and personal interaction and micromanaging only makes it worse.
"a. An Engineer in private practice will not review the work of another engineer for the same client, except with the knowledge of such engineer, or unless the connection of such engineer with the work has been terminated."
FWIW my understanding is that at my university the response to poor evals is a visit from a colleague, to assess whether the teacher really is weak or if the material is just difficult. Depending on that, the instructor may get additional coaching in teaching technique. This seems like a sensible approach to me.
I always thought that lecturers, if there were multiple for a single course, would not attend one another's lectures under the pretense that they were busy, but with the actual reason that they didn't want to hold one another accountable for inefficient teaching methods.
"I don't care what you do, you won't care what I do, the typos from last year's slides repeat."
Eventually you get used to it and some even prefer it - I know I really get a lot out of pair programming with someone close to me in skill but with a much different background - but getting over that first hump is tough.
Regarding professors wanting to keep review data tightly sealed: in my view, if you can't by public disclosure of your evaluation, then you either don't feel you're meeting expectations, or have no desire to improve in areas where students feel improvement could be made.
Also, the biases pointed out in these reviews aren't unique to academia. Gender and age biases exist everywhere. This article sounds like it's just pushing the idea that students should have less influence in the hiring and promotion decisions of professors. You know, the very people that the teachers first and foremost serve at a university.
We agreed to not launch the system, in exchange for limited access to the data, as I described above.
Professors en masse opposed opening up this data. That opposition alone made me feel we were doing good work. If they had to stand by their public reviews, hopefully they could stand by the quality of the instructional experience provided.
I ended up making a startup to deal with issues like this, to sell SaaS solutions directly to student governance groups, rather than to institutions. Both control fairly significant budgets (student governance groups at mid-to-large sized institutions have 6/7 figure budgets to easily afford enterprise pricing). My platforms are student-first. It's more a passion project than trying to get me Zuckerberg levels of wealth.
Personality, difficulty, ability to teach (relatively), understanding the student's pressures etc.
When I say difficulty, I mean easiness (people take the class to improve the gpa) or difficulty as in: the professor wants to make an example.
Student's pressures: Does the professor demand more than a reasonable amount of time from the student?
----
All of this being said: I haven't seen a pragmatically taught course before. (As in the professor looks at the course as a means and is the person that helps facilitate the student through it) I think that's what students really look for.
What we're pushing back against is these reviews getting turned into yet another semi-useless, noisy and biased metric we're evaluated by.
You want to recommend for/against a class to your friends? Don't care. What I'm not interested in is my dean pretending there's deep insight there.
It should not be used as precise metric of performance, but as a general trend overall. Instead of one friend giving a bad review for whatever reason, now I have access to the hundreds of bad experiences or the 99/100 good experiences.
As a student I really don't care about the dean professor relationship. I am a paying customer of the institution. I want to know if that professor is worth taking their class.
Also if you don't care about students giving praise/negative reviews to a friend, that kind of says something.
Note I didn’t say I didn’t care what those reviews were. Honestly, they’re probably more nuanced than student Evans and thus more useful in choosing classes.
Serving someone well doesn’t necessarily mean making them happy. Students — people who by enrolling at an institution in a course of study admit their ignorance of that domain — do not seem to me good judges of an instructor qua instructor.
I've been to both American colleges and all-inclusive resorts and, no, they don't.
And yeah, they really don't.
There's genuine concerns that should be reasonably heard. One example is slowness in grading homework/exams/etc. What happens is professors get backlogged, and you have multiple homework/exams and the student missed on understanding some concept they were later graded on. If the student had regular grade updates, they would know where they needed to focus more on the course material to master it. Instead, those misunderstandings snowball and you end up doing worse on a final exam since you didn't know what course concepts you correctly understood and missed.
I agree broadly with the wholistic assessment of what American colleges have become. But the review process is still germane-- classes are the core offering of college.
Classes may be what students take when they go to college, but I always thought of them more as ice-breakers for the forming of relationships with instructors and fellow students and a way to foster a community of intellectual enquiry.
I accidentally cut myself a little too deep while doing dishes a couple years ago (opaque, soapy water and sharp surfaces don't mix well). I couldn't get the bleeding to stop (without constant pressure held for about an hour), so I went to urgent care. I knew that I needed stitches or that "glue" they use to seal wounds. I didn't care which solution the Doctor picked, but I knew I needed something.
Students have some sense of what a generally good instructional experience looks like. It's better to collect this feedback, and have people that are experts in instruction examine the reviews, synthesize with their own knowledge of what makes instruction great, and use that information to improve.
But saying we should totally close out students since "they don't know what they need" removes a fruitful data source in determining where professors can improve.
What if the feedback also said the doctor wasn't personable? Maybe the doc could be a bit warmer with his patients... and a patient would be absolutely qualified in determining whether or not that bedside manner is present.
Many students are there not because they want to learn but because they want to get that piece of paper that is the ticket to a successful career. A teacher forcing them to learn, to spend time and effort studying and working, is not seen as a positive if your goal is simply to get a degree. The ideal teacher for them is someone who just give you an A no matter what. That would be the quickest and easiest way to guarantee achieving that goal.
This is the true issue here - the disconnect between what many students want from a university education and what that education is actually supposed to be.
If that's the case, many of the students in your post are right -- just giving them the degree could save them (and the school) a lot of time and money. Maybe your art history class is a cost-effective way to get a world class education in late Italian Romantic oil paintings, but as a way of proving you're smart enough to work in a law firm (or even "broaden your horizons and become a better citizen through rounded education") I can't imagine it's terribly efficient.
It's in every student's individual interests for their school's standards for admission and grading to be low, so they can obtain the credential easily.
But it's in the student body's _collective_ interests for the school's standards to be high, so the credential retains and improves its signalling value.
There's a reason a C- student at Harvard doesn't transfer to a deprived community college where they'd be at the top of every class :)
In my experience, exams are often full of ambiguous questions, questions testing knowledge that is not part of the course syllabus, etc.
If the grading was fair, I would agree with you, but it rarely is. And IMO this is what students really hate: they put in a hell of a lot of work. They actually get to grips with the material, and for stupid reasons out of their control they end up with no credit for it.
Comments are tricky since they're both qualitative and might need to be scrubbed for anything personally-identifying.
You just can't throw out the baby with the bathwater here just because there are things to figure out. Most of the reasons people have cited here seem like lame excuses, all easily addressable if you could get sane people to get aligned behind a well designed system. Unfortunately not everyone involved in these conversations is acting sane (or objectively in the best interests of the community) and therefore getting everyone aligned in practically impossible.
This whole conversation, BTW, reminds me a the similar debate around public access to doctor and surgeon outcome data where some doctors are amazing and others have horrendously bad success rates for specific surgeries but if you go to that doctor for that surgery you almost never are given access to that history. A large portion of the medical community is very hostile to the idea of things being otherwise which, IMO, is indefensible from a public health POV and only makes sense if you are a crappy, unscrupulous doctor seeking to avoid accountability.
(At my school many teachers were slow to mark and return assignments, so this benefit wasn't really achieved - but it might have been had feedback been more timely)
There is also the third option, which the author is arguing for: that the professors do not think the ratings are an accurate reflection of their teaching skills.
Perhaps the engineering statistics professor I met who bragged that no one ever got an A on his final, because teaching wasa competition between him and the students. And the government professor I had who was inordinately proud of the fact that his course was required because three soldiers from Texas stayed in China after the Korean war and spent the rest of the first class going around the room having students introduce themselves and then mocking them. (I dropped the class the next day.) And a number who were just disorganized and incompetent, but protected by their relationship with other faculty. And the professor who was given tenure for political reasons, after threatening to fail an entire class (of a required course) of computer science undergrads because they weren't electrical engineering great students.
She also answered any questions asked in Mandarin, in Mandarin. When asked to translate an exchange for the rest of the class, she blushed and said 'it was complicated'. At the end of the semester, the class was so far behind, 1/3 of the materials for the course exam were delivered during an optional study session.
And so would the electrophysiology professor who spent over an hour of a grad seminar explaining how to use a floppy disk. Because he had trouble with computers.
I've seen courses where the students were memorising MATLAB scripts by rote for exams because almost none of them had sufficient understanding to have any chance of recreating it in the exam. Why? Because rather than teaching this stuff, the lecturer spent several lectures explaining the details of the representation of floating point number (this was a first-year class for math majors, not people doing CS).
But it was hard as student representatives for us to prove any of this, because we didn't have access to feedback responses.
That 'third option' is just an opinion about the data.
But alternative isn't bias someplace else. The alternative is engaging in biased activity or not doing to.
How the university thinks they can claim ownership of data that come's from students I will never understand.
They might've started doing so to combat the notion that private education is worth less because it just might be easier.
University is a place of higher learning, not higher teaching.
If you're relying on being spoon fed the course, you're doing it wrong.
From an individual standpoint, sure, a student is always well-advised to take charge of their own learning process. But if you can do that, what do you need a university for?
At this point I'm pretty sure the answer boils down to "a piece of paper and a dating pool."
The more expensive ones sell connections that can open the doors to wealth and power.
I could pretend that I could've learnt the same amount by sitting at home with a reading-list for four years, but there's no way I would have.
I commented on this recently in the context of MOOCs -- https://news.ycombinator.com/item?id=18511176
That said - it's perverse at the other end and I do believe that there is unconscious sexism.
We had a really smart bunch of people, and some of the female teachers were a little more apprehensive in front of us which I think signals to people subconsciously.
Another way: the feedback has to be interpreted.
I suggest that the profs should not be 'graded' by students - rather there should just be an open ended opportunity for feedback.
There were some asshole teachers employed at my college and they were called out for it accordingly.
This is obviously anecdotal but in my experience these ratings seem to reflect my perceived quality of the lectures quite well.
I've got an A in that class, but I wouldn't say that I learned much from it due to that attitude. I don't think that the CS department here has any way to tell that something like that is happening without evaluations.
The universities I went to for my undergrad and graduate programs both had an informal rating system for professors and I definitely encountered some of both.
One of the most highly rated professors I encountered was also one of the best educators at the school. He had a reputation as being a brutally hard grader. But most of the students who left his classes with some bruises and (what would have been in any other class) a mediocre grade, ached to take more from him because you felt the immense value of his teaching.
I had other professors who taught poorly, tolerated cheating, or gave non-sequitur exams on material they never covered and so on. There was absolutely no recourse or way of providing feedback on these professors and even ones who received universally poor reviews at the end of the semester stayed on and even attained tenure in some cases.
The only real recourse was for students to rank professors amongst themselves and simply starve out poor professors with lack of registrations. That's the only signal universities seem to respond to.
Teaching is an art, just because someone isn't doing it well doesn't mean they can't do it well. In fact in some cases I imagine they are doing poorly because they would rather be working on their research area. Providing the opportunity to course correct rather than starve out is good.
Basically the idea is that a representative of a group of students comes together with other such representatives and school staff (I don’t know the English words, basically the managers of a school) and discusses the good and the bad of what’s going on. I have heard some positive experiences with this method.
If you’re not doing these things, perhaps pitch it to people. Or poll other students and ask someone at the school for an appointment.
Grading on a curve is not without problems, particularly when class sizes are small, but it has a lot of benefits. It makes it easier to compare students across schools when the schools use the same curve.
Why should someone else's ability influence my grade???
Grades only make sense to compare within the same class at the same university during the same semester.
The grading in my course is not relative. I set a rubric that I assess each individual on. Sure, within the class you can compare grades, but I wouldn't say it is particularly meaningful. Comparing between classes or universities is completely meaningless.
People just want a simple number that can accurately represent someone's knowledge/learning/skill/effort, but it doesn't exist.
It's fairly easy to boost your GPA by mixing in easy courses
For starters, the quality of students is not uniform across universities. Middle of the pack at Harvard is a very different thing from middle of the pack at a party school. To get a useful ordering, you'd need to give every CS graduate the same standardized test, like the bar exam.
With few exceptions there is no mastering the material, or at least there shouldn't be. Not in college. There's always more complex material you can use to differentiate students. When you grade on a curve the test should be difficult enough that even the best students will typically get at least one question wrong, with most students falling along a nice bell distribution. (I say most because it's probably better to err on the side of a handful of students clustering at the top rather than at the bottom, especially if the bottom means flunking out.)
This can be more difficult for smaller, seminar style classes. One solution is giving the professor leeway to shift up and tighten the curve so he's not forced to give Fs or Ds. These classes are usually in the latter years of a program, particularly in undergraduate, where it's less important to winnow students out and ensure challenging curriculums.
That latter point is important. If you don't enforce some sort of statistical distribution how can you gauge the quality of your curriculum? If everybody is getting As is that because all your students are smart and disciplined, or because the material is too simple? If you require a bell distribution, then the curriculum will by necessity be a good fit for your cohort of students. In this way a university can maintain a quality curriculum without having to resort to external metrics or comparisons with other schools.
Instead the differentiation was between departments and courses. It was well understood that X majors were Y majors who couldn't hack it, and that within a major, certain classes were for high-achieving masochists while others were schedule padding. (This was crucial information, not just dick-measuring. You could easily put yourself in crisis by taking a class beyond your abilities, or multiple hard classes at once. Friends helped each other avoid such nightmares). If you asked me to evaluate a classmate's transcript, I wouldn't even look at their grades, only at the classes they allowed themselves to have grades in.
No one has ever asked me that, or to see my transcript.
If a class is filled with C and D students (objectively), the curve will spread them out and those C's become As. If a different class is filled with A and B students, they are spread out and those B's can fail or become C's. Now the first group's curve A's are not as good as the second group's A's, and are, in fact, worse than the second group's F's or C's.
I often set the curve in my classes. I was offered money to not do my best on exams because of how it affected other's grades. My ability or lack there of should not affect another's grade which should reflect their demonstration of subject mastery.
Perhaps methods to decouple evaluation scores from student grades should be better explored without antagonizing anyone? Perhaps scores after the first week should be compared to scores mid-way and then last?
Maybe all of that temporal basis is flawed too and we need to try crazier things. A weirder option could be to randomize the evaluations given back to teachers and to have the teachers select which ones seem to be theirs and some consideration is given to professors that are accurate for themselves. Basically it’d be a test of whether a teacher could identify their own teaching methods’ strengths and faults in contrast with other teachers’ and encourage diversity in styles among faculty. If a teacher wants to grade hard, I think they should be free to do so if it’s shown to be in students’ best interests. This is vaguely similar to how classifiers in machine learning training works but with a really different goal in mind.
If 45% of students are failing, then either the class needs to be changed, or more effort needs to be made to prevent students from taking a class that they are likely to fail.
People trying to judge students based on grades would then have to have a rough sense of how the institution works and how a grade should be interpreted.
But this is what they have to do anyway, curve or not. For example, employers already know that a B- at an Ivy League institution is a somewhat poor grade. (At Harvard, 75% of all grades given are B+ or above.)
Admittedly I think the Ivy League system (which effectively is grading on a curve, just a very narrow one with the mean set to an A-) is pretty good. There's certainly little incentive for students to want easy classes, because in terms of grading there is basically no difference between an easy class and a hard class. It's as close as you can come to just not giving grades at all, but doesn't mark you out as a weird hippy school.
...I don't really accept that.
The problem with grade inflation isn't just that it becomes more difficult to judge student competency; similarly, it becomes more difficult to judge curriculum quality. If everybody is getting As when the curriculum is challenging, they'll continue getting As if it degenerates. Enforcing a distribution helps you to maintain a stable relationship between the abilities of your student body and the quality of the curriculum.
In my experience, they aren't, for a number of reasons.
And especially in bootcamps since things are so fast paced and often times emotional, the reviews are not objective. Many are too positive or too negative.
I would imagine a truly poor teacher would standout for the number of unsolicited complaints.
I don't think this is effective, but it happens.
> I got a terrible rating, and its publication humiliated me.
While this doesn't mean that the article can't make valid points, large parts of it also complain about "today's youth".
I suspect this got upvoted mostly based on the contrarian headline...