Study finds professors’ attitudes affect students’ grades | Hacker News Reader

Study finds professors’ attitudes affect students’ grades

arstechnica.com

75 pointspseudolus56 comments

There where some clearly recognize able outliers in motivation where I studied. There was a mathematics professor who would try to fix the edu gaps and gaps in understanding for every student willing to listen often at expense of her own time. There was the embedded systems expert who tirelessly showed fascinating stuff to less then ideal students like me. There was the algorithmic prof who's lectures always transported this Flair of "remember all this did not exist half a life ago. There is still adventure around and novelty to be found". All still had feynmans fire in their eyes. It was a privilege to study beneath them.

I'm not saying the claims aren't plausible, but it sounds like this could be p-hacking? "No other factor the researchers analyzed showed a statistically significant difference among classes—not the instructors' experience, tenure status, gender, specific department, or even ethnicity."

To me it looks more like due diligence. If you present only the main result every single reviewer will ask "Well couldn't the instructor's gender/ethnicity/tenure/experience explain this result?" Then you're forced to include this analysis either way.

That's true. We can't know which came first though, I think the proposed solution to this is pre recording of methodology and hypothesis.

Honestly, I'm inclined to believe this study was done ethically, but I wanted to point out the possibility.

Right, but I don’t think them saying they examined the other variables makes it more or less likely they were p-hacking.

It's not p-hacking as much as it is conflating statistically significant and actually significant. The fixed teachers gave whites/asians 7% higher marks while the growth teachers gave them 5% higher. That might be a statistically significant gap, but a 2% expected increase in grades is nothing. And way less of a difference than the gap between the grades fixed and growth teachers gave.

Any gap can also be easily explained by grading styles. A fixed teacher probably places more emphasis on tests and the entirety of the work over the course of the semester. A growth teacher probably places more emphasis on homework and gives weight to improvement over the course of a semester. Asian/Whites with their relative head start over URMs would on average do a bit better under the first teacher.

two years' worth of students’ grades in those instructors’ classes

...grades assigned by those instructors? I haven't looked at the details but that's what I think is the biggest flaw in this study. It would not surprise me if the "growth mindset" instructors were also more lax with marking, introducing more noise in the grades while simultaneously making them higher. (I worked in teaching CS a while ago, and have seen some pretty horrible things...)

I'm personally of the "growth, but with a limit" mindset --- I wonder which side that stands on their scale.

> grades assigned by those instructors?

Probably not directly. These are STEM classes at a big public university; I'm guessing most of the grades were assigned by TAs, and only approved or minimally adjusted by professors.

Biases like these and many others are obviously problematic throughout our education system.

Is there a good way to abstract testing away from educators so they can focus on teaching? Isn't that the way it should be done?

In high school I found video games and chess, which captivated my competitive drive. Shouldn't education have such objective competitions where possible? I think that would eliminate such biases, captivate more minds, and propel human progress. No?

> Is there a good way to abstract testing away from educators so they can focus on teaching? Isn't that the way it should be done?

Ideally I think this should be done at a high level -- i.e. separating the institutions that do the teaching from those that do the testing and award the credentials.

More practically in the short term, it would be great if teaching and grading were separated and allocated to different staff members. (Preferably with a proper firewall between them, to guard against politics and favour-trading.) There are just too many conflicts of interest and emotional pressures involved. This separation may also increase the likelihood that grading would be taken seriously, with markers given enough time and incentive to do the job properly.

I’d like something a little “panoptic”. Have everything the student interacts with the teacher be recorded on a reviewable medium. Every minute of the teachers class gets recorded from multiple angles - no blind spots. Have teachers drop in on other teachers feeds randomly. This could be recorded material but should most definitely be done so that the teachers don’t know one another. Give the students the ability to flag a grade for review. (If the student flags grades often then that should trigger a review. Maybe a discussion with a school counselor.)

I’d also like surveillance in place with the express intent of catching and correcting bullying. But that’d encounter political push back. Bullying, though, should be taken as seriously as bringing a gun to school and a kill list. Lots of counseling, parental oversight, etc

And if all of this is too much then we scrap public schooling and offer online teaching with plenty of library study spaces.

You're certainly right in that people would object to the construction of an out-and-out panopticon.

Ask yourself why any part of the lecture and homework should be kept private. We already know to not assume privacy in public spaces. Teachers are in an environment where they should not assume privacy. Student grades are personal and should be under their control who sees them - but it should be possible to challenge a bad grade. There are malignant teachers in the world.

Some people aren’t particularly fond of performing in front of a camera to have their performance reviewed by—the data will be leaked—anyone.

It should be possible to flag a bad grade, doesn’t mean we need to institute a surveillance system to achieve this. The same goes for bad actors playing teacher.

If you're ever been victim of bullying or abuse, especially by an authority, you'd consider the right to record yourself in public to be important.

The ACLU even has an app that can start recording by voice command. You should use it any time you are in a scenario with disproportionate inbalance of power (like when a police officer pulls you over).

Exactly. For me, school bullying marred me deeply. Even to this day, going on 30 years later. I didn’t know what to do back then to fight it or stop it. Even that I should have. Someone seeing it on camera could have caught it and forced the issue. I wish someone had every second of every day in one way or another.

I am an adult and live with targeted hate and bullying. It's extremely difficult.

That must be tough. Do you have any one, or any services, you can reach out to for support? You're not alone.

To be honest, it's hard to confront, and I don't even really know where or how to begin to get help.

My email address is in my profile, please feel free to contact me.

For me, it took me until I was 22 to realize the bullying actually stopped once I changed high schools. To this day I feel naturally distanced from people not dissimilar to fear of bullying coming back. I don’t have an answer. Just sharing.

I have been the victim of bullying and abuse, both at school by arse-hat students, teachers, year-level co-ordinators, and by the police.

I have, at times, walked in to situations with my phone's voice recorder running. I have, at times, photographed and filmed the police.

What does the right to record yourself in public have to do with institutionalised panopticon style surveillance?

Bullying is horrifically effective at protecting itself. One huge reason I never got help with the bullying was fear. Fear that moving me in the class room would further isolate me from my peers. Fear of retaliation. Etc.

They had me convinced I deserved the abuse or that it wasn’t abuse. They were “just telling me to expect to live my life alone and that no one would ever love me” out of the goodness of their hearts.

Why seek help with something that’s “not even a thing”?

That’s why third parties need to oversee student interactions.

Thanks for this comment. It is accurate at describing my circumstances as well. Although in my case I think I am fortunate a lot of people want to help me, but are scared to, or dont know what to do. Unfortunately, my immediate peer group is where this started.

I missed a chance to directly reply to this.

I wasn't necessarily arguing for institutionalized/state surveillance, but that ideally, by my own experiences, I'd wish I'd have had every moment of my interactions in public available. It's simply too easy for people in power to abuse others.

Well, in theory certifications such as the CFA, PMP, GMAT follow the same rationale as yours. Other countries such as Brazil and China use national tests to follow the same rationale.

Do you still think that would be the best approach?

I think it would be a interesting exercise to judge grades the same way we judge other types of psychometric testing.

Test-retest reliability / concurrent reliability (not sure which applies):

Doesn't look great. Grades can vary dramatically between classes, even if those classes are closely related.

Predictive validity:

Not sure. I'm not sure what, if anything, GPA is meant to predict, and I've read varying things about how well it predicts success in one's career.

Face validity / construct validity:

Abysmal. What are grades supposed to measure, anyway? Work ethic? Motivation? Willingness to obey authority? Intelligence? Demonstrated mastery of the material? I have no idea.

Whatever they're meant to measure, they measure a lot of things no one would want them to, like the student's race and sex, whether or not they're working while in school, when the class starts, etc.

> abstract testing away from educators so they can focus on teaching

Some systems attempt this, at the level of individual courses, and what seems to happen is the educator focuses ever more on bureaucracy. It means that they can't be free to adjust the course to the class & the time, but must pre-commit to some plan which is approved by someone else, and the exam will be submitted many months ahead of time to get checked by other people, who do this in addition to their own teaching. All supervised by ever more administrators, incentives to make themselves unavoidable.

Some things like bar exams are external and standardised. (And GRE etc.) But trying to run every course this way seems a terrible idea.

It’s probably just politics, but it looks to me like the GPA disparity between white and minority students is another issue entirely. This article making it the primary focus felt really sloppy and forced. Can’t we just report that professors with shitty attitudes affect students grades negatively and leave it at that? That’s what seems to be the common issue here, let’s have an inclusive solution rather than only focusing on one party.

Are you suggesting that once found out about the statistical significance they should have hidden it?

There were a lot of better quality results that aren't reported from this study: http://advances.sciencemag.org/highwire/filestream/211174/fi...

Instead that article talks about some p=0.041/0.049 (!!) bs.

I think one of the main motivations for studying the growth mindset is to understand why underrepresented minorities perform poorly and to develop strategies to improve their outcomes. This evidence suggests it does and it is important that the authors report that finding.

I take issue with the assertion that the time that a module requires is a good assessment of its difficulty.

I wouldn't be surprised if it is considered one of the variables. Most people lose their attention as the minutes go by, so if we add it as a multiplier to other direct variables, it can help better define difficulty.

Yeah, seriously. My friend is taking a beginner-type CS course (how to use various software, basic Excel stuff, macros, databases). The assignments are extremely time-consuming but laughably easy.

On the other hand, some of our math courses have assignments that are very short but extremely difficult. A person who is an expert in the material could probably do the assignment in about 20-30 minutes but for beginners it takes hours to figure out some of these proofs.

I find this "growth mindset" stuff disturbing.

Let me start by saying that I don't believe a word of it. The evidence I have seen suggests that intelligence is important in pretty much all intellectual activities, can be accurately measured by IQ tests, and cannot be improved. This is also, as far as I can tell, the strong consensus of the psychological community.

Assuming this is true, the proliferation of this "growth mindset" thinking has at least two scary thoughts:

1. If we encourage people to adopt a growth mindset because it will make them do better, than we are telling them what to believe without caring whether or not it is true. We are encouraging the adoption of convenient beliefs. Is that what educators should do?

2. It may lead to building an even more unequal society than the one we already have. Imagine the person who has no natural ability at all in a given field and does not succeed in it. If that person has a fixed mindset, they can conclude they have no ability and move on. If that person has a growth mindset, will they have any other option than to blame themselves for being lazy? A growth mindset means that mere bad luck is turned into culpability. Success is largely the product of luck, whether due to the wealth of one's parents or one's natural ability. If everyone in society adopts a growth mindset, then successful people can claim they are morally superior than people who fail.

Even assuming this research is accurate, it's not some happy uplifting story about how being positive can solve all your problems. It's a nasty ethical can of worms.

Sure, if a growth mindset didn’t actually improve outcomes it would be negligent to promote a growth mindset. However, the evidence suggests that a growth mindset does improve outcomes. In the interest of improving outcomes a growth mindset should be encouraged.

Your second point is true, that success is often determined by factors outside of an individual’s control (e.g. luck, parental income, school district, discrimination, etc). But we should educate people on which factors that lead to success are under their control, and it appears the growth mindset leads to success and is under their control.

>In the interest of improving outcomes a growth mindset should be encouraged.

In the interest of teaching students real science and encouraging critical thinking, what should we do? Even if the growth mindset works, the fact that it works doesn't imply we should teach it.

My point is that the growth mindset is real science and additionally makes students better at learning real science.

It may be the case that a growth mindset causes more people to waste effort in a field that they have a severe lack of talent for, but for those who have average talent but hit the wall that many professions have and struggle to overcome it a growth mindset can be an important tool in overcoming that wall and reaching their average fixed potential. Beyond that, specifically for people with high fixed ability, a growth mindset can drive them to develop their talent far beyond what they might have been content with under a fixed mindset.

I'd definitely agree that more research needs to be done, but at the moment I'd say that optimisism about your personal potential is most likely a strongly positive thing for most people.

What you say is all true true. It's also orthogonal to my concerns. I am willing to assume that having a growth mindset works 100% and makes everyone's lives better. That doesn't necessarily mean it should be encouraged.

A "growth mindset" is a scientific belief. It's not a personality trait, a positive outlook, or some sort of optimism. It is the belief that intelligence can be improved through hard work.

(You may notice that I consistently have "growth mindset" in scare quotes. A "growth mindset" is a scientific belief, and I wouldn't call it a "mindset" any more than I'd say doctors have a "germ theory mindset" or physicists have a "Copenhagen mindset.")

How intelligence works is a scientific problem. Encouraging a "growth mindset" because it makes students do better means weighing in on one side of the debate just because we like its conclusions.

If intelligence is largely fixed, as I believe it is, then encouraging a growth mindset means lying to people because we think they can't handle the truth. That might be the right thing to do, but it's a dangerous habit to get into.

If it's a convenient belief that also produces good results, then the utilitarian in me is less interested in truth. If it works for some people, that's enough.

At the age of 14 I took a Mensa test which came back with an IQ of 176. Up to that point I'd had reason to suspect I was smart. Once I had the 'proof', things went very badly for me, essentially I stopped working in school, especially work I didn't like. I got very confused and quite depressed when I didn't get the best result. I was smart and quite isolated in a small northern former coal mining town in England. Perhaps the tests were stupid. I eventually scraped into university and did very badly there (but fortunately found the nascent internet and a passion that sustains a career).

So 15 years later when I start reading about the growth mindset theories I'm like 'fuck yes!'.

I was thrilled and relieved when my youngest daughter (very smart, but prone to giving up if she doesn't crack a problem quickly) came home after starting secondary school and gave me a lecture on growth mindset. It's a core part of the schools ethos. They are a selective grammar school where everyone is smart.

It doesn't need to be absolutely empirically true to an effective learning tool. The mind is a slippery fish and they vary a lot. I see it as a tool like the pomodoro technique or flash cards which work better for some people than others.

TL;DR Growth mindset tools are especially important for smart people. It's just one technique in education.

I'm not entirely sure you and the parent poster are necessarily disagreeing. Imagine we are watching a 10 man marathon about to start. And you and I see that of the 10 competitors, 3 have a ball and chain strapped to their legs, 5 are just somewhat weighed down, and then 2 of them are somehow riding scooters. Yet from the perspective of each of the competitors they all look pretty much the same to one another.

I'd fully agree that telling the 2 that they should just try their hardest is definitely the right thing to do. Letting them know their advantage, especially prematurely, would be only likely to drive hubris and create a story of the tortoise and the hare. But if they apply themselves, the sky's the limit. Yet at the same time, what of the 3 with the ball and chain? They're not going to win, or even be able to compete in this field, regardless of how hard they try. Trying to convince them that they can do anything they want with hard work might make us feel better about ourselves, but I doubt is going to make them feel great as they find failure after failure regardless of the amount of effort they expend.

Maybe the best solution is indeed to just pretend, but I think that would only be because it is a least awful solution. I certainly cannot see any argument to be made that this is a desirable or ethical idea as it relates to the three we're talking about.

> Identifying information was removed, but some information, like entering SAT scores and ethnicity, was retained.

This, together with grades and the classes the student took, doesn't seem like it would be very anonymous?

In a recent college mathematics course I took, the professor took an hour in the first week to explain research on the growth mindset. More professors should do this!

Marking students assignments is a total fucking minefield. Perfect code? But poor refetencing? Fail. Its almost pointless, like i tell them, your grade doesn’t matter, make sure you can code and you’ll ace the interview for your dream job.

> make sure you can code and you’ll ace the interview for your dream job

I would say that your ability to code has nothing to do with whether you pass an interview. For example, here's a rejection I received recently:

> the decision was made mainly around our estimate of your level of enthusiasm for the company and the role. We are committed to hiring people who are strongly aligned with our mission and excited about the organization and what we are doing. In our conversation, you didn't express much interest in us as a company. [...]

> As I mentioned above, it is very important to us that our employees are excited about the company. We value that just as much as we value a fit from a skills and experience perspective. Again, I am sorry things didn't work out, but I hope you find this feedback helpful.

> you didn't express much interest in us as a company

I have never worked at a place where I gave a shit about the company. I come in, I do my job, try my best to ensure things get better, eventually I roll out.

Never mix business with pleasure.

That attitude is adaptive for you and maladaptive for the company.

Sentencing is harsher the 30 minutes before lunch then after lunch.

We need to have AI run everything to prevent bias.

Can the AI just stagger lunchtimes?

The sentencing study that found those differences was looking at parole decisions. It is not at all clear that the difference before and after lunch (and a similar difference after an afternoon break) shows any bias.

A later paper showed that you would get the same effect from a simple, rational time management approach. Cases that are granted parole tend to take longer to decide than those that are denied parole.

If the judges can estimate before the hearing which type of case it is likely to be (from things such as the thickness of the case file, what kind of representation the petition has, etc.), they would tend to schedule the cases that look like they will take longer first thing in the morning, or after lunch, or after the afternoon break, and they would schedule the shorter cases closer to break times.

This would result in more granting of parole in the morning and after breaks than before breaks.

Here's an article about this [1].

[1] https://mindhacks.com/2016/12/08/rational-judges-not-extrane...

Yup. And this should have been blindingly obvious to everyone involved in writing/reviewing/editing the original paper -- the effect size reported was just far too large. <10% parole before and >70% after lunch? That's not a mood swing. Especially since a lot of parole cases are very very routine, and not really close judgement calls.

If this was at all realistic, we'd also see similar effects in people in other fields - bus drivers more likely to crash before lunch, etc. That we don't see such an outrageous difference implies that the original study is not tracking the relevant variable when it looks at before/after lunch.

How do you prevent the AI from learning the lunchtime rule?

Or should we leave the lunchtime factor out and just dilute the unfairness across every sentence?

AIs don't get hangry because they haven't eaten.

Reply on news.ycombinator.com