New Study: Rewarding Good Teachers and Firing Bad Ones Accomplishes Nothing
motherjones.com
motherjones.com
A few years ago she moved to a school on the edge of a native reserve. She is now officially a "low performing" teacher. She insists she works twice as hard as before and has much more positive impact on the lives of her students. She's been robbed and beat up by students. Yet she now devotes her life to these kids, and would probably be fired in this type of experiment.
My guess is that the 'good' and 'bad' teachers in the study were almost totally dependent on the kids they had.
Unfortunately, people don't like what the IQ scores show, and so instead the Gates Foundation has to spend millions on a grand experiment that "teaches us nothing" as Mother Jones would put it, mostly because they refuse to acknowledge just how precisely it confirms the uncomfortable things we already know.
Friend of hers have taught in schools where a cycle of poverty inflicts the population. Most children are raised by single mothers because the father is in jail. When the father is released he has no opportunity to make a honest living for himself and the cycle continues.
The uncomfortable things we already know have nothing to do with IQ. It has everything to do with centuries of mistreatment of groups of people.
However “IQ” at least as how we measure it, has clear demographic corollarion, this does not mean it’s “genetic” at least not in a racial manner. However it does seem to be hereditary even if through nurture and environmental factors alone.
The sad truth is that until we as a society come to terms with that there likely won’t be an effective solution.
Time and time again various charities and governmental programs discovered that trying to uplift the least privileged through a new coat of paint doesn’t work, even the gates foundation does not target the “weakest” schools as those are essentially beyond hope at least at this point.
I also don’t think that the GP’s point of rating teachers relative to the current potential of their pupils rather than using local, state and national averages is inherently wrong or discriminatory, if anything proposing that all of only those at risk and underprivileged youth would have new textbooks and cleaner class rooms that they would perform just as well as privileged kids who go to private schools or to private in all but tution public schools.
And while forcing children away from their families and putting them in schools where they would be abused is terrible one thing that does seem to work is just that minus the abuse and the forcing as children with less means that get into good schools through either scholarships or voucher based programs tend to perform extremely well.
That said the most important factors as far as underprivileged youth goes seem to be missing from the study. Test scores are not the most or even an important factor for such social programs that seek to change the future outlook of these kids for the better. Teacher or well “educators” can have and should have an impact far greater than pure academic success and that is they should imprint positive social character and personality traits.
So while the scores were not improving has the well being of the kids improved? Has underaged crime rates dropped? Drug and alcohol use? Violence? Was there psychometric collected to see if kids developed traits that would increase the likelihood of them succeeding in life? Are their happier? That is the data I would expect to be collected and in my matter much more than their scores on some standardized tests.
Let's keep in mind that residential schools were absolutely the most progressive thing going at the time. Let's reach out to these poor beleaguered people and give them the benefit of our modern school system! You can see how easily the story could be sold to people as a very moral act before the actual results of the way it was handled were known, generations later.
> The uncomfortable things we already know have nothing to do with IQ.
Why establish a false dichotomy here? You've obviously pointed out an uncomfortable truth which I was already well aware of. There's no reason we can't also be aware of the implications of IQ and any number of other factors determining school performance of any intersectional cohort against standardized metrics.
> The Carlisle Indian Industrial School, founded by the US Army officer Richard Henry Pratt in 1879 at a former military installation, became a model for others established by the Bureau of Indian Affairs (BIA). Pratt said in a speech in 1892, "A great general has said that the only good Indian is a dead one. In a sense, I agree with the sentiment, but only in this: that all the Indian there is in the race should be dead. Kill the Indian in him and save the man." Pratt professed "assimilation through total immersion." He conducted a "social experiment" on Apache prisoners of war at a fort in Florida. He cut their long hair, put them in uniforms, forced them to learn English, and subjected them to strict military protocols.
https://en.m.wikipedia.org/wiki/American_Indian_boarding_sch...
It seems similar to how a "bad doctor" might be the one who is taking the difficult to save cases and having higher fail rates. Difficult children make it difficult for the teachers, but the mediocre ones won't want to bother with them.
In order to improve the output of a system, you have to understand it. You can’t just blindly incentivize on the randomly distributed results. If this research leads to a more humane work environment for teachers, freer than before of misguided incentive programs, then that will be a good thing, indeed.
Nope. The former is invalidated by the latter, if you take the statement at face value. Everything appears randomly distributed...oh wait that isn't true either as you always have ranges and generally 2 axes (one being time). Most importantly, this doesn't contribute the conversation in any way. -1
Edit: grammar.
What’s the actionable item here? Should we fire all teachers and replace them with hamsters who would coincidentally also not exercise full control?
So, no, the answer is not to throw away the experience and insight embodied in the faculty. The answer is to try to understand and remove (or compensate for) the special causes of variation in the population of students, as well as the special causes of variation within the faculty. One of the other things Deming said that I find very insightful is that the interesting thing about geography is not where the borders are, but why the borders are where they are. Applied here, the interesting thing about the variation in grades or test scores is not the variation itself, but why the variation is there, and why is it shaped the way it is. These 'why' questions only have answers that are close to the data and contingent upon the systems, in contrast to blanket incentives that try to sweep an understanding of the system under the rug of punishment and reward.
Is educational achievement closely correlated with some teachers and not others? Is the correlation confounded by other variables like the student’s family income or choice of breakfast cereal?
Most of the political discourse tends to treat all of those situations as the same.
(Growing up, my classmates were mostly in the "sufficient positive influences" camp, and yeah, the teachers who had big impacts tended to be the really bad ones who destroyed momentum/interest in a topic for one or more of us. There were also high-impact inspirational teachers - but they were more of an "alter the trajectory of what someone wants to do in their life" sort of thing, which isn't really measured by standardized tests.)
There are bad teachers. I've encountered a few that few students could learn from. Isolating this effect from the effect of the outside factors would be very hard, though.
It's not random, but it's not primarily attributable to teachers.
Good neighbourhoods with low crime and stable, happy family's produce decent students and decent schools.
Not just a money or 'quality of teacher' thing.
No doubt some teachers are a little better than others, I personally believe that it's not that important.
Keeping families employed, stable, keeping gangs at bay, having good role models, low crime, low violence/stress for kids ... these are the big problems.
The teacher is there to set up the framework and guide the parents as to what to teach.
The parent has to bring it home.
The single biggest indicator of future student success is how many of the parents at the school are married with only one working parent. I.e the parents with the most free time to help their kids.
The latter is no more satisfying because when you can't tell the difference between good and bad you get a market for lemons, so you end up with a pile of bad teachers despite the fact that good ones would help.
This is basic theory of constraints: a chain is only as strong as its weakest link. The quality of a students education is a function of many variables, and the teacher is just one of those variables. This is why rewarding a teacher based on their classes performance is unfair. By luck they may either get a class filled with exceptional and privileged students, or a class of children with abuse and attention problems.
The only way to really get a good measure the quality of a teacher, would be to control the quality of the parents, and all the other variables too.
Or - just for the record - it depends on the kind of job the parents have, nowadays it is more rare, but to give you a single datapoint, I was raised by my both working parents, but both were artisans, they had an independent (each their own) laboratory/atelier, so, bar the time I was at school, most of the time I spent with the one or the other on their workplace, even if they were working they were right there, ready to assist in case of issues with homework or only to answer the odd question/doubt.
Besides the help they gave me with school, I learned a lot from them by simply being with them, including dedication to work.
I am forever grateful to my parents for the patience and dedication thay had for me.
There is a big difference when compared to two working parents (with a job as employee) as they cannot - even if they would want to - spend much time with their sons but also (I know a couple of examples) families where only one of the parents is working but the other one is nonetheless not interested in the kids' education, in the belief that the school should supply everything related to education.
I haven’t fully thought this out yet, but we do know one thing: if you are in a hole, stop digging. If something isn’t working, brute force rarely helps. We need innovation in education but not just pedagogical— but with the entire incentive structure for both teachers and students.
Crazy idea, I know. But sometimes crazy works.
While I understand and sympathize with the desire to quantify 'good teaching', it's a 'domain' that has a whole lot of noise.
Abstractly, agents will optimize for the metrics, and not what the metrics approximate. Now if an agent in the system optimizes for the metrics (and not what they're measuring), but the metrics themselves are solid, then you'll come out with a largely okay system.
I think the point of this article is that maybe we don't have good metrics to measure solid pedagogical performance, and so, in that case, when agents optimize for the metrics, and not what the metrics measure, then the system provides no value.
So maybe the crazier idea is back-off, and try a different approach that doesn't involve rewarding or punishing the teachers themselves.
The study shows that the envirnment where they were under constant fear of being fired was no worse measured in student performance thatn the prior one. So no that's not the case. Unless you want to go for the wild theory that the study actually did show an improved student score related to firering bad teachers which was counter acted by an opposite effect from fear of being fired.. which had the exact same but opposite magnitude.... that would however be taking Occams razor and throwing it in the ocean.
Your statement in very true in some locales, and not true at all in very many others. Teachers' unions are virtually nonexistent in the US South and Midwest, and only strong in the coastal blue states.
If "strong unions" made teachers lazy/bad, then we'd see poor performance in the strong union states like New Jersey, New York and California, and great teaching performance in the weak union states like Georgia, Louisiana and Mississippi. That... is not what we see.
It also presumes that unionization is the only way to rid yourself of the fear of getting fired. Unionization will alleviate the fear of getting fired based upon factors relating to competence, but another way to do it is simply to increase your competence. I don’t fear getting fired at all, partly because I know I’m competent enough that it’s incredibly unlikely, and partly because I’m competent enough that I know I could simply get another job if I wanted to.
I know that I’m competent, because I have enough experience to understand the problems that exist in my field, as well as how to effectively solve them. There is no randomness or luck involved.
While hard work can certainly allow you to capitalize on opportunities that come along, you have almost zero ability to create opportunities yourself. My current economic status is exactly because I randomly received an offer and worked my butt off to achieve success. Someone I know who is just as competent as myself isn't nearly as successful simply because he hasn't had the same lucky break. Bill Gates and Steve Jobs both worked hard, but both had random chance heavily weighed by who their parents were at birth which allowed them access to computing in a way that wasn't generally available.
This is how people know that the quality of education in the same class can change dramatically when you replace a teacher. But most of them don't know that the same is true in the opposite direction: the quality of education can change dramatically when the same teacher teaches different classes (or just when the most disruptive child in the class happens to be sick at home today).
There is also no teaching method that would make everyone happy. Some students prefer to work in teams, some prefer to work alone. Some students prefer to have explained how things work, some insist on receiving a list of stuff to memorize. Most people who have a recipe "how to fix education" simply want everyone to do it their preferred way.
(So perhaps the best approach would be to have different kinds of school for everyone? Problem is, people would not want to honestly admit some of their preferences. If you listen to internet, everyone prefers understanding to memorization. Then you start explaining stuff, and students follow every sentence you said with "do we have to remember this for the exam?")
Imagine a school in a well-to-do area where we expect all the students to do well and get A+'s. If an entire class of students gets A- on a standard test, that means the teacher actually negatively impacted the students. But because they all got A-, we reward the teacher for having done such a great job.
I propose this: for each student, far ahead of time, guess how well they're going to do on a standardized test based on previous tests, family situation, zip code they live in, etc. Predict based on all the things other than the teacher. Now measure a teacher on how well they did compared to expectations.
To do even better, don't rate teachers based on how well the students do on the test the same year they teach them, but rate them based on the following 3 years. Now you're asking what the long term impact the teacher has on students, on average. That's a useful measure to pay bonuses on.
The problem is that politically, it's hard to explain to the general public (or to politicians).
It seems like there must be a better way to help kids acclimate into adults and give their parents a break/opportunity to work.
Theoretically, this puts teachers on even footing, even when one might be teaching a class of Harvard-prep elementary schoolers and another teaching a class of inner-city kids who have been educationally neglected their entire lives. And data from student performance over time in the current educational system should be entirely sufficient to train the model.
It's not clear if the study referenced controlled for student variance this way; if so, it would readily explain the lack of results.
This headline is pretty click baity. A “better” teacher is one the produces better outcomes for their students. Firing the “bad” teachers, and rewarding the “good” teachers will obviously lead to better outcomes for students. This experiment was only testing whether the metrics they were using truly measured the quality of the teachers, and whether their insentive structure was productive. Turns out they weren’t.
Reminds me of the 10x programmer nonsense. Give people a decent work environment where they feel respected and most will perform well.
It surely isn't telling us what would happen if you replaced all the teachers in a school with teachers recruited from the top 5 or 10 percent of all teachers in the country. I really want to know what would happen if you did that. On the other hand, suppose it works really well. We still can't replace all the teachers in the country with the top 5 or 10 percent of all the teachers in the country, so it's not like that would be actionable.
https://www.theguardian.com/education/2015/jun/17/highly-tra...
From a practical point of view, we necessarily want to know if there is a way to distinguish between teachers at or around an average level, since this is what necessarily applies to most cases. (Research on MOOC approaches, video lectures, etc., could be a separate strand of experiment though, to see if it can leverage the top decile to a broader audience).
This study is saying (modulo the usual limits of generalizing from just one study), that teachers a little above the threshold don’t cause compounding improvements in outcomes over a ~5 year period.
But as always it’s rife with possible issues.
- maybe bonus incentives weren’t high enough to induce any different effort?
- maybe thresholds for distinguishing good and bad teachers were not set right, or need to be dynamically adaptable to different circumstances
- maybe there were just confounders, like the mentioned school merger, making student performance primarily causally driven by non-teacher factors over these years in these locations
- maybe the particular student cohorts just respond differently to teacher-based treatments than students in general would
- maybe students or teachers are affected more by social engagements than learning activities, so that the firing of a beloved, but ineffective, teacher causes a student to remain underperforming, or even resentful of a new, but high-performing teacher.
Whatever the case, this single study does not really offer enough to conclusively (or even suggestively) say that rewarding good teachers or firing bad teachers has no effect.
It had no effect in this case, with this methodology. Would need to see replications and variations of the type of experiment before aligning any beliefs about budgets or teacher incentives towards this result.
My Sister and SO are both teachers and we've had this discussion. They believe that if you took a low performing school, and replaced all the teachers with the best teachers in the country, you would find zero difference.
My sister has been beaten up by students, breaks up fights nearly every day, deals with child protection services, assault from parents, weapons in the classroom, drugs, gang activity, students who haven't eaten or bathed all weekend, etc. She teaches Grade 6. It doesn't matter in the slightest how good of a math teacher she is.
She's has it bad, but I think many people here may be surprised just how common her situation is.
i agree, there's only so much a single teacher, or a school full of teachers can do if a lot of other basic needs aren't being met.
http://www.citevault.com/#vam-efficacy
No, it is impossible. That's the whole point. If it wasn't impossible it'd be a great idea.
Switch 'teacher' with 'professor' and I highly doubt you would argue that quality no longer matters. The difference is that professors are less bound by stringent curriculums and teach more advanced material. This is where everyone really feels the difference. Whereas for public grade school teachers, an academically inclined student will only struggle under exceptionally bad teachers, less well academically inclined students will suffer without decent teachers even in grade school. The first quarter/semester at a good college typically provides a reality check for students who breezed through primary education.
What you could get out of this study is however that it doesn't matter if teachers find the job rewarding or not. It doesn't matter if you keep on bad teachers, in fact the study might point to the fact that teachers are not the bottleneck of student performance at all.
Yes, it gets more complicated looking at more nuanced measures. Like resources available for the students, their home lives and personal situations, their time available to study and focus.
Then again maybe they should look at discrepancies between teachers in districts that pay well and districts that pay less. Perhaps the answer isn't to pay some more and fire others, but rather pay all of them more!
I'm a fan of the Education Endowment Foundation(1) on this which looks at all the measures you can think of to try to improve education practices (in the UK).
In addition to the behind-the-scenes work most teachers have to deal with, such as departmental objectives, I think great teachers enter the profession in order to benefit children, not to get that fantastic paycheck . It's a socially focused decision, not an financial one, so if I were designing a system to grade a teacher's quality I would focus on attending to their efforts with students, not throwing more money at them.
If this conclusion is correct, teachers can be replaced low wage unskilled laborers!
/s
This (possibly MotherJones simplified) conclusion must clearly be bogus. Perhaps I should take a look at the actual study, because this article is terrible.
Teachers Unions are among the core groups supporting democratic and liberal candidates.
I love a good study but it is possible that they are misinterpreting the results of this study on purpose.
During the last election, I became very sensitive to this type of media manipulation. Tread carefully.
At the end of the article, they add that the study deserves a lot of credit even though it didn't work. Kudos! I was pretty happy to read that =).
At least teachers are only in control of one class room. The most mischief they can do is to buy animatronic dinosaurs for the one lesson where they mention prehistory to kindergartners.
Especially if they possess the skills/drive to switch to other more lucrative occupations.
Don't get me wrong though I think that teachers should be well paid, more than enough to live comfortably and happily so they can enjoy what should be a very fulfilling job for them.
Taylor ... was a fraud [1]. Lied about his data in order to get paid more. Even paid the people he "tested his theory on" to act out the outcomes he wanted to see for company higher ups. And he acted like what I've come to expect from managers: first he commits scientific fraud, then writes a book and titles it "Principles of Scientific Management".
Because when you go for bullshit, the big lesson is: don't just bullshit a little. Go all out.
Weber and Fayol were no different, and even Ford ... let's just say that if you focus on his technical accomplishments and skip all the management bullshit you will not be missing much (besides a big part of Ford's ideas were high wages, so I doubt many managers will be repeating it).
As can be seen, among many other places, on wikipedia [2], the fact that Taylorism was an outright fraud has not stopped pretty much every management school in the country from expanding and pushing on with his "work".
It's like most of the famous economists. The top ones were simply frauds, who got important positions like chairman of the Fed, or secretary of the Treasury and just happened to be in office during big events. Their theories were wrong (really fraudulent, as they didn't believe in it at the time, which makes it a fraud rather than just being wrong, and Keynes, Hayek, Friedman and Greenspan have admitted as much, and let's face: they were "the greats". If they were frauds, then people like our current Mnuchin, Powell or Geithner aren't even frauds : they're ignorant and actually proud of that (Mnuchin has said in an interview that not understanding much economics makes him a better secretary of the treasury). Yellen actually had credentials, and seems to need another year or two before she'll confirm she was a fraud too, she has already admitted her theories were wrong, so all she has to do is also admit that she knew they were wrong when she decided to implement them. And boy, she was, like Keynes, more than a little bit wrong).
Once money or power gets involved, humans, "scientist" or not, become lying bastards.
At least this research was honest. But it's not saying what people want to hear, and so it will be ignored, as it wouldn't contribute (and that's being polite) to the careers of the decision makers implementing these policies. Meanwhile, Trump is happily pushing on Obama's efforts in this regard, who simply continued from Bush's policies, ... and so on and so forth. Don't worry: Trump is not about to let such a detail as science stand in the way of looking good to a few people, and let's just not kid ourselves that Hillary or whoever gets elected in 2020 will be better.
[1] https://www.theatlantic.com/magazine/archive/2006/06/the-man...
[2] https://en.wikipedia.org/wiki/Scientific_management#Relation...
I think you underestimate how much teacher unions want to hear this result, and their political influence