What happens when we pay public high school teachers based on performance
pubs.aeaweb.org
pubs.aeaweb.org
"Using the expiration of preexisting collective bargaining agreements as a source of exogenous variation in the timing of changes in pay, I show that the introduction of flexible pay raised salaries of high-quality teachers, increased teacher quality (due to the arrival of high-quality teachers from other districts and increased effort), and improved student achievement."
"The main dataset contains information on the universe of Wisconsin teachers, linked to student test scores to calculate teacher VA."
"Student Test Scores and Demographics.—Student-level data include math and reading test scores in the Wisconsin Knowledge and Concepts Examination (WKCE 2007–2014) and the Badger test (2015–2016), for all students in grades 3 to 8, as well as demographic characteristics such as gender, race and ethnicity, socioeconomic (SES) status, migration status, English-learner status, and disability."
What this study is saying is that paying teachers more for improved performance by their students on standardized tests results in higher standardized test scores of their students. That does not surprise me. In my experience, this system encouraged teaching to the test (focusing the majority of instruction on test prep rather than traditional instruction) and widespread cheating. Both of these efforts raise standardized test scores. The study assumes that standardized test scores are a direct measure of teacher quality and student achievement. In my experience that is not true.
Could you give an example of where this has worked?
Apologies for the risk of conflagration :)
Please don't use Hacker News for political or ideological battle.
As for teaching to the test, what's the real downfall of doing that? Colleges use standardized test scores as the biggest or one of the biggest criteria in admission.
This comment is a perfect encapsulation of AI hype.
Recitation is not competence. It doesn't matter if Bobby can say "4" when I say "What is 2 + 2?". What matters is if Bobby can add.
If you try to actually create tests that way, you will discover that the institutions and political atmosphere around you will punish you harshly. It's "unfair" to give a test that you can't pass by memorizing things, you see. And the people who will ultimately make the decisions about what you're allowed to do with your test are rarely going to care much about actual mastery, and often wouldn't be able to recognize mastery if they did care.
In practice, standardized tests are always going to be easily gamed, so if you make people's and institutions' rewards dependent on them, you will end up diverting more time, energy, and other resources away from actually teaching mastery and into meaningless gaming.
No. Most if the time, state laws set limits on what school districts can do.
After so many then failed I told them that just writing the formula would be the bonus question on the next test.
Very few people got the bonus question.
In order to drill students to memorize, that students have to be present enough and interested enough for it to stick...
The class was a required class for teachers to get their education degree. Back then, they had to have that degree in order to teach in the schools.
I doubt that many of them were thrilled for math, especially the more foundational topics we were going through so yes, their intrinsic motivation was mostly not high.
However their extrinsic motivation should have been off the charts. They literally couldn't get their degree without passing the course. Many were repeating the course having failed it before, so their chances were running out.
I do not recall any of the students being the normal 18 to 21 college students - - they were all older. They better understood why they were choosing their degree (looking back, based on demographics, I suspect many were divorcees prepping for a career or moms whose children had started school).
I gave the memorization question as a way to see how much of it was the work they were putting in and much was my inability to explain the difficult material in a way they could understand.
I boiled it down to "just write the equation down" to see if it were my teaching or their desire.
The inability for many to successfully replicate the formula indicated to me that they weren't putting the time in to memorize.
Since these were more adult learners going for their teaching degree, and many repeating the class having failed to pass it the first time, my observations beyond "I wasn't capable" are: - the course material was difficult (a rapid survey of math foundation, including some group theory) - they couldn't make the time because of other life commitments, exacerbated by being a summer course where we met every day.
I had one student, who had never scored higher than 50 on a test, show up after the final to ask if there was anything she could do to pass the class as it was required and she had flunked now 3 times and couldn't repeat it. "All I want to do is teach 3rd grade, I don't need any of this" she explained. There was nothing i could do...
That said, in some circumstances only teaching the test could be an improvement.
"But if we do that," the test manager complained, "the code will only work for the tests."
"Well, wouldn't that be a nice change." observed the Director.
In order for the test scores to be granular enough to be used as measure of performance of individual teachers, the testing would have to be done fairly often. The schools and school districts that are already lacking funds for basic stuff like maintenance and teacher and staff salaries would have hard time to pay an additional subject to administer the test.
With the frequency and volume of testing, it would likely be too costly to do the testing in an external testing facility (which most districts do not have), so it would have to be done on site. I remember my kids telling me that every time before they took a standardized test (it happens once a year around here), the teachers told them basically "we cannot give you the answers, but look around, the answers may already be there, there may be a poster hanging on the wall or something else like that that may be helpful."
So by incentiving the teachers to make the students perform well in some tests, why teach real lessons when you can just throw at them cheat codes for the test.
When I worked for the DoD, most of us were focused on doing the right thing for the country, and we worked diligently towards that.
The pay was mediocre, and potential performance bonuses were minimal.
But the job stability let us focus on the big picture, and we did.
I haven't worked anywhere else that had the same dynamic, even much better paying jobs in the private sector.
Marxism places a lot of emphasis on materialism, which was always one of the biggest targets for its critics. One man's profit is another's stolen labor value. Both systems of economic (that should be the clue here) thought are very materialist and don't do a good enough job assigning value to intrinsic motivation, believing in something, following a passion, etc.
My wife is a special education teacher and so much of her work does not get reflected on a test.
It amazes me that teachers would be so unethical. They seem worse than the general population.
There's nothing about teachers that makes them less ethical than the the average <s>car emissions system engineer</s> <s>KPMG auditor</s> person
One of the big societal problems with pushing down wages for 50 years is that it makes society less ethical because people aren't comfortable enough in their finances. When people aren't comfortable with their finances, they are less likely to be able to quickly ignore unethical things that might help their financial situation even a little bit.
Quick example: You see a guy drop a $20 when stuffing his ATM withdrawal into his wallet. I would say, "hey bud, you dropped a 20." Someone else might just let him walk out and pick it up for themselves because it would help them a lot more than me.
It works, without the students learning more. Our universities are complaining that students arrive that can hardly read even though their grades say they are able to.
And yeah, we have standardised tests administered centrally so we can actually to some degree measure 'grade inflation' too, and know very well that employment/pay based on grading undermines whatever slim value grading would have otherwise.
Edit: Oh, and parents also exert such pressure.
If all it does is pull higher quality teachers from other places, it is zero sum and all you eventually end up with is the same quality schooling for a higher price.
I do think teachers as a whole should be paid well, but that is a separate discussion.
Not part of this discussion, but I did work with people who left public schools to work at charter schools because you could get 25-50% more there, but they rarely stayed more than 2 years because of the working environment being more toxic than at public schools.
I agree that there are problems with test-based evaluation, but overall I believe it's the least bad solution. It gives teachers a tangible incentive to figure out why what they're doing may not be working and try something different.
> This paper has studied the effects of the introduction of flexible pay for pub- lic school teachers on the composition of the workforce, teachers’ effort, and student achievement. A switch away from seniority-based salary schedules toward pay-for-quality in a subset of Wisconsin school districts resulted in high-quality teachers moving to these FP districts and low-quality teachers either moving to dis-tricts which remained with the salary schedules or leaving the public school system altogether. As a result, the composition of the teaching workforce improved in FP districts. Effort exerted by all teachers also increased and, subsequently, test scores improved.
I understand that using test scores makes for easier science, but I grew up with some level of "teach the test" and in my anecdotal experience you turn schools into the dullest cram schools when performance is measured this way. I did fine on tests but I hated it. The only reason I didn't drop out was because I didn't realize I had the agency to do so.
My main question is whether the gains are fundamentally zero-sum, at least in the short term. Some districts implement flexible pay and some don’t, and then the best teachers move to get paid more. And the places that get left behind…?
In the long-run, increased pay ought to lead to more high-quality teachers entering the profession. But in the short term, this scheme seems likely to redistribute talent in ways that reinforce existing patterns of inequality.
Aren't you assuming that rich districts with higher grades implement flexible pay? I think it would be the poor districts with worse grades that would be most likely to want to shake things up.
The issue is that this gets eaten up by bureaucracy and teachers are left living in poverty, hence them clinging to unions like a drowning woman to a straw.
That's a fundamental problem with teacher pay, there are no bonusses, there are hardly incentives or room for raises. It's all just tenure based. Once you're in, you're in.
Performance based pay in professions whose performance is difficult to measure directly leads to bizarre outcomes. I’m not disagreeing in general but I’ve seen this again and again in my career, and I spent a long time on Wall Street where bonuses -really- matter. That extreme brought out the extremes in how incentive pay distorts behaviors in unexpected and undesirable ways, which gets worse the further you get from a directly measurable outcome like PNL.
As a parallel example, hospitals who specialize in extremely difficult diseases with high fatality rates generally have abysmal patient outcome metrics over hospitals that punt anything complex to a specialty hospital. This plays out in policy spaces punishing the speciality hospitals despite the fact they are well known to be top of the industry in terms of performance and “extra effort.” Nominally though they should be shut down by all metrics.
This absolutely plays out in education. Not every classroom or school or district is equivalent in terms of the challenges they deal with. A teacher with a very challenged class who is a high performer and puts in extra efforts will be punished simply because their baseline was much lower than a teacher who punches the clock in a class of affluent students who have private tutors.
There are lots of incentives. If you get a master's degree, your pay automatically goes up!
Does that help anyone but you? Well, it helps the school of education that you paid for your degree. Does it help anyone else? Of course not.
But that doesn't mean there are no incentives. We stated what we wanted, and we got it. We have more incentives than we need.
Think of it as government-performance based teaching quality variations.
>But in the short term, this scheme seems likely to redistribute talent in ways that reinforce existing patterns of inequality.
Only if disadvantaged districts are less likely to switch to performance-based pay. What is the basis for thinking this will happen?
- teaching to the test rather than educating
- trying to get rid of left behind, slow, or difficult students (already an issue for generations in test-oriented private institutions — as opposed to the more remedial “last chance” ones)
- ignoring the groups which have the lowest odds of contributing to the metric (which groups it is depends on the weighting / averaging between pupils)
This is a problem because even a bad teacher is very likely better than no teacher, so we can't actually discard very many of the bottom teachers before making things worse.
The problem is that people in power thinking that this is mostly a teacher driven problem. This would be the same approach as saying that a the amount of money a doctor gets paid should be based on how much weight their patient loses (e.g.) Anyone who thinks teaching is so easy should go try it for a month. This kind of approach also misaligns incentives. The fact is we have good teachers and bad teachers, just like every profession. We need to rethink two things for our education system. Is it to just push kids to college or is it to maximize the number of students who can become gainfully employed and self-supporting adults. Then we need to restructure our resources towards that goal, specifically in terms of class size and ability level and extra help. The main change we need to make is to move investment of resources much more towards elementary age where it can have the longest compounding benefit. (Source: I'm from a family of a dozen current and retired public school teachers)
I do agree that if we increase the pay and benefits for ALL teachers, then we'll create incentive to get better quality teachers in the profession and that competition will get better results in the long term.
Only if school districts that implement flexible pay are also those with higher SES indicators. Which could very well be the case, although the study did try to control for these factors.
I do agree that unless you increase teacher pay vis-a-vis other professions, the effect will mostly be to rob Peter to pay Paul.
Yet she still obsessively talks about how bad the teaching systems is here in Ontario, Canada and follows all the studies.
I could 100% see a large increase in talent/quality jumping into the system and taking over the culture. Or at least heavily influencing it. The gaming for tests is also probably a short term effect as well, the old guard doing what it knows best, just more of it.
But Canada will be the last place this sort of reform would happen. The unions completely dominate discourse.
The Texas Education Agency (TEA) awards school districts for things like students earning industry IT certifications. This can be more than $1k per certification.
Teachers know what will happen if they try to excel in these areas: they will be fired. Coworkers explained this to me, but I did not listen.
I explained to a school board member that I was going to try to get my 9th graders certified. He replied that if even one student failed to earn their certification that I would have failed and he would have to fire me. I tried to reason with the idiot, but he made Dilbert's point haired boss seem competent.
I managed to get about half of my students Microsoft certified.
I left that school district in part because my life was hell there. One secretary in particular was very offended because she had to figure out how to spend the school district's money on certification exam vouchers and had to add students to a field trip to compete in an electronics competition. We took 2nd in the state, and I left the school district. Now I'm an adjunct at a community college.
If you want to fix Texas Public Schools, elect competent school board members.
If you want teachers to perform well, reward them instead of punishing them. When a teacher brings state money into the district, give them at least half of that money.
The problem with "heat pumps" is that they necessitate a "cold" side, from which they "pump the heat" to the "hot" side. Their goal isn't to increase the overall "heat" in the system, it's to move the "hot" all to one side.
A "resistive heater" can add "heat" to the system more evenly, but is less efficient and you won't see "temperatures" rise nearly as quickly.
And that both can occur at the same time.
How would you like to deal with corraling 23'ish hyper nut jobs all at different levels? That you have to be their parents, psychologist, DEI, identity fosterer, special education teachers etc.
Funding in many schools isn't there now for specialists to do these roles.... because teachers wanted to earn more after inflation, so cuts made - more responsibilites to teachers.
So the teachers are having to take on the role of many of society's "dump bucket" of stuff that parents should be taking care of.
This seems as though you're implying that it's the greedy teacher's fault that our school systems shove ever-increasing responsibilities upon them. As if the total pot of money in the system is a fixed, immovable sum, and so that if a teacher wants a raise it must mean that something elsewhere has to give.
Is there any shred of evidence to support that kind of thinking, though? We can quibble about salaries and outlier districts all day long if you want, but fundamentally it's not like teachers are the most highly-paid group of workers out there. And governments manage to find money to increase funding to other systems year after year (the police, for example).
I believe the opposite. I believe teachers are underpaid, not greedy at all, and under staffed. Often education is under funded (in areas of the country such as mine).
Using metrics to understand at a large scale what happen, if a method work or not, where to improve etc might be useful and harmless. Using them as a way to prize or penalize have regularly very bad results.
The ancient quis custodiet ipsos custodes it's equally valid for metrics, who evaluate those who design the measure?
E.g. https://www.bostonfed.org/-/media/Documents/Workingpapers/PD...
Since teaching career progression in the US is mostly tenure based I've always wondered why there wasn't a more longitudinal approach to their assessment.
The product is the students ability to achieve over time. One batch of good test results doesn't measure anything other than the teachers ability to get good outcomes for the test.
Even running a study like this inside the same school you would get wrong data and come to the wrong conclusions, just think about it for a second.
Worse is people wasting time in academia researching ideas with so many holes in it.
This is why measuring schools makes no sense, the creators of the study were likely top students at their time, yet they lack some basic common sense.
Grading is broken, first start by coming with a better alternative
From my reading, the Nordic countries view teaching as a high value profession on par with doctor and lawyer, and this results in potential teachers having to work very hard to land a job and keep it.
According to the main rule of taxation, you're going to get less of what you're taxing. I we can generalize it and say that you'll get more of what you're incentivizing.
Its a valid concern that there are some teachers that are not good, and would be weeded out by performance based metrics. However - everyone else at the school knows who the bad teachers are. They're obvious. They're the ones leaving their class unattended. Doing the bare minimum.
These people can't be weeded out right now because there's a shortage of teachers. The need to keep a warm adult body in that room outweighs the need to discipline and then fire teachers that aren't doing their jobs.
As a thought experiment would we also suggest performance based police salaries? Or do we acknowledge that being a cop potentially sucks and that one aspect of making sure we don't only have bad cops is making the job attractive enough that we're not just picking from the bottom of the barrel.
/edit
To be clear - I'm suggesting we encourage more people to become teachers by paying them more and secondarily making them deal with less bullshit.
I thought one of the causes was unions? qv "rubber rooms" etc.
Looking at other HN stories today, maybe we swap all the Software Developer H1B visas into teaching :-)
You can argue the raw materials are something they can’t control but plenty of other industries have the same issue, real estate, hiring, and we still pay them on performance.
Think we’d see accepting poor behavior, teachers writing off or actively bullying students tank if this were implemented properly.
Funny how that happens.
They do every single day
The key findings of the research include:
Higher Salaries for High-Performing Teachers: Districts that adopted the flexible pay scheme offered higher salaries to high-performing teachers, which attracted quality teachers from districts that did not adopt the scheme.
Improved Teacher Quality: The introduction of performance-based pay resulted in improved overall teacher quality in districts that adopted the new pay scheme. This was due to both the attraction of high-quality teachers and increased effort from current teachers.
Enhanced Student Achievement: There was a noticeable improvement in student achievement in districts with flexible pay schemes. The increase in teacher quality and effort directly contributed to better academic outcomes for students.