Paying teachers more results in higher pupil performance | Hacker News Reader

Paying teachers more results in higher pupil performance

blogs.lse.ac.uk

35 pointsrobg35 comments

This is not time-series data. The study design here (a cross-sectional survey of varying countries, showing a bare correlation between two variables) is not adequate to show causation.

http://norvig.com/experiment-design.html

(By the way, the scatter of data points around the regression line in their plot suggests that the model is subject to large degrees of error in prediction.) It would take an experimental design (randomly assigning one group of teachers in the same country to receive pay raises while another group does not, with before-and-after comparisons of pupil performance) to show that paying teachers more results in higher pupil performance.

http://escholarship.org/uc/item/6hb3k0nz

There have been hundreds of studies of educational interventions over the years,

http://www.amazon.com/Visible-Learning-Synthesis-Meta-Analys...

and many thoughtful international comparisons of teaching practice,

http://www.amazon.com/Teaching-Gap-Improving-Education-Class...

http://www.amazon.com/Knowing-Teaching-Elementary-Mathematic...

http://www.amazon.com/Making-Learning-Whole-Principles-Trans...

but none of those conclude that simply raising teacher pay, without changing teaching practices and perhaps also the composition of the teaching workforce, will have much to do with raising pupil performance in any place. Raising teacher pay systematically has been tried in the United States (notably in the state of Connecticut) and has not been shown to markedly raise pupil performance.

An economist who closely studies education policy has suggested that pay and other incentives be used to encourage the least effective teachers to seek other occupations while rewarding the most effective teachers with increased compensation and more professional support.

http://edpro.stanford.edu/hanushek/admin/pages/files/uploads...

Such a policy, he estimates (showing his work in his article) would raise United States educational achievement to the level of the highest-performing countries. This is something worth verifying by experiment, although that will be politically difficult in any state of the United States

http://www.cato.org/pubs/journal/cj30n1/cj30n1-8.pdf

and perhaps in Britain as well.

http://www.economist.com/node/17849199

P.S. I'm curious about why the United States underperforms so much compared to salaries paid to teachers in the chart shown in the submitted blog post.

Monetary reward could destroy intrinsic motivation http://www.youtube.com/watch?v=u6XAPnuFjJc

I'm not sure about that. Doctors get paid a lot, but if you talk to them very few cite money as the reason they continue to practice medicine. I think increasing the compensation granted teachers could pull in talented folks who would make good teachers but also have other things that they like to do and are good at.

I know in my highschool all our science teachers had previous 'job' training. The chemistry teacher worked down in mines in (IIRC) Australia. The physics teacher had worked at a lab doing radiological work. The biology teacher actually went from field work, to university professor to highschool vice principal.

That sort of thing is strongly discouraged in the US by teacher pay structures.

My mum is a doctor and a teacher. As in, part time practicing MD (mostly women's health). She gets a decent salary teaching in the private sector, but only really does it because she likes teaching and hates working with sick people. In the public sector, they offered her a very low salary, and told her to get her paperwork together to prove she was senior enough (after over a decade of teaching, but she hadn't kept every pay stubb).

From tokenadult' comment:

Raising teacher pay systematically has been tried in the United States (notably in the state of Connecticut) and has not been shown to markedly raise pupil performance. </quote>

See the video from my comment above to understand why the following obvious suggestion might not work in practice:

> P.S. I'm curious about why the United States underperforms so much compared to salaries paid to teachers in the chart shown in the submitted blog post.

Because in the US if you flunk out of high school you can still get a job at a gas station and supplement it with welfare and food programs.

In other countries if you flunk out of school you starve to death.

Yep, no welfare state in Finland or the Netherlands. I hear over there if you break a leg they just shoot you.

Everyone in the US has the opportunity (or close to everyone) to attend a near-standard public school in the US. Not all countries subscribe to this model, I believe. Although the US average is low, I strongly suspect that the top 10% of US students perform at or above the levels of all other countries.

As for paying more, this increases competition as stated, but you'd need all other factors to remain equal. The class size in US public schools is now ridiculous, 30+ kids per teacher at the local elementary here. Education funding should basically be doubled imho, and the number of teachers per student dropped to 17 or 18 maximum.

Agree. Triple it, 10 students per teacher, and see performance shoot up.

Yeah, my immediate reaction was that it looked like a practically uncorrelated scatter plot with most of the correlation signal coming from a few outliers on the high/low pay ends. At $35k, the outcomes scatter from 100% to 35%, which would be the same magnitude as raising pay from $15k to $55k according to their fit.

There are so many systemic differences between those countries that could correlate with pay that interpreting it is pointless without controls. Not to mention that there's a difference between test performance and learning, there could be "optimization by proxy" effects in play.

This seems like they walked in looking for proof of their already established conclusion. If you look at Finland, Australia, Korea, Japan and New Zealand you see a cluster of countries who are paying their teachers well and who aren't getting results (Edit: By this I mean the countries below them aren't getting results yet are paying the same amount). Compare Denmark to Finland or Spain to New Zealand and then explain to me how the authors' thesis rings true.

Plus we have anecdotal evidence that contradicts their conclusion in the U.S: Individual State salaries vary pretty significantly ($37,000 to $59,000) yet education results don't seem to reflect those salaries

Teacher Salary By State: http://www.teacherportal.com/teacher-salaries-by-state

State Education Rankings: http://www.vermontbiz.com/news/september/state-education-ran...

I don't entirely disagree with the thesis that making "teacher" a more revered position will draw better people to it. But there are clearly other factors at play that the authors chose to disregard because they wanted to prove their already established conclusion.

(If it were me I'd be comparing Switzerland, Germany and Sweden and trying to figure what the differences are there)

"If you look at Finland... paying their teachers well and who aren't getting results."

I'd been given to understand Finland is generally regarded as one of the better education systems in the world.

http://news.bbc.co.uk/2/hi/8601207.stm http://siteselection.com/ssinsider/snapshot/sf011210.htm http://schoolmatters.knoxnews.com/forum/topics/how-does-finl...

You could very well be right. My point was the authors' conclusion didn't match their underlying data. So the validity of their underlying data didn't come into play.

> I'd been given to understand Finland is generally regarded as one of the better education systems in the world.

Of course Finland (and most Northern European countries) has another advantage - they're teaching Finns.

Pretty much anything you do in those monocultures "works". So, their success with any program tells us nothing about how the same program will work in a cultures where things do fail, let alone diverse environments.

You read the graph wrong. Finland, Australia, Korea, Japan and New Zealand get better results for the same amount of pay. The farther above the line in their graph, the better the relative performance of the teachers given their income and vise versa, the countries below the graph have worse results given their income.

I think I phrased it badly but the point still stands. The authors point was that teachers pay is the deciding factor in results. My point was those countries pay the same Denmark and the UK yet are getting better results. So pay is not the deciding factor.

Let's be clear: both explanations are that higher salaries result in better teachers which in turn results in higher student performance. It's not that saying that paying a teacher more results in higher performance.

My first thought when I read the article title is that teachers would have more money to pay out of pocket to get materials for their students.

A lot of teachers do this, from kindergarten to high school. I've had to hang out in a educational supply store more than once in my life when I and <pick any friend that is a teacher> have passed by one. That teacher then blows a sizable chunk of coin without fail.

Arts & crafts, books, lined paper, sci calculators, and of course... stickers for that job well done.

Some US states have 60/40 plans built into their education budgets. What that means is 60% of the money the state gives to local schools must go to the "classroom." Now, how a state defines classroom is very loose. Most of the time, supplies, working facilities, new desks, books, and teaching materials fit into it. What is the other 40%? Administrative and teacher salaries, lunch programs, and transportation.

In my experience, schools in the US actually waste a huge sum of money because they have incompetent administrative staff (with the added bonus that they're overpaid).

I found the conclusion of the movie "Waiting for Superman" interesting basically the major obstacle for improving education in the US is that you can't fire bad teachers because of union contracts.

Well Yes but there's a backstory to that.

If you talk to teachers who worked in the pre-union days they'll tell you it was a nightmare. Because it was run by low-level politicians (school board members) who were cowardly to the point of causing havok.

Put it this way. Most people don't know the political positions of their individual school board members. So if someone comes to their house and says "this member is bad" they'll generally vote against them on the theory of "better safe than sorry". Which meant every crazy parent who thought their kid was treated unfairly would raise a fuss and get teachers fired because the Principal answered to the School Board members and those members had extremely tenuous holds on their position. Meaning teachers were getting fired fairly regularly, often for doing nothing wrong.

Teachers Unions were born out of that abuse. The problem now is we've swung to the other extreme. Where even teachers who are bad can't get fired. But I wouldn't say the problem is the union contracts. If anything it's that we haven't found an appropriate metric for measuring teacher performance.

If anything it's that we haven't found an appropriate metric for measuring teacher performance.

Modern VAM (Value Added Modelling) methods tend to work well. This is basically building a statistical predictor of student performance, and then measuring whether individual teachers students meet this predictor.

The teachers unions fight tooth and nail against using them, but they work well.

Care to provide some evidence for this assertion?

I am aware of the some of the research into building multilevel models of student and teacher performance, but one of the issues here is that if these metrics are used, there is no test set which can be utilised to validate the model. This is an absolutely huge issue when you are essentially hiring and firing people based on the metric.

...there is no test set which can be utilised to validate the model.

The collection of students used to develop the model are randomly partitioned - one partition is training data, the other validation data. This is regression 101.

This publication goes into detail: http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf

Note that if you believe VAM doesn't work, you must also believe that we have no statistical evidence that teacher quality matters at all. Is that actually your belief?

"student performance"

In practice, this means "standardized test scores." Does anyone actually think that's a good metric? I'll bet you most teachers don't think so.

And then I see stuff like this: http://voices.washingtonpost.com/answer-sheet/teachers/new-s.... This links to an EPI study that casts a substantial amount of doubt on VAM: http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf.

Among other things, it turns out there's a lot of fluctuation of how much "value" a teacher adds from year to year. Teachers in a very high percentile one year end up in the bottom 40% in another year, for example.

Actually, downthread you provided a link (http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf), which says among other things:

"At the current time, VAM may show promise for lower-stakes, diagnostic purposes. Examples include identifying teachers who might be low or high performing so that follow-ups can be done to verify the VAM findings. Inferences would need to be circumspect because of possible bias or sensitivity to the measure, but they could be a starting point for administrators (such as principals or superintendents) to target teachers for more thorough review."

There's this, too:

"The research base is currently insufficient to support the use of VAM for high-stakes decisions. We have identified numerous possible sources of error in teacher effects and any attempt to use VAM estimates for high-stakes decisions must be informed by an understanding of these potential error."

This is far more circumspect than "it works well!" Nevertheless it sounds to me like teachers' VA scores are effectively treated as grades, which is not consistent with the report you linked, nor the EPI report.

In practice, this means "standardized test scores." Does anyone actually think that's a good metric

If it isn't, we've got bigger problems - specifically the fact that we have no ability whatsoever to evaluate whether a school is doing a good job.

In that case, we need to fix our measurement process and waste as little money on unproven methods in the meantime.

Among other things, it turns out there's a lot of fluctuation of how much "value" a teacher adds from year to year.

If this is due to statistical noise, then teacher evaluation needs to be done on a multi-year basis. If this is due to internal variation in teacher quality, then maybe we need to either control the variation or accept that teacher quality varies widely and is mostly out of our control.

Note that if VAM doesn't work, then all the statistical studies claiming to show that teacher quality matters for student achievement are bunk. All those studies make some attempt at VAM, and show the variation due to teacher quality is large. Are you asserting that this is the case?

Eric Hanushek and Steven Rivkin's conference paper "Generalizations about Using Value-Added Measures of Teacher Quality"

http://edpro.stanford.edu/hanushek/admin/pages/files/uploads...

explores several of the theoretical issues and facts that are simply not known at present about how to evaluate teachers. The paper points out that for public policy, if existing models for teacher evaluation programs by value added are better than what is currently done to evaluate teachers, even if the value-added estimates are not perfect, then they may be worth implementing more generally. This definitely is an issue that deserves further research. On my part, I think a truly effective teacher is worth her weight in gold, and I would like to see the system identify and reward the best teachers, not least because then other teachers will be likely to learn from their techniques and emulate their examples.

I know everyone seems to want to pay teachers a higher salary, but in most school systems, they are already guaranteed a pay raise every year through the unions. Even if they did nothing to deserve it.

Most jobs tech jobs start out high, but plateau at a certain salary range (you also don't automatically get a raise every year). If you are a teacher long enough (10+ years), you will get up to the $70-80K range (I know a couple of older teachers that get paid this much). With summers and many more vacation days, it evens out in the end.

If you want better performance, we need better teachers. Because the unions will not allow us to put a measure on performance, the result is bad teachers getting into the system with the inability to fire them.

I'm also finally glad they are putting an end to "rubber rooms". If you don't know what they are, take a look here:

http://www.nytimes.com/2010/06/29/education/29rubber.html

Another waste of our tax dollars that has been going on for years....

I agree with the general notion but don't think raising all teachers' salary will lead to better result. While Korean public school teachers are relatively well paid, top private 'Hakwon' teachers are paid several times more. Private education is a hot 'business' in S.Korea.

Sad part is that the goal of private education in S.Korea is to enter good universities, not to get good education. Sadder part is, after graduating, many can't find jobs and those that do usually don't work in the field they majored in because they picked their school by reputation and their majors by ease of entry.

I bet that the correlation is much stronger for teacher starting salaries than for the 15 year in salaries that the study mentioned. What really matters is attracting the best people to be teachers, and when people look at jobs they mostly look at starting salaries.

But for some reason in the US we tend to have a very steep seniority based pay scale, with a far greater inequality between old and young teachers than you see in pretty much any other industry. You could argue that more experienced teachers are better, but studies show that experience stops mattering after 5 years, and it was also my subjective experience in school that older teachers didn't seem to be any better at teaching.

UK note: over the last couple of decades there has been a lot of effort to 'raise standards' (contested vocabulary) in teaching over here. Partially pay awards, but also degree only recruitment, only certain kinds of degree accepted for Secondary school subject specialism, staff development requirements &c

Public sector pay and conditions will drop relative to private sector in the UK over the next 10 years or so. If and when the business cycle picks up, you might see the 'quality' (contested vocabulary) drop again.

Figure 1 above provides an insight into the relationship between teacher salaries and pupil outcomes, showing a clear statistical association between higher relative teachers’ pay and higher standardised pupil scores across countries.

Aaaah! Correlation again being confused with causation. Omitted variable bias!

They do this in China. IMHO, it is a pretty mixed bag. Mostly they teach kids how to take tests.

Reply on news.ycombinator.com