Hacker News Reader

Comment by diob | Hacker News Reader

Check out "The Tyranny of Metrics" book for why this won't work.

Weird to say it's simple.

I’m sure I already know the answers from experience after becoming obsessed with KPIs as the solution to startup problems to seeing how companies just game vanity metrics. But could you provide some examples of why it doesn’t work in a school environment?

My best guess: It is possible to weed out non-productive KPIs, they are still extremely useful for breaking down reality distortion fields, but it takes an honest and thoughtful person with a good grasp of both the business and how the stats are collected - combined with management buy in. I have a feeling highly bureaucratic gov organizations won’t have either the talent, iterative approach, nor incentives to actually find the right metrics that matter.

It's too variable. A teacher has a variety of classes with a variety of children that changes completely every year. The outliers are enough to impact the metrics significantly but leaving them out paints an unclear picture. As well you have additional high variability of an individual students performance at any given time. This variability is greatest in the non-outliers which matter the most.

I answered that in the OP.

Metrics work when kids are prepared to learn in school, but many are not. Black boys in our worst neighborhoods have a >5% chance of being shot at least once by age 21. In the worst cities, 1/200 Black men and boys are shot every year. If you consider only those aged 13-55 in the worst areas it's more like 1/100 -- per year. This is not a simple environment to educate in. Kids don't have a safe place to play. Kids are in gangs. Kids are obese. Kids don't go to the dentist. Kids fight. Kids have high lead levels, asthma, and various trauma.

I don't follow your point. Are you saying we shouldn't commit to education metrics because a subset of the population aren't capable of fully engaging with education system? What about the kids who are engaged in education? Should they have to put their systems on hold or change it as a way to help(?) those who can't fully participate?

Those seem like two different problems with two different solutions. For ex: solving the environmental issues for the smaller subset and better correlating grades with actual outcomes in education for everyone. Especially if it risks masking problems and may ultimately harm more than it helps.

Oh I think we are referring to two different things. One poster suggested that performance-based pay for teachers is a simple solution and that it could be implemented with metrics. I countered because that is not simple. In addition, many teachers are doing support roles, library, phys ed, social work, reading specialist, etc.

OTOH metrics and objective standards are definitely worthwhile indicators. Agreed. In fact, in the modern age of data I think we ought to look at more things to see if there is a latent pattern positive or negative. Reading and math scores are not enough. I want to see physical fitness data. I want to know how many words per minute kids can type. How many dental cavities do they have. Can they pass a civics quiz on state or federal government. Do they know basic geography and have some literacy about the world.

Make it a sliding scale, performance incentives in the tougher inner city schools should be greater than a easy suburban school.

Yep, or target the incentives based on the composition of the situation on the ground. If your classroom is mixed with 1/3 IEPs for autism and other issues you have different achievable goals.

But this would affect any method. So its not really relevant when deciding a method right?

Not sure what you mean. You don't want disincentives for teachers -- it's already an issue where good teachers trying to improve rough schools get associated with poor performance through no fault of their own.

Incentives need to be based on obtainable goals and targeted improvements. Payouts (bribes!) to students for good grades and scores actually really work. Money helps kids internalize the societal value placed on their performance. Similarly, targeted bonuses for teachers work too, but only if formulated relative to the characteristics of enrolled students. See the literature by R. Fryer, for example

Math tests are hard to game.

The current metric is gamed pretty thoroughly, wouldn't you say?

Besides, businesses operate on metrics every day. It's the bottom line. It works. You just have to pick metrics that are the goal of the organization.

Reply on news.ycombinator.com