So, I had a great teacher. He did not care about grades at all. He would give bad grades, and when kids would complain give them good ones if they wanted to retest. He would play games with students regarding this. Now the students who wanted good marks to count towards proficiency were really pissed. He put it quite simply: "I'll give you all the A's you want; but when you go back you will still not speak the language. I do not really care and that is not the focus of my classes. I am here to teach and discuss the language and material with you. That is what interests me." I was one of the few that enjoyed that course, and I was admittedly the best student. I continued to be a strong student in all courses, even those I failed (like economics and computer science) because I was far more interested in material than grades.
In the end, I graduated with a high GPA. I see this as a cautionary tale. Now if only I could finally learn C and submit kernel patches like I promised myself in college. But such is life.
It is funny people laugh when they hear me bust out into Arabic in front of them. I told them I am a language major by trade, and did IT in college to make money. Many people congratulate on mastering these "magical bodies of knowledge that are witchcraft, because surely I (the person I am helping) could never get it." If you want scary, these are other people married to Arab men and women, as am I. Decades of exposure mean nothing.
Now why is that? Grades breed mediocrity. Good students stroke their ego, and most students give up when they cannot compete with good students. So we encourage people to believe "I am a magician that speaks computer" and "Arabic is impossible" (if you beleive that, try teaching and explaining inconsistent English grammar; Arabic has encoded in the morphology a lot of syntax that would scare you into seeing what is like a human programming language) and accept mediocrity because evaluation and rank is more important than effort. I see it so often, even from students, I mess with them when they come to my office. I never bitched to change my grades. Why? I was busy reading. Who cares!
Please keep being that professor, I hope you save some of them. What is the most cautionary part of that tale is the people in my story all end working for government, some as counter-terrorism and intel guys. They cannot string a sentence together in Arabic, despite the A, and dictate USG policy "on dem terrorist A-rabs" so this culture will get us a real far.
It matters a lot when student wants to study medicine. Law firms look at grades when selecting graduates. If you want to go to grad school, your grades matter a lot too.
So, in a sense, grades are real competition.
A reason why an effective team needs a few older guys who've been around the block a few times :-)
KPIs are just that. Indicators. Temperature is an indicator of good climate. If someone's answer to "it is cold in here" is to put a lighter under the thermometer then the problem isn't the thermometer, right? It is the people who are confusing indicators from goals. And that is a culture thing.
If that is your climate though you can still recover from it (I know we are doing way better than before at least:)).
First steps are to embrace ambiguity and uncertainty in evaluations. The culture twist (when I have seen it) comes from "Noone recognises my work" -> "Lets define KPIs for your work" -> "I am not evaluated fairly, look at how much work I do compared to guys that are evaluated better" -> "Lets define guidelines for evaluations based on our KPIs" -> "Everyone (except me) is cheating to score high on the KPIs to get better evaluations" -> "We need better KPIs that can't be cheated on" -> "We are doing a whole amount of stupid stuff because people worry about KPIs".
I think the first two steps are fine. There is direct business value in being able to compare how you are doing compared to the previous month, or some other department or whatever.
The problem comes in the third step - once the KPIs determine how someone is evaluated, then you are in a downwards spiral, because now you are saying that how you get there isn't as important as getting there. To get out of that you need to embrace the ambiguity that comes with letting evaluations happen, and maybe be "unfair".
Basically - uncertainty in evaluations is better than defining evaluations on KPIs. Solve the uncertainly by more discussions and reasoning, not by more rules.
But you're not using indicators to do some kind of science, you're trying to use them to make decisions about who gets fired and who gets a raise.
If you're a self-interested human being, you're going to do whatever you can to make sure the indicator points to "give me a raise" not "throw my ass on the street." If the indicator is not a perfect measure of job performance, then you're going to let job performance slide to maximize the indicator.
Thats the thing - the indicators when used right are used for science. Namely improvement of self or others. It gets harmful when they are solely used to determine if you should get a raise or get fired.
They can certainly be used to indicate problem areas i.e. "This guy is producing 50% less according to RANDOMKPI", does that make sense or do we need to fix something? But if they are used like "X is at 80% of RANDOMKPI, he will be fired if he is not at 90%" then they are harmful, obviously, because they dont take into account the much more important things like "X is at 80% of RANDOMKPI because he is unofficially the teacher of how to deal with anything in our payment systems, so he spends half his time helping out others, which adds way more value to us than if he was just implementing features" or whatever.
Once KPIs start to drive evaluations, you're down a slippery slope.
Having to suck it up and deal right now. I appreciate you reminding me that KPIs are just a tool. :)
This sounds like a cheating problem, not a measuring problem. The same things happens when people try to measure schools.
My sense is you can't give up on measuring things, because you can't scale an organization without metrics. You just have to balance them with understanding what is really going on. Or rather you need to have the metrics tell us "What's going on with the business" rather than "What's going on with the individual".
The real issue is that identifying the correct things to measure and reward is really, really hard.
I read once about a factory that was trying to improve output so they started rewarding shifts of workers for reaching production goals. But to achieve those goals workers started cutting corners and quality went down. So the company installed a device to measure the quality of the widgets, which worked for a while but then quality went down again. The workers had figured out they could just keep measuring the same in-spec widget over and over again to make the machine happy. Next the company altered the quality measurement machine so that it would issue an error if it did two identical measurements in a row. Again quality went up for a while until the workers figured out they could just rotate a couple of good parts through the machine to fool it.
The TL;DR version is that any job worth doing has some hard to measure components, so the best you will get using measures as incentives is to lag in the hard to measure parts. So the recommendation is not to stop measuring, but to make individual measurements invisible to management, and be just tools that people can use to improve their own performance. If your team is doing far worse than average in a measurement that you all find important, you will work on it anyway.
I think you have this a bit backwards - the cheating is actually the symptom of people trying to measure stuff in schools and then make those measures directly affect social status and future success of students. Cheating exists because the system doesn't reward you for learning the material, but for passing a test.
Here is one example of many -> http://www.cnn.com/2013/04/02/justice/georgia-cheating-scand...
They would apply the scores to the metrics used to determine suitability for promotion, probably because there isn't much you can measure to measure the performance of a guy who sits in a hole all day.
You make an unfair comparison, however. Numerous factors that constrain ant behavior (brain size, genetics, sterility, energy requirements, etc.) simply do not obtain for an organization of unrelated human beings.
- lower system downtime = don't do maintenance or do it in a half-assed way
- decrease system load time = make everything possible "optional post-load components", thus reducing load time by factor of, say, 10, and increasing time to combat readiness by a factor of 5.
People can pervert any metric they're measured by. Just look at what companies do to squeeze that last little dollar of profit out of their customers.
Short term that looks pretty good though, longer term it is a guaranteed to fail strategy. When and if it catches up with you you're deep trouble.
It's far from perfect but making things quantifiable does have a lot of benefits.
Your combination of multiple KPI amounts to somewhat more complicated KPI. There will be perverse incentives unless you get perfectly right. And that is possible only if the work, including the long term consequences of decisions, are easily quantifiable - which is not the case of most non-trivial jobs.
I think it is rather simplistic to say that having more than one KPI can be reduced to "a more complicate KPI". Sure, it won't solve all problems with badly setup KPIs but to allow for detailed incentive structures you need detailed KPIs.
My major point is still that decision making based on KPIs should not be made solely on what the figures (or any red/yellow/green flag next to it) show. It requires analysis - which is something that I've always seen done.
The good thing about KPIs is that it can be a great communication tool and open discusions about why a trend has changed or similar things.
The problem is that KPIs are misused like, 99.9% of the times, starting from elementary school. It's rare to see someone using such scores as (one of many) evaluation input and not an output. It's good that you put effort into doing real analysis; sadly, not many people do that.
I think that KPIs, or grades, let you as a manager/evaluator be lazy. You don't have to think, to investigate why the scores are bad, to talk with poor performers. Probably you'll be even discouraged from doing so by your superiors / the whole system - you are not allowed to completely disregard the scores at your discretion.
In my experience, KPIs work better as a tool for communication and as input for analysis rather than for optimization of goals. In a previous job I worked with aggregating the Balanced Scorecard for a 10,000 employee corporation. On such an aggregate level where the KPIs are composites of many underlying decisions there is less of a risk of perverse incentives. Instead, they are used for comparing trends across time and similar purposes.
KPIs targeted at individuals or smaller teams are very different than that example though.
I wouldn't put new features online.
I general I would do little that might cause the system to be unstable, even if it was a actually a good thing.
This applies to all of us.
http://blog.deming.org/2013/02/the-idea-of-performance-ratin...
http://www.folklore.org/StoryView.py?story=Negative_2000_Lin...