"Measuring the wrong thing can be worse than not measuring at all."
I'm calling bullshit on that one.
Yes, measuring the wrong thing can incentivize poor outcomes, but movement in the wrong direction is visible and can be corrected, whereas lethargic inaction just lazily drifts into a morass.
You have to measure outcome. Period. We can have great discussions around what kinds of measurements to have and how to apply them, but money should not go out the door without some indication that something is happening as a result of it -- even the wrong thing. In fact, big visible indicators of the wrong thing happening as a result of measurements is a terrific result. If we're arguing over measurement, the game is already lost.
Another way of stating this is that let's say I pick a measurement to implement. It's a terrible measurement, and it causes much pain and suffering. You come to me and say "This is a terrible measurement and causes much pain and suffering!" To which I respond "How do you know this?"
So you show me a study -- another form of measurement! Then we have a conversation about including parts of your study in my measurement, scrapping my measurement entirely, and so forth.
We cannot have meaningful conversations about management without measuring things. Once we start measuring things, the world is wide open about which things to measure. But if you convince me that the act of measuring itself is broken, we have no further basis for any conversation around performance at all. Management has no function. There is no quantifiable return for any possible amount of money I might invest. Very bad.
We don't need to wait around to pick the perfect measurement or do nothing at all. We need to pick measurements -- any measurements -- and rapidly iterate towards finding what works best for us.
If the testing system is broken, and I have no doubt that it is, it's broken based on bureaucrats creating a large, brittle, hard-to-change architecture of testing, instead of just trying to continuously evolve what works best. It has failed because we put the same idiots in charge of school testing that were creating the mess the tests were supposed to fix in the first place.