Reinventing the wheel would be a major problem if our goal is to solve the education problem with this discussion. No one here has done even the basic work I would expect of someone trying to understand teacher evaluation as a solution and compare it to other alternatives.

I would argue that the whole point of HN is to think through arguments in other domains and build intuition by reasoning through problems and arguments. Otherwise, what's the point? No one is going to arrive at this thread and scan the top-rated comments for the solution to his school district's problems.

In cases where actual decisions are being made where the analyses are much more thorough and fully validating much more expensive, other techniques are available. First, one generally builds an awareness of the strengths of each team member which suggests where errors may be more likely. Additionally, one can check a random set of the most likely problem areas. Perhaps, most importantly, while everyone won't re-do every analysis, it's highly unlikely that an any important analysis will only be done once. So one can expect that the high-level results are generally consistent.