I've been hearing loads of reports of kids getting marks way off what was predicted, and clearly the algorithm is... wrong. This essay is a good explanation of why. One thing that annoys me a bit is that we as a society have discovered all sorts of interesting thing about statistics and decision theory, causality, etc, but then we don't apply them. Why didn't Ofqual find some people who understood how stats work, and and ask them for advice?

It seems clear that if you just take a distribution from previous years and lay them on top of a ranking, you are creating a lot of problems. And you are hiding your incentives on the penalty function, which is something he mentioned. People have correctly figured out that a good kid at a bad school will not be found, and that the marginally second best kid in a year is going to get marked down.

Whatever you think of exams, and I'm no fan, they do give people that mix of noise and signal that society needs to put to rest the issue of who deserves what. They're not entirely a lottery, not entirely skill or hard work, but if we need a way to distribute X number of future doctors, that's the long established way to do it.