> they tend to become a terrible metric

Not exactly. When a test becomes study-able, it becomes a good metric for how hard people try and how much time/effort people put into their studies/career. And that is arguably what people making tests were trying to or would want to measure.