Hacker News Reader

Right, but say in 2016 you gave a 1/3 chance to Trump, 1/3 chance to Clinton, and 1/3 chance to Vermin Supreme. Exactly one of them will win, so your calibration for 1/3 will be perfect. If you do the same for elections with different numbers candidates, you can have good calibration overall without actually producing any information over the base rate.

View on HN

I disagree. Suppose you have an election with 3 candidates that runs every year. If you predict candidate A win at 33% confidence, but that candidate gets elected every year, your calibration will be way off. Calibration is about aggregating, so you can't really cheat it. Your 33% prediction has to be right 33% of the time - it's really hard to fake that.

Maybe I'm misunderstanding your position, though.

That's right if you only predict the probability of candidate A, but if you are predicting 33% for all three candidates each year, your 33% prediction will consistently be right 1/3 of the time in aggregate.

Reply on news.ycombinator.com