There is an easy, quantifiable metric that uses this idea: the Brier score.
Essentially, if you make 100 predictions, the Brier score will measure how well you are able to make precise predictions. If you always estimate the odds that X will win to be 50/50, you won't do nearly as well as someone who correctly predicts the outcome each time.