If a prediction model was consistently accurate, the creators would be breaking Vegas instead of publishing predictions on a blog.
For a single match, no. Across a sample of predictions? Elementary. (If 60% of the games you give white a 90% chance of winning are, in fact, won by black, you don’t need a lot of games to determine your model is wrong.)
Essentially, if you make 100 predictions, the Brier score will measure how well you are able to make precise predictions. If you always estimate the odds that X will win to be 50/50, you won't do nearly as well as someone who correctly predicts the outcome each time.
> Our findings indicate the probabilistic predictions are highly accurate with a mean absolute error that is comparable to, albeit slightly worse than, Vegas betting lines.
Meanwhile, tons of people are throwing ML models at sports bets, models that they often don't understand, and Vegas is keeping up with it by adjusting the payouts.