Hacker News Reader

I don't think there's an expectation that a skill rating is comparable throughout 20 years, because both individual players and how the game is played (the meta) changes continuously.

But if that's true, then why would rating inflation be a problem?

The game itself has not changed, so it still makes sense to compare players across time. It would be nice if we had a quantitative way of doing this; so we can make statements like 'the average proffessional player today is better than 20 years ago, a typical modern pro would win 60% of the time again one from 20 years ago).

In some sense, it is not surprising that we do not have a system that accomplishes this. Since it is impossible to see the results of a game between players living in different time periods, we cannot get any data to prevent drift. You can still try to normalize the rankings. However, unless you have some independent way of measuring skill, you would need to make an assumption about the relative strength of players. Assuming the average skill of a proffesional is constant across time is probably not accurate, but closer to reality than what you get with unchecked inflation.

the game itself has changed quite a bit, and the number of people playing it, and the dominance that they achieve has also gone up quite a bit.

You can sort of solve the inflation problem by zscoring the elo. Now a person's score will tell you how much better or worse they are than the median player, assuming an underlying normal distribution (reasonable).

Of course, scores will only be comparable if the average skill of all players remain constant. I would imagine this isn't true, but the drift over several decades is probably small.

Unless you start introducing some purely objective criteria for skill, which can never work, this is the best you can do. It's still way way better than a straight elo system though.

Rating distributions are often not normal because some subset of players study the game and take it more seriously resulting in a bimodal distribution. See [0] for an example in Chess.

[0] https://chess.stackexchange.com/questions/2550/whats-the-ave...

That's fascinating, thanks! It looks like you can model it as a mixture distribution made up of two underlying normal distributions.

Even without the bimodality, you wouldn't expect a normal distribution of ratings.

1. Assume that chess ability is normally distributed in the population.

2. Assume that people who are terrible at chess are more likely to stop playing chess than people who are successful.

Then you've sampled the underlying normal distribution mostly from the top end, and that new, highly skewed distribution is what you'll see when you measure everyone's rating.

The idea that chess has not changed in a long time is simply not true. Two huge and relatively recent changes were the addition of chess clocks and premoves.

And aside from the mechanics of how the game is played, there have been massive changes in the popularity of chess (first massively upwards, recently possibly down slightly), as well as how analyses are done.

It would be very difficult to account for these factors in a way that keeps comparisons across 30-year+ time spans meaningful.

This might not be great for a sporty-sport, but I think that for a video game this would actually be an advantage. This kind of a rating inflation would mean that long-term players would see some numerical progress without really doing much better.

It would also inflate the ratings of people new to the game later in time.

A newbie is started off with some nominal rating; I forget the number, but let's say it's 800. Most likely that newbie is going to lose his first matches, and some proportion of those newbies will get frustrated and quit

That seems like a simple problem to fix. When somebody quits, just subtract 800 points from the remaining ranked players, scaled accordingly such that their relative win probabilities remain the same.

Of course, the other issue is if the number of active players increases over time. In that case, it's not so easy to fix unless you start scaling down the number of starting points given to new players.

Perhaps a better thing to do would be to construct a model of the rating inflation over time and use that to correct for historical comparisons. It's still not particularly meaningful though, because you have no way to measure actual skill inflation.

You don't have to formally quit the game to stop playing. I played one ranked chess tournament in high school, quit for ten years, and then picked it back up. What would you do with my points?

If you choose to delete them, that means that everyone will have constantly eroding ratings unless they keep playing.

Your points could be added back in when you resume playing. There’s no reason to throw the data away.

> It doesn't suffer from the weaknesses that you cite, but even so, the problem of "rating inflation" is widely discussed.

Ah yes! Inflation is also a problem I've seen in competitive online games. Rating inflation was a serious issue with World of Warcraft PvP arenas circa 10 years ago (iirc Blizzard hard capped arena ratings at 3000 during WotLK). I don't follow chess much, and I'm not exactly sure how chess avoids it (or even if it does).

By the point you're playing ranked matches in chess, you're generally invested enough to keep playing. However, chess has a (statistically) significant inflation problem, to the point where you can only compare scores within the same decade or so meaningfully.

It seems there was a lot of rating inflation in chess, but at the top level, at least, it's stopped - the number of players over 2700 has been pretty constant for 5-10 years, a few dozen players. In 1990, only Kasparov and Karpov were rated over 2700.

https://2700chess.com/

https://en.wikipedia.org/wiki/1990_in_chess

There's also an inherent deflation effect. Players tend to get better over time. In the simplest case, if we start with a pool of players rated 800 and let them play for a year, at the end they'll be better players but still rated 800 on average.

Most chess Elo systems have an inflationary component where young or new players (who are overall faster improvers than the player pool at large) gain and lose points faster than established players (in detail, either using performance ratings or increased k-factors or both). In a balanced rating system, the sources of inflation and deflation are roughly equal. You can tweak the parameters to keep it this way, though it's not trivial to tell whether there is "real" inflation over the years or whether players are simply playing better - or indeed, what's the difference.

Why don't they increase the bar for newbies to get into such a system?

If they know that some people just play a few games and then quit, let's say they only can get Elo when they played a specific amount of time or won at least n games etc.

There is a minimum of 10 games before people start being ranked. People who quit early don't get ranked. People who have played 10 games gain a new long-term goal.