Hacker News Reader

Comment by noir_lord | Hacker News Reader

Glicko2 is a better system.

https://en.wikipedia.org/wiki/Glicko_rating_system

Glicko2 has a fatal flaw for massively multiplayer games such as Pokemon Go. Basically, by intentionally losing certain games, you can increase the volatility in your rating, and then you can win a few games in a row against low ranked players and leapfrog people who are actually consistently trying to win.

See:

[1] Farming Volatility: How a major flaw in a well-known rating system takes over the GBL leaderboard. https://www.reddit.com/r/TheSilphRoad/comments/hwff2d/farmin...

Does this flaw only apply to massively multiplayer games?

This further strengthens my belief that Glicko 1 is much more preferable to Glicko 2. It's simpler (no volatility parameter), nearly as predictive (if not just as predictive), and it wasn't specifically designed for a monthly chess schedule, so it fits online games much more easily.

Depends on the environment really. If you're looking at measuring performance where competitors are not able to game the system (like, in professional sports) then Glicko 2 does a slightly better (but statistically significant) job than Glicko 1.

Depends what you mean by better. I agree Glicko approximates your true skill level better and faster. It's also not as fun as ELO in my view. With ELO when I win a few games in a row I get an exciting chance to play against much better players than usual. This is a fun challenge. It's necessary to learn new things. It gives you highs and lows. It's just fun.

With Glicko, once it pinpoints your true skill level, it feels like 2 meters of mud around you. You will never get out of it and you will be forever matched vs the same group of players stuck near you.

So yeah, I don't like Glicko. I have quitted services over it (even something like chess problem solving website). It's just not fun imo.

It sounds like it's not that you don't like Glicko but rather you'd prefer more variety in matchmaking. You can accomplish that easily by changing the way matchmaking picks your opponent.

There's also the TrueSkill rating system from Microsoft (developed for Halo), which claims to be better than Glicko:

"Glicko was developed as an extension of ELO and was thus naturally limited to two player matches which end in either win or loss. Glicko cannot update skill levels of players if they compete in multi-player events or even in teams. The logistic model would make it computationally expensive to deal with team and multi-player games. Moreover, chess is usually played in pre-set tournaments and thus matching the right opponents was not considered a relevant problem in Glicko. In contrast, the TrueSkill ranking system offers a way to measure the quality of a match between any set of players."

https://www.microsoft.com/en-us/research/project/trueskill-r...

I noticed trueskill is pretty effective at matching people together, but the stickiness of rankings makes it frustrating if it’s user facing. People want to see the number next to their name change after 4 consecutive wins.

I think most people would want to see a significant boost if you won 40 out of 50 games. Winning 4 out of 4 doesn't mean much.

First, a good article about TrueSkill I found when looking for systems to rank pairs of foosball players: http://www.moserware.com/2010/03/computing-your-skill.html

Second, apparently Tinder uses an ELO ranking system for their algorithm. I would add a link but everything I'm finding online right now is very clickbait-y.

I'd be interested to read more about Tinder's application. Off the top of my head, i don't see how it would apply to a dating app.

>What was it, though? It was a part of our algorithm that considered how others engaged with your profile. ... this part of our algorithm compared Likes and Nopes, and was utilized to show you potential matches who may be a fit for you, based on similarities in the way others would engage with profiles. Based on those profile ratings you received, there was a “score” — in the sense that it was represented with a numeric value in our systems so that it could factor into the other facets in our algorithm.

From: https://blog.gotinder.com/powering-tinder-r-the-method-behin...

Its not a deep description at all, but it sounds like when someone viewing your profile is treated as a game, which way they swipe is a win or loss for you and your change in score depends on their score as well? Basically if a desirable person swipes right on your average profile, its indicating you are desirable and moves you up more than say the inverse scenario where an average profile swipes right on a desirable profile (I guess this would be akin to a low ranked player losing to a high ranked player).

Edit: I guess the outcome would be people of equal (as perceived by others) attractiveness being matched.

Is the issue multiple players or is it teams? I didn't have any problem implementing ELO for multiple players in games where there can be multiple losers but one winner. I modeled it as simultaneous wins for the winner, with a K-factor divided by the number of players.

But I wasn't concerned with being mathematically rigourus, maybe there is some glitch with this approach? It was for an EDH group. (BTW, don't do this. We no longer play EDH. It makes the spiky players even spikier, and then they get saltier when they're targeted as they have a far higher ELO)

I think the issue is that, outside of formal tournaments, there are rarely stable teams competing against each other in typical online multiplayer. each team is a random assortment of players, where anywhere from zero to a full team are queuing together. people might queue with a friend of a very different skill level. you need to track the skill level of each individual player and then estimate the win probability for a team of players who may never have played with each other before. I play a lot of csgo, and while far from perfect, I'm surprised how good the matchmaking is. I win pretty close to 50% of my matches.

> It was for an EDH group. (BTW, don't do this. We no longer play EDH. It makes the spiky players even spikier, and then they get saltier when they're targeted as they have a far higher ELO)

Yes please keep tournaments and rankings away from EDH for the love of god. CEDH is a different story because it's a completely different mindset, but if I sat down for normal EDH and was told we were tracking Elo I'd run far, far away.

It was mainly to track strength of different decks we all played, since you could check ELO of either. It was the strongest player who was upset by it, since once people realized he won 50% of 3 and 4 player games he was always targeted. But yeah, I wouldn't do it again.

For certain types of multiplayer games the recent Elo-MMR is even better (faster, provable guarantees) [1][2].

Quote from the paper regarding TrueSkill:

> The main disadvantage of TrueSkill is its complexity: originally developed by Microsoft for the popular Halo video game, TrueSkill performs approximate belief propagation on a factor graph, which is iterated until convergence. Aside from being less human-interpretable, this complexity means that, to our knowledge, there are no proofs of key properties such as runtime and incentive alignment. Even when these properties are discussed, no rigorous justification is provided. In addition, we are not aware of any work that extends TrueSkill to non-Gaussian performance models, which might be desirable to limit the influence of outlier performances.

[1] https://github.com/EbTech/Elo-MMR

[2] https://arxiv.org/abs/2101.00400

It is fascinating to see improvement in this space.

People -really- care about good ranking systems. I personally saw this when I administrated rankings for a local fighting game scene. Back then we used Glicko since matches were 1-on-1.

Trueskill is also under patent. Which is considerably more off putting than its rigor.

Reply on news.ycombinator.com