Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings

It's glad to see the old technique is used for new models.

You may also learn from 7.33 dota update which uses a new ranking algorithm called Glicko.

could you provide any reference? Is it a variant of ELO?

Valve listed some reason for making the change.

Elo != ELO.

One is a rating system named after the creator, Arpad Elo. See https://en.wikipedia.org/wiki/Elo_rating_system

Surprised to learn StableLM is worse than plain LLaMA. link to their leaderboard: leaderboard.lmsys.org

I’ve heard that it’s really bad for it’s size

This is a very good idea.