Hacker News Reader

Comment by marcodiego | Hacker News Reader

This may look low: ELO for mediocre players is 1500. But if it is obeying the rules of the game, then this is big. This is a signal that if it learns some expertise, like discovering how to use or create better search algorithms (like MCTS and heuristics to evaluate a state) and improve by itself (somewhat like alphazero did), then it may eventually reach superhuman level.

It may then reach superhuman level in any task simpler than chess which is enough to destroy many human jobs.

EDIT: From the article: "With this prompt ChatGPT almost always plays fully legal games." Relax: we're still far from that.

View on HN

According to https://chess.stackexchange.com/questions/2550/what-are-the-... median rating is 1148 (252,989 Players). So it's beating half of humanity at a mind sport and it wasn't even specifically trained for it.

There's a difference between chess.com and USCF ratings.

That's USCF ratings, chess.com ratings are massively inflated in comparison.

The median chess player is usually described as mediocre (if you ask chess players). They suck as badly as the median clarinet player in your high school band/orchestra.

I think current LLM architectures are limiting the strategies it learned. MCTS requires recursion but GPT is always executing a fixed number of steps. Allowing language models more flexibility by a variable number of steps, for example through recursion of the model, would eliminate this hard bound but they are harder to design and train. We have just been able to train GPT-sized models.

I'm sure the MuZero chess policy network would reach much higher than 1400, and that has no notion of recursion either. (And also wasn't taught the rules explicitly)

Yes, I wouldn't say it's impossible, but that's just a hard limit from the architecture. MCTS would have to terminate after a few iterations. And the number of steps is not that big, just massively parallel.

Reply on news.ycombinator.com