In a sample of >1000 games, GPT-3.5-turbo-instruct plays chess with ~1800 elo

github.com

4 pointssebzim45004 comments

This is better than the vast majority of players on lichess.org, and also much better than GPT-4.

I checked a random sample of games manually, and all of them quickly diverged from each other and from the lichess database (of ~5B games), so it is inconceivable that the training data contained a significant portion of these games.

Hopefully OpenAI releases an instruct version of GPT-4 so we can see if scaling continues to bear fruit.

EDIT: I played against it once. It got a winning position and then blundered mate in 1. Considering the chat model can barely find legal moves this is still very impressive IMO.

What did you use to play against it?

I just did it manually, but if there is interest I will make a lichess bot so we can get an accurate elo and everyone can have a go.

You can play against it here.

https://parrotchess.com/

Reply on news.ycombinator.com