Hacker News Reader

seraineon Manipulating Chess-GPT's World Model

The Stockfish program can be set to play at strength level 0-20. Estimates of the levels' Elo is provided here: https://github.com/official-stockfish/Stockfish/commit/a08b8...

seraineon Manipulating Chess-GPT's World Model

I updated the article to also mention this in the second blog post. All games were against Stockfish level 0, with nodes searched limited to 100,000 rather than a time based search limit to limit variability due to different processors.

seraineon Manipulating Chess-GPT's World Model

The code for this is located here: https://github.com/adamkarvonen/chess_llm_interpretability

seraineon In a sample of >1000 games, GPT-3.5-turbo-instruct plays chess with ~1800 elo

What did you use to play against it?

seraineon GPT-3.5-instruct is a ~1800 ELO chess player. 150 games of GPT-3.5 vs. stockfish

Correct. An example can be seen here:

https://github.com/adamkarvonen/chess_gpt_eval/blob/master/g...