This is also one of the first things I test with new models. I did notice that while it still plays very poorly, it is actually far more consistent with the board state, making only legal moves, and noticing when I win than is GPT4o.