The issue with the models isn't that they play a bad game, but that they persist in making illegal moves. An average intelligent human can be told the rules of chess and then play chess, badly, within the rules.
AFAIK, current models will still sometimes make illegal moves even if given the entire game state (e.g. in FEN notation), so it is not purely an issue with the models’ ability to keep track of sequences of moves.