https://en.wikipedia.org/wiki/Chinese_room I think about this once in a while. At some point if it does the thing almost perfectly is it still not doing the thing?
I was more talking in reference to why the LLMs in the above linked paper were producing so many illegal moves, and it is because they are not hard constrained by the rules of the game. Of course, a loop can prompt until a valid move is produced and then rendered on a screen. But why do this? I suppose, who am I to say what should be done or not, but a specialized tool being better than a general one at its specific job isn't particularly surprising.