Asking GPT to play chess directly using its reasoning only tests its reasoning ability to model chess state. Which it really is NOT optimized for.
This is also true for humans - people who don't have years of chess training can't really tell which moves are legal given an algebraic notation transcript. These people might have good strategic skills in different areas. Chess is just a very, very specific skill