Given the lc0 policy network plays at a 2000+ strength on its own, I would expect that with enough finetuning gpt4 would be able to play way above 1600 strength.

It's possible that finetuning would basically be training a new network from scratch and the resulting network would forget everything apart from chess. It would be a really interesting experiment, GPT-2 is probably too small but I think llama-7B might be sufficient.