I just tried ChatGPT on a simple find an error in the proof question today and it completely butchered it ignoring the main constraint that two variables a and b were natural numbers. It said, "a and b might not be natural numbers" and then went on to produce a very elaborate, completely wrong answer.