ChatGPT has been retrained with a method called Reinforcement Learning with Human Advice [0], effectively making it a very different model:

[0] https://openai.com/blog/deep-reinforcement-learning-from-hum...