Actually if ChatGPT gives you a bad/wrong answer, and you reply telling it is wrong and why it is wrong, it will answer with something that you think is correct.
So you are already doing a kind of real time RLHF in the chat. That is how DAN was prompted to exist.