it is a result of trying to align the AI model after it has been trained using self-supervised learning with an addition human feedback step. OpenAI has a model / process called InstructLM or InstructGPT
see e.g.
https://arxiv.org/pdf/2212.10496.pdf
https://arxiv.org/pdf/2203.02155.pdf
also an approach to solve the alignment problem.