it is a result of trying to align the AI model after it has been trained using self-supervised learning with an addition human feedback step. OpenAI has a model / process called InstructLM or InstructGPT

see e.g.

https://arxiv.org/pdf/2212.10496.pdf

https://arxiv.org/pdf/2203.02155.pdf

also an approach to solve the alignment problem.

https://en.wikipedia.org/wiki/AI_alignment