But it does illustrates how it's easy to get trapped into a local minima.
They're doing what you tell them to, not what you want them to. Same with GPT-3; it's predicting what (from its experience) it thinks a human would say, not what is ethical to say.