Wow - [1] should be considered required reading. OpenAI is baking in human employee biases during fine-tuning.

I would want to see the exact set of posed completions and paired responses.