> To claim it is biased implies OpenAI did this on purpose

They did do it on the purpose—the RLHF. The RLHF process was definitely biased, deliberately or unconsciously, in favour of certain political, social and philosophical positions (apparently the same ones held by many in Silicon valley/OpenAI teams responsible for this).