Exactly, I think that by their very design, LLMs are very sensitive to how a question is framed.

But I wonder how much of that comes from RLHF itself or just from the way token prediction works.