I believe sycophancy is a side effect of RLHF and whatever reward function it explicitly and implicitly optimizes.