The reinforcement learning objective including things like thumbs up/thumbs down captures people's preference to be flattered.