These are two very different things.

One is talking about an improvement made by making control flow changes during inference (no weights updates).

The other is talking about using reinforcement learning to do weight updates during training to promote a particular type response.

OpenAI had previously used reinforcement learning with human feedback (RLHF), which essentially relies on manual human scoring as its reward function, which is inherently slow and limited.

o1 and this paper talk about using techniques to create a useful reward function to use in RL that doesn't rely on human feedback.