'finetuning on synthetic data' = same LM task with BCE objective, ie. training on next word prediction on good text.

'rl on sparse rewards' = RL objective - reward is given by deterministic evaluation, reward is passed back as sparse signal. there is no 'ground truth' data to compare against.