Hacker News Reader

Thank you for the reply. Maybe I'm daft or out of the loop, I don't see the difference between this method and what I have seen previously as "fine tuning on synthetic data".

They also mention in the source tweet that the quality of the base model is important, which is not really accounted for in the $30 figure.

View on HN

'finetuning on synthetic data' = same LM task with BCE objective, ie. training on next word prediction on good text.

'rl on sparse rewards' = RL objective - reward is given by deterministic evaluation, reward is passed back as sparse signal. there is no 'ground truth' data to compare against.

Ah now I see the difference. Thank you!!

Reply on news.ycombinator.com