the core insight is that you can train on extremely sparse deterministic reward signal and it just works
that MoEs are better for a given compute budget has been known for a while.
that MoEs are better for a given compute budget has been known for a while.
They also mention in the source tweet that the quality of the base model is important, which is not really accounted for in the $30 figure.
'rl on sparse rewards' = RL objective - reward is given by deterministic evaluation, reward is passed back as sparse signal. there is no 'ground truth' data to compare against.