I saw this when 3.5 came out: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...
Haven't followed up on all the comments in it but speculates on why chain of thought improves when training on code.
Haven't followed up on all the comments in it but speculates on why chain of thought improves when training on code.