Thank you for sharing these, I will definitely check them out! The concept of superposition here is new to me, but the way its described in these articles makes it very clear. The connection to compressed sensing and the Johnson–Lindenstrauss lemma is fascinating. I am very intrigued by your toy model results, especially the mapping out of the double-descent phenomena. Trying to understand what is happening to the model in this transition region feels very exciting.