I found that in LLMs the self-reference defined as "referring in generated text to itself" (so a bit different than in Scott's blog) comes largely from the chat template

https://openreview.net/pdf?id=3O2A23MhNi