Hacker News Reader

Anyone have any ideas on how they're doing text insertion using an auto-regressive model?

yes, they are most likely finetuning with this type of pretraining https://arxiv.org/abs/2103.10360 quite easy to build