A Controlled Study of Attention-Only Transformers

The writing does come off as ai but I think the experiments are interesting