A few more transformer implementations that I’ve found:

https://github.com/pbloem/former/blob/master/former/transfor...

https://github.com/openai/blocksparse/blob/master/examples/t...

https://github.com/google/trax/blob/master/trax/models/trans...