It's a very new thing. Pretrained ImageNet models were first released around 2011. Pretrained transformer models have only been released recently.

Also, transformers are only useful for short text, not full documents (AFAIK).