Interesting, but in not sure you're completely right about XLNET. I heard it takes an absurd amount of resources to train. Even more than the BERT variations. And this is likely why there's not a ton of interest in it
Nobody has even tried to create a spanXLnet (akin to spanBERT) How many years will be wasted before researchers get out of the BERT local minima? I'm a afraid it might last a decade