Hacker News Reader

The pretraining approach was used in vision for years before it was successful in NLP.

Not really on unsupervised/self-supervised data though, right?

(nor on the same scale of corpora, as far as I can tell)