It is indeed:

We introduce new datasets derived from the fol- lowing sources: PubMed Central, ArXiv, GitHub, the FreeLaw Project, Stack Exchange, the US Patent and Trademark Office, PubMed, Ubuntu IRC, HackerNews, YouTube, PhilPapers, and NIH ExPorter. We also introduce OpenWebText2 and BookCorpus2, which are extensions of the original OpenWebText (Gokaslan and Cohen, 2019) and BookCorpus (Zhu et al., 2015; Kobayashi, 2018) datasets, respectively.

From https://arxiv.org/abs/2101.00027 (The Pile: An 800GB Dataset of Diverse Text for Language Modeling)

> > Probably HN is already part of The Pile

Everybody smile for the camera, or we could just moon them, or both!

Smaller in scale, yeah, but probably high bias towards technical content and written by people who (mostly) care about how to write properly. There is at least 37,354,035 items that could be indexed with lots of it high quality, percentage wise probably higher quality/post than Twitter, Facebook and other sources.