Yeah most of the dataset most NLP models use is common crawl; most of which contains Reddit, which is probably at gpt3 is giving those sorts of results.