The majority of the LLM training corpora comes from Common Crawl and social media - they're not a curated source like Open Culture.