Common Corpus: the largest public domain dataset for training LLMs

huggingface.co

7 pointsinternetter2 comments

Obviously, there is a lot less data here than other LLMs, but this data might also be higher quality. I'm curious how it works out.

uh-huh; if this is any indication, I gravely doubt that: https://huggingface.co/datasets/PleIAs/US-PD-Newspapers/view...

I couldn't figure out their search syntax but both date>2000-01-01 and just 2000-01-01 didn't cough up anything, so I also gravely suspect this data will only be antiquated language usage, too, to say nothing of the objectively horrific OCR job they did

Reply on news.ycombinator.com