Hacker News Reader

I don't mean evaluating a model at the end of training, I mean datasets used to quantify the relative performance across models and algorithms.

Copyrighted data would make it harder to share and use, to reproduce results. I don't remember coming across a benchmark dataset that was copyrighted.

View on HN

That's very odd because I can't for the life of me think of an ml benchmark dataset that is not copyrighted. Copyrighted datasets are everywhere in ml research.

Just looking at the eval datasets used in the HF leaderboard, ARC is CC BY-SA, TruthfulQA is Apache, and HellaSwag and MMLU are both MIT. The first dataset many people will explore ml with, MNIST, is GPL-3.0 license. The audio courpus datasets I've used have all been similarly permissively licensed, for example LibriTTS is CC BY 4.0. LJ Speech, one of the main datasets used in Tacotron is public domain. Maybe I should have been more explicit and used the term permissively copyrighted instead to help avoid confusion.

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

Reply on news.ycombinator.com