Hi again! We had a back-and-forth about this a while back regarding the paper and I think we didn't end up on the same page regarding the "public data" definition in the paper (found it! [0]). I love that you're upfront in the paper, because it's silly how most people just don't acknowledge it (though they usually don't redistribute it publicly like the pile does).
I think the gist was us disagreeing about the relevance of
> Public data is data which is freely and readily available on the internet. This primarily excludes ... and data which cannot be easily obtained but can be obtained, e.g. through a torrent or on the dark web.
That last phrase is what got to me. It puts things in the same category that feel too different. E.g. the harry potter books in vs this comment I'm writing. They're both available within a few clicks from the search bar (one because I put it there, another because it was put up against the wishes of the author and owners), but that commonality doesn't feel relevant.
Excluding torrents especially seems like a cop out explicitly to get around the issue of "X is the top result when i google it" being so common as a torrent. I think you're trying to exclude that content as public because then it defines too much as public? But torrent vs ftp doesn't feel at all relevant when it's just google plus a click or three. Or searching on pirate bay plus a single click.
I imagine a judge looking at the copyright status of someone's pirate site and saying they can't redistribute the content, and the pirate responding "okay we'll take down the ftp server and put up a torrent instead, so that it's not public. If you google us (or search on pirate bay), the top result will stop saying 'X download' and now it'll say 'X download torrent'" and expecting the law to be on their side.
I didn't really buy the arguments in section 7 either. The usage points seem legitimate, but don't cover redistribution.
> But we will be making the license-compliant subset of the Pile public when we are able to and will give it equal prominence on our website to the “full” Pile.
This is fantastic and I want to sincerely thank you for that.
I'm trying not to be combative, but I feel like publicly redistributing other people's work does raise the bar quite a lot higher than just using it to train.
[0] https://news.ycombinator.com/item?id=25616218