at least for the pre-training data there are some open source torrent clients [0].

[0] https://news.ycombinator.com/item?id=42971446