Doesn't this dataset very obviously violate copyright?
Of course, for a user to use the dataset, they'd have to download it. Whether or not that's a copyright violation depends on their local laws.
That's probably still illegal. The implied purpose is copyright violation and piracy. Judges aren't computers - they're capable of discerning when someone is trying to skirt the law by saying "here's something you could use to do something illegal...wink wink nudge nudge".
Those other sites that got in trouble are links to all pirated content, for the purpose of pirating content.
And we still haven't decided this isn't fair use (it's non-commercial and used only for research, so it can't really be said it harms the copyright holders' interests), and fair use is by definition not a violation of copyright.
> [May 16, 2024 Update] Due to the recently added anti-bot measures by the data holders, our downloading pipeline is no longer working. The video links are still accessible through a browser but not via our python crawler. We are working on a workaround and will make an update once we find one. At this time, we are still providing the parquet files to researchers, but researchers will need to find a way obtain the video data. Thank you for your understanding.
Ah, so on top of making a “dataset” of a category of specific works, they are also making people hammer the servers of other parties who never agreed to pay the bandwidth for a bunch of “researchers” wanting to download all of these files. Classy.