As the article points out, the problem is that by doing so you create a map for people to find CSAM just by diffing the datasets.
Get the dataset minus the illegal data back. It's essential to maintain parity with closed source efforts.
I'd bet that OpenAI also has this in their dataset too. And nobody will ever even know. Open source is the only AI that can be knowingly scrubbed and cleaned.
there is no panic here. every reasonable person agrees this is a problem, and it will take very little effort to fix it. these AI companies are not hurting for money. every single one of them is valued in the billions of dollars. they can spare "0.5 ppm" of their funding to go do what almost everyone agrees needs to be done.
I'm not saying that more shouldn't be done, but that doesn't mean that the solution is easy or that a perfect solution even exists.