Hacker News Reader

Comment by 127361 | Hacker News Reader

This dataset contains over 5.8 billion images, so 3226 of them being child pornography is a contamination rate of 0.55ppm. It's likely you will find this material anywhere there are large collections of images.

View on HN

Somewhat reasonable position - except images were found that matched known and catalogued CSAM images. One would have thought that checking against that would be bare minimum for cleaning a large dataset?

Access to PhotoDNA and other 'perceptual hashing' APIs is not open or freely granted. It's not unreasonable that a team compiling a database would be unable to perform this level of cleaning.

PhotoDNA is explicitly free to LE, Nonprofits, and UGC-backed companies. LAION would have easily been able to get access to it for free.

In fact, at that scale, they would likely be able to get assistance. EDIT: Stanford themselves acknowledges in the original paper that bulk PhotoDNA access was given.

Did they even try?

That seems less like an argument against removing/preventing the inclusion of CSAM and more like an admitting on behalf of those building these image sets that they're incredibly irresponsibly sourcing content for the aforementioned image sets.

I'm not saying this is easy to do, in fact I'd wager it's incredibly hard: but it seems increasingly obvious that there was no curation, no standards, no approvals process, nothing but a ramshackle mad dash to include as much visual content as possible to train these models on, with zero consideration of whether the people who created it were alright with their content being used that way, if the content was legal to use either by licensing or by being incredibly illegal just by it's own existence. We get more and more stories of the ethical lapses of the massive companies/organizations behind this tech and the resounding chorus from my fellow engineers is seemingly just "well just remove it and let's keep going," or shrugging shoulders "that'll happen, we're trying to do research here."

And fair enough but like, when you're doing research, you don't just run out and grab every chemical you can get your hands on and pour them all in a bucket and see what happens. How is this stuff not being found? What good are these datasets if they include this type of material? How can you trust the results you get from your training anymore once you know what went in?

Well I would consider a 0.55ppm contamination rate very low indeed? Mistakes happen in the creation of such datasets, it's human nature?

Just as we can't police a society to the extent it's completely free of crime, otherwise we will not function as a society and/or we will find ourselves living in a totalitarian state?

Including CSAM in your image set isn't excusable by "mistakes happen".

Also, the totalitarian analogy is total nonsense.

I think this kinda shows that you don't really understand the problem. If you had a human sitdown and review each image, there still would be images that get through. Even if you told him to err heavily on the false positive side.

Just to be clear, you are saying that no collection of images can ever contain any CSAM, ever [1]. If some activity might run the risk of including it, that activity must be stopped.

Is that correct?

If so, why are you not calling for the internet to be shut down? The collection under discussion is just a subset of the internet.

You'll also want to either ban or enforce review of all camera output, including closed-circuit cameras. Someone is going to have to watch all those watchers pretty closely...

As for totalitarianism, well, how are you going to make any of this happen? As we just saw you'll need client-side visibility into everyone's visual data stores. Not to mention old photo albums world-wide contain images that would set off ImageNet. There are famous paintings and statues that would, too.

[1] Do you exempt NCMEC? If so, how do you justify that?

You abstain from coffee, chocolate, and fish too I trust? Otherwise you're undoubtedly supporting trace amounts of slavery and child labor. I bet the device you're reading this with was also made with such (and if it doesn't then it's still supporting a genocidal regime).

Get real. Scraped data is going to have problems. The expectation should be that reasonable measures are taken to filter proactively and that reactive measures are taken to remove illicit content found after that.

They did curate the dataset – that's what the LAION-Aesthetics dataset is, which is what StableDifffusion used. I'm pretty sure they never recommended using LAION-5B directly as it has low quality images and would take more time to train and run. It's just a collection of links.

LAION is just a large scale scrape of the web, it's a bunch of images with the alt text. So you can say the same about the web itself.

> How can you trust the results you get from your training anymore once you know what went in?

Before taking a moral high ground, please take your time to learn how it works. LAION is well known for its notoriously poor quality simply due to being humongous. There's all sorts of unwanted crap in here. Everyone training on it knows that and does filter it, because it's practically unusable otherwise.

"Look everyone working in this field knows you can't just use the datasets as provided, you have to filter out the bad quality images, the dead animals, and the child pornography."

blank stare

Yes? If you crawl the web or use a result of that, you have to filter out the crap. That's... how it works?

Is really the idea that after taking an automated sampling of things posted online, some of those things will be things people post online, such alien one to you?

0.55ppm is low but you can still be prosecuted if that contamination is found on your drive.

The laws regarding child pornography, which were likely drafted by moral entrepreneurs[1] who created a moral panic out of it, are completely insane. Such persecution belongs in the 17th Century, not the modern age.

1. https://en.wikipedia.org/wiki/Moral_entrepreneur

Frequentist analysis is exactly the wrong approach here, for the same reason that one can't defend a crime by pointing to a much greater number of legal acts.

No, we accept road traffic accidents as fact and do not shut down entire road intersections due to deaths that occur. The same thing applies to child pornography, and 0.5ppm is certainly a tolerable amount in an extremely large dataset.

There are something like ~42000 traffic fatalities in the United States every year, we do not adopt draconian measures in dealing with it, even though loss of life and severe life-altering injuries occur. And such traffic accidents are absolutely devastating for the victims and their families.

Unless the dataset is particularly severely contaminated, the known illegal images should be removed and the rest of it should be allowed to stay up.

>No, we accept road traffic accidents as fact

Have you considered that maybe you shouldn't?

This is how nuclear was killed.

And deaths resulting from cancer and a whole list of other ailments. The act of living itself carries risk.

It’s 340TB, for reference. AIUI it was scraped from the web.

Reply on news.ycombinator.com