Exploitive, illegal photos of children found in the data that trains some AI | Hacker News Reader

Exploitive, illegal photos of children found in the data that trains some AI

washingtonpost.com

"The researchers scanned for the abusive images by looking for their “hashes” — corresponding bits of code that identify them"

I can't find the actual study, but I am wondering if anyone actually verified the photos are CSAM, and not a collision or even just misidentified images?

There is substantial evidence that the technique used is highly likely to produce false positives when ran on a large dataset like this.

Found the study- https://stacks.stanford.edu/file/druid:kh752sm9123/ml_traini...

The study says 825 were confirmed using one method, and 183 with another- not clear to me if those are unique or duplicates of each other. Also, "confirmed" only means "is or is likely" CSAM, and they don't provide any numbers as to how many were classified as "likely". It also isn't clear how they define "likely".

EDIT: Previously I said the article was overstating the number of CSAM images found, but I since re-read it and the article may be citing the correct total from the two methods)

I thought quality hashing algorithms have no known collisions? Like, it's theoretically possible, it just has never happened, anywhere, not even once.

Why would a collision happen here?

Cryptographic hashes aren't supposed to have collisions... the types of hashing that PhotoDNA and other similar systems use are not cryptographic. In fact, some say they aren't even real hashes. The reason for this is because they want images that have been altered to still match, whereas with a cryptographic has, even a minor change will change the hash. So it is more a a "similarity" check than a proper hash.

You're confusing hashing as in SHA-256 with perceptual hashing, which is more like a blurry thumbnail of an image converted into numbers.

Any minor alteration to an image will change it's cryptographic hash completely, making those pretty poor tools for finding images that might want to be hidden. Perceptual hashing should match e.g. resized versions of the image, but with them two "blurry thumbnail hashes" can match accidentally.

"Contractors in Kenya say they were traumatized by effort to screen out descriptions of violence and sexual abuse during run-up to OpenAI’s hit chatbot"

https://www.wsj.com/articles/chatgpt-openai-content-abusive-...

While an AI accessing real abusive images is increasing demand for those images and thus arguably has real victim. But it does make me think, if someone were to ask an AI that is trained on normal pictures of children and normal pictures of consensual adult pornography and ask it to create a fake child [0] in a pornographic situation, where is the victim?

I know some jurisdictions have laws to the point that a drawing of two stick characters having sex with the words "Lisa" and "Bart" on them would be illegal, so presumably an AI generated image like above would also be illegal, but I don't believe that's a uniform law across the world.

[0] https://this-person-does-not-exist.com/en

Indirectly, if AI-generated CSAM materials flood the internet, it would provide cover for real CSAM which would get lost in the crowd making it more difficult for law enforcement to identify and prosecute offenders.

This is your regular reminder that the LAION dataset does not itself contain any image data at all: it is an index of image URLs on the the public internet which has been classified as having decent-quality tagging (image alt-text) and a few other bits of bulk, automated analysis done.

The accessibility of CSAM content links within the dataset means those images are already on the internet being made publicly accessible by another website, which was found during the LAION image crawl.

(interestingly this does rather imply that if LAION was able to compare images to the relevant CSAM hashes during the crawl then they'd simply be able to exclude them automatically - although I imagine that database is always growing)

Gift link for anyone who can't view the article: https://wapo.st/41sWoLj

dupe? https://news.ycombinator.com/item?id=38708099

We need to dissect training data more. Too much push to enhance models without ensuring the input data is free from issues.

https://archive.ph/uNQCv

If you are attempting to index the whole internet then you are doomed to eventually download something illegal.

Reply on news.ycombinator.com