Hacker News Reader

Comment by 127361 | Hacker News Reader

I think, over a dataset of nearly 6 billion images, it's just impossible to ensure that it contains no child pornography.

Like trying to ensure all my kitchen surfaces are 100% free of germs. Not possible, or the amount of effort and disruption caused by the cleaning will make it impossible to get on with my life. Nasty stuff exists out there. We don't want too much of it, though. That's my opinion about it.

View on HN

You have to ask yourself not “how impossible” it is , rather how much effort was put into mitigating this. The answer is probably a nominal amount but obviously not enough

I'll bet OpenAI and Google are also contaminated, but that we'll never know and it'll never be cleaned.

With open source efforts, we can eventually reach a state of public certification. It can be fully vetted, unlike closed source models.

The paper says that almost 500 of these were identified by MD5. That indicates that about 0 effort was put into mitigation.

So why all the worry about re-uploading a clean dataset because someone "could do a diff" to find the bad images? Could they not use the same md5 search to find the same?

i guess the argument is that these images were a needle in a haystack and now you have given the public a supermagnet to go find it.

like right now, if someone tries to download old laion and find those images manually, it would probably take them a very very long time. if they had the diff, they would find them in a few minutes.

i dont think its a good argument, but its an argument. the researchers found them without a diff so likely someone else can too. so theres not much reason to leave them in that i can tell.

Yes, I get that. But that's not what the researchers did. They used a database of know hashes. Can bad actors not do the same?

I don't think it's easy to get access to the PhotoDNA database. And it would be a service that people query against, so they'd have an audit trail.

You're not allowed to have the MD5 hashes these researchers used to find these images, so it's not possible to reproduce this result. These hash databases and APIs are tightly controlled, and only law enforcement, major tech companies and top researchers are allowed to use them.

How does that logic work? Unless your definition of "mitigation" is "modifying CSAM before including it in the dataset"?

Like, "identified by MD5" doesn't mean that they didn't attempt to remove images! A lot of CSAM out there is not yet on those lists, and a lot of it is being added as we speak, so if LAION-5B was filtered using one list of MD5 hashes, that doesn't mean it doesn't contain CSAM that's on a different list (or a newer version thereof).

And you have this problem no matter whether LAION actually took any steps to remove CSAM before publishing the dataset or not. Unfortunately, any sufficiently large such dataset will contain some bad content (how we should deal with this is a different question that I sadly don't have an answer for).

If it's all new, I guess that's one thing. But in general, if LAION wanted to know if this content was in the collection, they could have queried the same database. Some new heuristics turned up others, but I mean the basic search could have been done.

md5 hashes are not a reliable indication of anything, since producing images that have collisions can be done in realtime.

https://www.exploit-db.com/docs/english/46047-md5-collision-...

That's if you have control of both images. With a given MD5 hash, you can't make another file with that hash.

Still not an argument for ever using them as reliable evidence. You have to trust whoever made the original hash to not have generated it with a duplicate, innocuous image providing a collision. These hashes are not verifiable as they appear to be parts of secret databases, so you are trusting the authority of that database not to do something like make innocuous colliding images for the sake of violating your personal freedoms.

Reply on news.ycombinator.com