Hacker News Reader

Would it be ethical to train a model to identify child porn so it could be excluded from training sets, or even the internet, automatically? It seems like an ideal very useful application for CV AI, but you might have to train it on CP to make it effective so…

View on HN

I can see some practical difficulties with this. The first one is that CSAM is radioactive and can’t be touched, so only a very limited circle of people could even approach this task. Related to that, the model weights would contain representations of the training data, which if distributed could then be used to get some of the source material back out.

Hashes of existing child porn are created and disseminated for automated detection already, a CP detector model isn't too far from that.

Hashes just detect old unmodified content, not new stuff, or even AI generated stuff.

Wasn’t the hashing mechanism that Apple was proposing using capable of dealing with some attempts to obfuscate the CSAM image with edits?

It's also super easy to change the video slightly to alter the hash.

Obviously, but the idea of creating a useful hash function in this case is to make it invariant under those sorts of to a human trivial transformations, whilst avoiding collisions between genuinely different content. Microsoft created something called PhotoDNA that tries to do this.

Reply on news.ycombinator.com