Hacker News Reader

Someone is going to figure out how to make false positives, and then an entire genre of meme will be born from putting regular memes through a false positive machine, just for the lulz.

Someone else could find a way to make every single possible mutation of false positive Goatse/Lemonparty/TubGirl/etc. Then some poor Apple employee has to check those out.

View on HN

If the process of identifying the images is done on the device, then a jailbroken device will likely give an attacker access to the entire DB. I'm not sure how useful it would be, but if the attacker did have access to actual known CSAM images it probably wouldn't be hard for them to produce false positives and test it against the DB on the jailbroken device, without notifying the company.

If Apple is indeed using CNNs, then I don’t see why any of the black-box adversarial attacks used today in ML wouldn’t work. It seems way easier than attacking file hashes, since there are many images in the image space that are viable (e.g., sending a photo of random noise to troll with such an attack seems passable).

What's a CNN?

Convolutional Neural Network e.g., deep learning

Reply on news.ycombinator.com