Hacker News Reader

This seems very obvious. Neural networks are the "best" image detectors we have. It's documented that they can be easily fooled.

The insidious thing is that this can used as a pretext to make the filters flag more images since this would seem to make it "harder to hide illegal images", until Apple just personally scans everything.

View on HN

There's a Pandora's box here, too. Bad actors can simply intentionally disseminate memes that match arbitrary bad hashes. Trigger the 20 image (or whatever count) rule for millions of people if you're successful enough.

The system can be attacked at the opposite end as well. Governments can claim hashes are CSAM when the hashes are actually images typically held by political enemies.

I think apple claimed that it requires a database agreed to by at least two governments. Count the countries that discriminate against gay people or authoritarian regimes that regularly cooperate with each other and you quickly realize that Apple just advertised its services to the most morally corrupt and evil people alive.

Matching arbitrary bad hashes for NeuralHash is not sufficient. You also need to somehow match unknown-to-you bad hashes for an unknown-to-you second hash algorithm before you even get to the "30 images" and "human review" parts of the pipeline.

That Pandora's box is locked pretty tightly.

Reply on news.ycombinator.com