That is not bad. As a tool to filter down what apple human reviewers need to look at this is pretty good.
Ultimately these images will make it to a human reviewer who can make a call as they would in any flagging system.
Could a backend server side system do a more precise hash (96 bits is not a ton) prior to human review?
Imagine that you play a game of craps against an online casino. The casino throws a virtual six-sided die, secretly generated using Microsoft Excel's random number generator. Your job is to predict the result. If you manage to predict the result 100 times in a row, you win and the casino will pay you $1000000000000 (one trillion dollars). If you ever fail to predict the result of a throw, the game is over, you lose and you pay the casino $1 (one dollar).
A casino that makes no adversarial assumptions about the clientele could argue as follows: the probability that you accidentally win the game is much less than one in one trillion, so this game is very safe, and the House Edge is excellent [3]. But this number is very misleading: it's based on naive assumptions that are completely meaningless in an adversarial context. Some of the clientele will cheat. If your adversary has a decent knowledge of mathematics at the high school level, the serial correlation in Excel's generator comes into play [4], and the relevant probability is no longer less than 1/1000000000000. In fact, the probability that the client will win is closer to 1/216 instead! When faced with a class of adversarial math majors, a casino that offers this game will promptly go bankrupt. With Apple's CSAM detection, you get to be that casino.
(reposted based on my comment on last week's thread [1])
[1] https://news.ycombinator.com/item?id=28236102
[2] https://blog.roboflow.com/neuralhash-collision/
[3] https://wizardofodds.com/gambling/house-edge/
[4] How to crack a linear congruential generator? http://www.reteam.org/papers/e59.pdf
But I really wonder why you think this is an important objection, do you think a lot of people want to go to the "get flagged for child porn" casino?
If these new images never make it to the NCMEC database, then new CSAM content will be completely NeuralHash-proof. However, if these images eventually make their way into the NCMEC database, then everybody who has the perfectly innocent originals will be dealing with an adversarial environment.
The remark that an image might not be in the database applies equally to all CSAM pictures, and so is not actually dependent on any technical aspect of the hash at all.
The fact that you can modify an image to the point that it gets a different hash isn't important compared to the (separate) issue of how you keep your database up to date. And detecting old CSAM is in any case not as important as tracking and interdicting production of new images.
The second remark is good, but mainly because it reminds us to demand Apple ensures that additions are scanned for malicious images before being added to the central database. If the operator determines that a certain images collides with some known public image they can notify Apple. Apple in turn modifies the hasher to dissolve the collision, roll out a patch, and can then update the database.
Even then, for that to affect _John Doe_, they would have to make 30+ images whose hash matches that of images in _John Doe’s_ iCloud account.
I think that means they could target individuals, but only if they knew or could guess what photos they have in their account.
They also might be able to target groups of individuals, say people who went on holiday to Paris. It would be interesting to see whether such people have enough overlap in the sets of Neuralhashes of photos they took there.
Just look at political memes. What is the chance that both sides will have a few people try to create memes for their opponents which are adversarial. They'll take care to make sure those images aren't on their own apple devices before spreading it to areas where the other side likes to share memes.
Another example are the very people trying to be caught. Someone who is against current laws about the subject might seek to create as many false positives as possible to overwhelm the system. They might even specifically target otherwise legal baby photos to manipulate in this way as it would make it more likely they get past any sort of second tier manual review and result in law enforcement wasting resources.
Lastly, this is nothing new. Planting such material and flooding websites with it has long been a tactic used by some. Up until now it has been limited because doing so requires violating the law yourself and few people hate others enough to go through that level risk to self. But this is mostly risk free because creating adversarial images like this isn't outright illegal and even in cases where there are laws against it violating those laws is extremely different than violating actual laws against CSAM in every way that factors into a person's willingness to break laws.
> The only interesting metric is the false positive rate for normal images.
Wrong. The only interesting metric is the false positive rate _in practice_: i.e. how likely are false positives to affect innocents. Indeed, Apple is presenting their "one in a trillion" number as reassurance, as if it was the probability that an account that doesn't distribute CSAM gets flagged by their system. But that probability depends strongly on adversarial questions, and cannot be calculated using optimistic assumptions about all images being "normal".
Perhaps I could even do that without revealing my motives to you.
If you send actual CP, then that's a wholly different matter.
And human reviewers are in the process. If you've got 30 matches and they are all pictures of bridges or whatever do you think the FBI is going to show up at your house?
[1] https://pseudorandom.resistant.tech/obfuscated_apples.html