So what’s the actual realistic issue here? This keeps getting thrown around as if it’s likely, yet not only are there numerous steps against this in the Apple chain, this would already be a huge issue with Dropbox, Facebook, Microsoft, Google, etc who do CP scanning according to all of the comments on HN.
That's trivial. If the attacker can get one image onto your device they can get several.
It's very easy to construct preimages for Apple's neural hash function, including fairly good looking ones (e.g. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... )
> collide with another secret hash function
The supposed other 'secret' hash function cannot be secret from the state actors generating the databases.
Also, if it has a similar structure/training, it's not that unlikely that the same images would collide by chance.
> also have a human look at it and agree it’s CP
That's straight-forward: simply use nude or pornographic images which looks like they could be children or ones where without context you can't tell. It's a felony for them to fail to report child porn if they see it in review, and the NCMEC guidance tells people when in doubt to report.
Besides, once someone else has looked at your pictures your privacy has been violated.
Also if you’re a gov actor trying to frame someone, why bother with a pre-image when you could put the real images on it?
None of that is new today — all that’s new is Apple is joining the effort to scan for CSAM, and instead of doing it on server they’re doing it on device right before you upload in a way that attempts to be more secure and private than other efforts.
Once the capability is in place on everyone's devices, how are we supposed to guarantee it will never be used maliciously? Just say no to the capability.
> Also if you’re a gov actor trying to frame someone, why bother with a pre-image when you could put the real images on it?
Because the capability for this is now built-in in everyone's phones.
Say I sneak (psedo-)child porn onto your device. How do I get authorities to search you without potentially implicating myself? An anonymous tipline call is not likely to actually trigger a search.
With automated mass scanning that problem is solved: All users will be searched.
It's already happening. Except we just choose to SWAT people instead, since it's faster, easier, and there's effectively no liability on the behalf of the caller.
I think the big cloud providers scanning your private data is of dubious ethics, but it's like complaining that your mail carrier is reading the content of your postcards.
So long as you send unencrypted data to a third party your privacy will be limited, regardless of what our laws or norms say. People usually know this, and so many do avoid uploading things to these places or encrypt what they upload.
When its your device itself doing the scanning, ahead of any encryption-- then that protection goes out the window.
Sometimes the same violation of privacy is made more acceptable by a clear boundary that you can stay on one side of to protect your privacy. Your devices vs someone elses devices is the most clear historical boundary in this case, and apple is breaking it.
I don't think it's unreasonable to expect the erosion of the private boundary to have an effect. And we can't say that the scanning by providers does nothing, -- the convictions prove otherwise. We can only hope that all those convictions were deserved, the nature of this crime is such that its hard to prove someone wasn't framed.
At which point everything you brought up about attacks on the hash function is completely irrelevant because the attacker can put actual child porn from the database on your device.
My primary concern about its ethics has always been the breach of your devices obligation to act faithfully as your agent. My secondary concern was the use of strong cryptography to protect Apple and its sources from accountability. Unfortunately, the broken hash function means that even if they weren't using crypto to conceal the database, it wouldn't create accountability.
Attacks on the hash-function are still relevant because:
1. the weak hash function allows state actors to denyably include non-child porn images in their database and even get non-cooperating states to include those hashes too.
2. The attack is lower risk for the attacker if they never need to handle unlawful images themselves. E.g. they make a bunch of porn images into matches, if they get caught with them they just point to the lawful origin of the images. While the victim won't know where they came from.