>It compares on-device image hashes with hashes of known CP images.
No, it compares against hashes supplied by a government agency (NCMEC). Apple has no way to verify the hashes are in fact CP, as it is only re-hashing hashes.
No, it compares against hashes supplied by a government agency (NCMEC). Apple has no way to verify the hashes are in fact CP, as it is only re-hashing hashes.
Presumably at this stage is where malicious hashes would be detected and removed from the database.
An unaccountable "Apple Employee" who is likely (in the US and other countries) to be a LEO themselves will see a "visual derivative" aka a 50x50px greyscale copy of your content.
There is no mechanism to prevent said "employee" from hitting report 100% of the time, and no recourse if they falsely accuse you. The system is RIPE for abuse.
>Presumably at this stage is where malicious hashes would be detected and removed from the database.
Collision attacks have already been demonstrated. I could produce a large amount of false positives by modifying legal adult porn to collide with neural hashes. Anyone could spread these images on adult sites. Apple "employees" that "review" the "image derivatives" will then, even when acting honestly, forward you to prosecution.
Of course there is. The judicial system.
(Although, to be clear. I don't live in America and I might be more worried about this if I did.)
The recourse should be before this reaches the law.
1) Spent who knows how long in jail
2) Lost your job
3) Defaulted on your mortgage
4) Been divorced
5) Had you reputation ruined
Money can't fix everything, and trusting the courts to make you whole years after the fact is a foolish strategy.
How, if 1) the original content is never provided to Apple, and 2) the offending content on consumer devices is never uploaded to Apple?
This entire system was a way for Apple to avoid decrypting your photos on their servers and scanning them there.
Hypothetically, if Apple implemented this system and switched to E2E for the photo storage, you'd be more private overall because Apple would be incapable of seeing anything about your photos until you tripped these hash matches, as opposed to the status quo where they can look at any of your photos whenever they feel like it. (And the hash matches only include a "visual derivative" which we assume means a low res thumbnail.) I say hypothetically because Apple never said this was their plan.
You can argue about whether or not Apple should be doing this. But it does seem to be fairly standard in the cloud file storage industry.
I suspect the key would be that there'd be a team verifying that something is actually child pornography, because the system is a perceptual hash rather than a strict comparing-bytes so before someone looks at it they're not certain.
As a taxpayer and customer, I concur. I'm glad someone is doing that job. But I don't want it to be a corporation.
A "third party" paid by apple who is totally-not-a-cop who sees a 50x50pz grayscale "image derivative" is in charge of hitting "is CP" or "Is not CP".
I don't understand how anyone can have faith in such a design.
So, to get flagged, you need many hash collisions destined for iCloud. Then, to get reported, some number must get a false positives in the Apple review, and then some number must somehow fail the full review by NCMEC.
https://www.theverge.com/2017/4/12/15271874/ai-adversarial-i...
If we can, then, hypothetically we need to get non-CSA images onto important people's iPhones so they get arrested, jailed for years and have their lives ruined by Apple.
Disclaimer: I buy Apple products.
These images can be 100% indistinguishable from the real thing. Without knowing the source of the images that they are putting in the database, how do they know the images are actually illegal?