It seems to be getting a lot of attention because it's an easy target, but it doesn't seem to be a very meaningful one given the system they say they use.
[0] It's federally illegal for anyone to have CP on their system, or to view it.
[1] So the workaround is displaying a distorted low-res version instead https://www.apple.com/child-safety/pdf/Security_Threat_Model...
[2] You can replace CP with anything your oppressive regime is trying to snuff out.
This is not correct. Having a few will put those photos through a second, blind hashing algorithm. It wouldn't be flagged for human review unless the images in question passed the second stage and, even then, you wouldn't have any of the information for those hashes to match.
Having a few will have the flagged photos sent to their servers, now liable to be obtained by external forces. Then processed by a second visual hashing algorithm, liable to the same type of collision attacks, using obfuscation as its promise of security.
> Once Apple’s iCloud Photos servers decrypt a set of positive match vouchers for an account that exceeded the match threshold, the visual derivatives of the positively matching images are referred for review by Apple. First, as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, independent perceptual hash. This independent hash is chosen to reject the unlikely possibility that the match threshold was exceeded due to non-CSAM images that were adversarially perturbed to cause false NeuralHash matches against the on-device encrypted CSAM database. If the CSAM finding is confirmed by this independent hash, the visual derivatives are provided to Apple human reviewers for final confirmation.
So to recap, this second blind hash you're talking about is run _after_ the server has already received and decrypted the visual derivatives (what else would it be hashing?).
Perceptual hashing is highly susceptible to collisions. I'm not sure what you're talking about with "hiding a different thumbnail through resizing". All you need is nude-looking colors in your photo to let the derivative look like credible CSAM. No resizing tricks necessary.
Having a image sent to LEO is effectively being accused of a crime.
What prevents a judge from approving a search warrant on the basis of an image that matched a hash? The answer is, of course, “nothing”. What guarantee do we have that these matches aren’t used as a signal to some other automated extra-judicial search? Again, none.
None of this is fantasy. Take a look at the no-fly list and and no-knock warrants and ask yourself if you really want an automated system making accusations to LEO.
e: clarified that images are sent to LEO, not just hashes.
"Those targets will not meet themselves"
"What if that person has CSAM but tried to obfuscate it. Would you forgive yourself if you let it go and it turned out you were wrong?"
There's a fair point in there about public defenders not meeting that bar, at least in terms of the quality of their representation given the amount of work they have, though.
Lawyers aren’t always doing their maximum. It doesn’t matter that they get paid, you also have to be among their top important customers, just like aircon repairmen.
Also, the picture is not CP but by that time, the phone is already entirely searched.
Law enforcement has no incentive to exonerate you, but they have tremendous incentive not to waste everyone’s time, including their own, on fake cases with little chance of conviction.
Sadly, in the US prosecutors regularly mislead the court because our adversarial system of law is seen by many to obligate them to zealously prosecute their case up to the boundary of the law. (That the prosecutor should conceal evidence, mislead, exaggerate, right up to the point where any further and their actions would be an unambiguous crime)
As a result, there have been child porn prosecutions over totally legal material rescued only by the actress in question showing up in the court. ( https://www.crimeandfederalism.com/page/61/ )
At the end of the day, Apple's decision to reprogram their customer's private property to snoop on their users private files and report on them creates a risk for those users. The risk may arguably be small, at least today. But it is a risk none the less.
We don't need trillion dollar corporations snooping on users phones like masked vigilante crime fighters. Batman is fiction, and in even the Dark Knight when he spied on everyone's phones it was clearly portrayed as an erosion of his moral character out of desperation.
It’s getting a lot of attention because this trivially violates the last 5 years of apple marketing/branding, and as per usual, by depending on “won’t someone think of the children!” logic (which basically justifies anything and everything)
Apple isn't exactly known for software transparency or allowing software that monitors iOS internals.
Many reasons this is bad.
Obviously the "Slippery slope" argument around it scanning more in future and being abused by the state.
There is the possibility that its exploited/bug that sends data when it shouldn't or falsely reports you (not a collision). Eg. a bug that reports all photos as bad instead of listening to results of scan.
Obviously the possibility of collision sucks. But thats somewhat constant across implementation options.
It wastes your CPU resources and network resources to do something that only benefits apple.
It's a bad precedent because other companies will likely do this now, repeated the above arguments with increasingly less trustworthy companies.
It seems that this system might be extremly resistant for collision to prevent them ending up to human review.
https://news.ycombinator.com/item?id=28305946
> It wastes your CPU resources and network resources to do something that only benefits apple
Okay, how practical is this problem? iCloud already scans your files to know whether they are need to be synced. And this scan applies lesser times, only once before upload per file.
Encryption does not mess with the filesize, so we are talking about few bytes of extra metadata per image here. Regular image with raw format is tens of megabytes.
> Encryption does not mess with the filesize
No but giant bloom filters take storage.
People are routinely arrested over the results of field drug tests despite their high sensitivity and high probability of false positives.
I dunno man, I’m not to worried about this chain of events.
Building these types of systems is antithetical to a free society.
Can you predict the US' political climate in twenty years' time? No? Then don't build this.
And how do you propose that all of these evil regimes are going to get their images into the NCMEC database? The hash DB will only include photos that are in NCMEC and a second countries’ CSAM database.
And it will be trivial to verify that the hash DB is consistent across different countries.
All they have to do is say: in order to sell iPhones in this country, you must do what we want. Apple will capitulate.
I am russia.
I take actual child porn from my vast kompromat databases (perhaps sent to me helpfully by facebook), or have my agents make some more themselves. They have many talents.
I use adversarial modification to make the child porn match many images of gay pornography popular with the peoples of my country. I add these modified images to my child porn to my databases and also ship it off to the relevant agencies in other countries. It's obviously child porn, so of course they add it.
Apple staff forwards me matches, the images look pornographic and they hit the database. Failure to report child porn you discovered is a felony, so they will error towards reporting.
If for some reason apple doesn't forward on enough of the matches, I hack their severs, kidnap their staff, or simply order them to provide the data they are already collecting (on penalty of not being able to sell in russia anymore). I can continue to use the pretext of searching for child porn to do all this with a smile.
I think this is all obvious enough, and I'm sure the people who work in this business are smarter than we are, have capabilities we can't imagine, and can come up with even better attacks.
The downside it that this makes the human review process tricky. Because there is, understandably, no actual CSAM image to compare the visual derivative with. Images that look similar enough (contain nudity or a child in a swimsuit) may get past the human review process.