Apple’s New ‘Child Safety’ Initiatives, and the Slippery Slope
daringfireball.net
daringfireball.net
“ Fingerprints from images of similar content are not themselves similar. Two photographs of the same subject should produce entirely dissimilar fingerprints. The fingerprints of your own photos of your kids are no more likely to match the fingerprint of an image in NCMEC’s CSAM database than is a photo of a sunset or a fish.”
You and I can’t know this is true!
On a population level scale, everyday people will have their photos analysed by a human working for Apple, and potentially arrested and charged for false positives.
And on a world scale, this will be a tool mandated by governments via legislation to silence dissents, trace the origins of leaks, and can even make possession of things like the Snowden documents illegal.
The NSA is probably thinking about how they give hashes of their top secret documents so they arrest the next Snowden before he/she gets the word out.
I don't understand how he can write this when there is an established record of Apple folding to such demands for access to lucrative markets.
> If they don’t, and these features creep into surveillance for things like political dissent, copyright infringement, LGBT imagery, or adult pornography — anything at all beyond irrefutable CSAM — it’ll prove disastrous to Apple’s reputation for privacy protection.
With all of these possibilities allowed, either the matching technology will miss most slightly-deformed matching images (criminals will quickly find effective minimal distortions), to avoid too many false positives, or will end up matching too many of people’s personal images that consist of their young kids in a state of undress (whose parents don’t have a picture or 2 of them in such a state from their childhood?).
So, either this technology is ineffective and not worth the potential misuse, or it is a grave threat to privacy.
1. What if the NSA/FBI, through the Patriot Act, demand Apple provide bulk interception safety vouchers ('metadata') for "counter-terrorism" purposes? Much like we learned from Snowden that this happens for all cell phone records for decades?
This is not a strawman argument: it is a question concerning applications of laws in force today, that have been applied for mass surveillance already in other contexts.
2. The whole on-device spyware is a single feature toggle away from working on local files too. We're software engineers. We know that "if (iCloudPhotosEnabled) {" is a simple boolean comparison.
Combine the two, and you realise Apple has just built the most intrusive mass surveillance tool in history, disguised as child safety.
Apple's choice of announcing all 3 features together isn't a mistake. This is one of the best PR department in the world. Logic requires us to dig deeper.
While I think your other critiques are reasonable (with an asterisk I'll get back to), this is a little conspiratorial-sounding -- and I would submit that we have a lot of circumstantial evidence from the last ~36 hours of reporting that the PR generated around this has not exactly been positive. I don't see how announcing all of these features together benefits Apple. (And at risk of being a little conspiratorial-sounding myself, I suspect there are other more intrusive mass surveillance tools already built and in use by other corporations and governments.)
As for that asterisk: assuming the service works the way Apple describes it, I don't think it's comparable to cell phone call metadata. You could obviously use the system to search for other known photos besides CSAM and also search photos that are local to the device, but other use cases still need to fit "this photo matches the hash of this other photo" (I'm not even sure if you can abstract "photo" to "other media file"). Photos, of course, have metadata that would be interesting to law enforcement agents, but if this does anything with that metadata then the service doesn't work the way Apple describes it -- and if we assume that Apple is starting out lying about the way the service works, then all bets are off anyway.
I really enjoyed the clarity. Helped me understand what was going on without trying to inject a final conclusion for me.
[1] Credit to "The Economist has yet to see a war it does not like" https://www.prospectmagazine.co.uk/magazine/what-the-economi...
Do you believe the facts he’s provided about how the system works are untrue?
Take any person’s set of photos that they have ever put online, anywhere (publicly, via email, etc.), feed it to this back door and you can have their identity and GPS coordinates in fractions of a second.
It’s a horrifying, dystopian tool that should not exist. In any form, for any reason. Nobody can be trusted to be a gatekeeper for something like this.
It’s clear that the correct solution is to blindly encrypt photos on the user’s phone before uploading them to iCloud. Full stop.
That's very cold comfort. Gruber himself says that this applies to most customers.
Do you disagree with the authenticity of the facts provided?
Why put this system on the phone unless you want to expand it in the future? Cloud encryption with Apple being able to decrypt it could not be the reason. System can be also easily changed anytime from Apple to report on offline images.
If you skipped the article to read the HN comments, I wouldn’t bother. The article’s discussion is much better - just read that.
We have already seen plenty of examples of SWATTING gone wrong where online trolls call fake threats to the police so law enforcement responds and ends up killing innocent people. What's to say this won't happen with this privacy violation too where someone pranks others?
This along with the obvious abuse potential by governments to silence protests, political and cultural movements is a very dangerous edge of the sword. It's very similar to the censorship debate. When the wind blows in the wrong direction (and it eventually will), people will complain. But by then, it will be too late. Just like the patriot act.
Also I find it disgusting how pointing out the obvious dangerous precedent of such privacy/free speech/spying violations gets labelled as "child porn sympathizer" / "racism sympathizer" / "terrorist sympathizer".
I would expect such privacy violating tech to come out of likes of Facebook or Google before Apple which claims to market itself as the privacy pioneer. How did this get through top people at Apple without someone stopping to say "Hey, this might not be a good idea" is beyond me.
Their site still says:
> "What you share from those experiences, and who you share it with, should be up to you."