What was crazy about this? The media outrage around the PR was incoherent to me at the time, and it seemed no one took any time to understand the present realities or details.
What’s crazy about this?
Another way of expressing the concern: does your iPhone work for you, with the help of Apple’s services, or does your iPhone work for Apple? Working for me means not having software designed to report me to the police for how I use my device. The hash database for now includes CSAM hashes; there’s no reason it couldn’t be extended to include similarly heinous material, like Winnie the Pooh memes, Hong Kong freedom posters, rainbow flags, hormone therapy instructions, or anything else offensive to the regime.
People that are comfortable with their device working for a company with the assistance of the user choose Android.
It wasn't like constantly monitoring your private residence, more like setting up a breathalyzer checkpoint on the way to the freeway.
Your apartment complex decides they don’t want to be party to anything illegal. Just in case, they set up a police precinct in the lobby. They set up hidden cameras in the lobby, the hallway leading to your apartment, and the balcony of your apartment. "All public spaces of course" the crowd consents. And if their AI model detects anything suspicious, they send the video to the detective. Because you aren’t doing anything illegal, you have nothing to worry about, right?
Everyone scans for CSAM. I am not conjecturing on the ethics of scanning photos here, I am suggesting that moving from server- to client-side scanning had no effect on any of the things you are ranting about. Hence why I do not understand the outrage.
> Working for me means not having software designed to report me to the police for how I use my device.
Let me fix it: “how I use iCloud Photos, a hosted service on Apple’s servers”.
You agree _in your rental contract_ not to dump explosives in the apartment complex trash compactor, and that the apartment complex has the right to process your trash.
The apartment complex stops looking inside everyone's bags that land in the compactor (a bit late to prevent it from getting into the trash), instead sniffing bags as you put them into the trash chute on your floor, and tagging the bag with a red sticker if it the chemical composition matches a particular pre-registered explosive from a list.
At the other end of the chute, a counter checks if you've dumped a dozen bags with red stickers, and if so, a human may open one to see WTF you're up to. If you're indeed dumping explosives in bulk, they let the bomb squad know.
EITHER WAY, that bag would be headed to the compactor, you are affirmatively putting it there, and any explosives are a breach of agreement. In no other case besides you opening the chute to dump a bag is it sniffing anything else anywhere, except when you open the chute to send these bags to the compactor, bags you decide the contents of.
Once the feature exists your local data might become accessible to a government warrant, which would make the iPhone the opposite of a privacy oriented device.
If it's only for iCloud uploaded data they can simply do the scanning there. There's no reason to use customer's CPU/battery against them.
This is specifically what Apple is trying to avoid - they are intentionally pushing an environment where the user must be involved to get the keys, by way of their account password and/or other enforcement mechanisms designed to ensure only the real user can access such keys.
This is discussed in detail in their Apple Platform Security guide: https://help.apple.com/pdf/security/en_US/apple-platform-sec...
All of the facial recognition, object identification, etc is all done on-device for the same reason. By contrast Google can and does do this in the cloud - and there are Google servers that intentionally have access to decrypt your photos.
iCloud backup was previously a vector that bypassed this, however they also today announced they are fixing that: https://news.ycombinator.com/item?id=33897793 "Apple introduces end-to-end encryption for backups"
But there are many serious problems with it. I'll isolate one: an on-device content surveillance mechanism is a slippery slope to a bad, bad place.
As the saying from the 90s goes, "child porn [CSAM] is the root password to the US Constitution." It is bad enough that people are willing to suspend their better judgement to do something about it.
But after you already have the mechanism accepted an in place, it is far easier to add to it. Pick your boogeyman: you're giving all of them a lovely tool to address their desires. Dissent suppression and Winnie-the-Poo detection for Xi, Erdogan gets to sniff out Golem memes, and choose-your-own horror for the coming dictator of the US.
It is far harder to tell a sovereign, "we could easily do that, but will not" than "we don't have a mechanism to do that." And pretending that it won't happen doesn't pass the laugh test - we have seen this show many, many times. But if you want to argue, start by explaining how Apple's jumping to implement the 10-minute-max sharing limit shows how they'd stand up to China about this.
It would help to note Apple also today announced "Advanced Data Protection" in iCloud which closes the hole where iCloud Backups, iCloud Photos and various other bits of data were technically decryptable by Apple. They've closed that (but it's opt-in, to balance the average user losing all their photos against other users desire to be secure even if it means losing their data). Details: https://support.apple.com/en-us/HT202303#advanced
However even without the "Advanced Data Protection" what I said about not having a workflow for any Apple server to "normally" request both the keys and photo data is also still good security.
Why does that dystopia require on device scanning? Why couldn't they just do it with an OS update today? It's not a reasonably slippery slope, given the actual mechanics of how the CSAM system was designed (perceptual image hashes, not access to arbitrary files)
> There's no reason to use customer's CPU/battery against them.
That's the better argument, but still not super strong. On-device scanning means you know and can verify what hashes are being scanned for, who is providing them, and when they change. Cloud scanning is a complete black box. None of us would know if Google was doing ad-hoc scans of particular users' photos at the behest of random LEOs.
See my comment here:
https://news.ycombinator.com/item?id=33903825
> None of us would know if Google was doing ad-hoc scans of particular users' photos at the behest of random LEOs.
Not your device, not your software. You should assume anything you upload unencrypted is scanned. This distinction was clearly voiced by the majority during the debacle of Apple's on-device scanning proposal. They basically said, "Scanning in the cloud is [choose one: fine, skeezy], but we draw the line at doing on-device scans. I don't want that software on my device."
That would be precisely as difficult to verify as verifying whether your Apple device is currently scanning your content.
This is what Apple was trying to avoid. Scanning on iCloud also requires that Apple can see your photos.
If the scanning is done on device, Apple could encrypt the photos in the cloud so that they can't decrypt them at all. Neither could the authorities.
> There's no reason to use customer's CPU/battery against them.
The amount of processing ALL phones do for phones is crazy, adding a simple checksum calculation in the workflow does fuck-all to your battery life. Item detection alone is orders of magnitude more CPU/battery heavy.
Apple can run software that represents Apple's interests on their own servers.
Running it on their servers means nobody knows how expansive the scans are, and there is nothing preventing ad-hoc scans for arbitrary content for individual users.
The actual scanning and apple self-interest all happened server side.
This was not some Google-style hotdog or not algorithm that was using "AI" to guess if a photo had a nude person on it or not.
2. What's going on with the "fuzzy comparison"?
[0] https://en.wikipedia.org/wiki/National_Center_for_Missing_%2...
Second, the overall aim of the technology also targets legitimate or inadvertent activities, that the law has also unjustly criminalized - say taking family pictures of your kids, 17 year olds sexting, or someone trolling you. Once again unjustly branding people as "child abusers", this time with "evidence" to back up the narrative.
But even in the case of a proper match to the NCMEC list for its bona fide purpose, that still is undermining the phone's owner's interests. That's the philosophical contour, even if you personally wish to brush it aside with the desire to catch people looking at evil images. Our society respects similar longstanding privacy boundaries, even if it does end up helping some "bad people".
It's really not in the NCMEC's interest to respect any of this, as their foundational dynamic is that there is a horrible thing happening in the world, and they must do everything possible to stop it. That kind of advocacy is certainly needed, but we shouldn't just accept their desires as if they're an unbiased neutral party.
Everyone just forgot that in the Apple system you don't get automatically banned and reported to the authorities if you get a match (or five). The matches get manually checked by an actual human, who will see in 0.2 seconds that it's not actual CP, but a highly distorted picture of a cat or some gibberish. Zero action will be taken.
Also: the scanning was only done if iCloud is enabled. If iCloud is enabled, the EXACT SAME scan could be done in the cloud. Why are people not objecting to this with the same fervour?
Which is weird, because it implies anything you used but didn’t buy can and should betray you, and it’s your own fault. You should have been smarter. But it makes a bit of sense, for some odd reason I can’t quite make tangible. When the samsung TV I bought starts showing me ads, when my pricey cable or Hulu package interrupts for commercials, I feel a tinge of this.
So now I think I somewhat get the feeling. Thank you.
The contrapositive isn't automatically true. You can be upset when a cloud host or ISP violates your expectations, even though you don't own them. You can expect Apple to basically work in your interests even on their own servers, since you're paying them.
The ownership thing is more about the structure of society. If computing and communications technology is going to move us forward as a distributed democratic society, then those capabilities must remain distributed throughout the population rather than being controlled by a handful of centralized gatekeepers.
And yes, we've strayed quite far from this ideal. Most people have little control over what their mobile phone does. Still, there is a distinction between a device not having your desired functionality because the manufacturer didn't create it (or even disallowed others from creating it), and the manufacturer deliberately creating functionality that harms users.