Although this is true, the same argument already applies to "your phone might be scanning all your photos and stealthily uploading them" -- Apple having announced this program doesn't seem to have changed the odds of that.
At some point you have to trust your OS vendor.
The best covert exfiltration is when you can hit individual devices in a crowd, so people already have no reason to be suspicious. But you're still leaving tracks - connections, packet sizes etc. if you actually want to do anything and you only need to get caught once for the game to be up.
This on the other hand is essentially the perfect channel for its type of surveillance...because it is a covert surveillance channel! Everyone is being told to expect it to exist, that it's normal, and that it will receive frequent updates. No longer is their a danger a security researcher will discover it, its "meant" to be there.
Letting Apple scan for government-related material on your device is a slippery slope to government surveillance and a hard line should be drawn here. Today, Apple may only be scanning your device for CSAM material for iCloud. Tomorrow, Apple may implement similar scanning elsewhere, gradually expand the scope of scanning to other types of content and across the entire device, implement similar government-enforcing processes on the device, and so on. It's not a good direction for Apple to be taking, regardless of how it works in this particular case. A user's device is their own personal device and anything that inches toward government surveillance on that device, should be stopped.
Another point made was that government surveillance never happens overnight. It is always very gradual. People don't mean to let government surveillance happen and yet it does because little things like this evolve. It's better to stop potential government surveillance in its tracks right now.
[0] I understand the emissions cheating was not supossed to be open, but a relatively open system allowed said student to take a peek and see what was going on.
Read evidence from past trials it is obvious. See also successful and failed attempts to subpoena this info from VPN services.
Only people with iCloud will be using the relay.
It is true on the surface the photos is disconnected from the use. However, Apple only needs a solid answer that handles the bad optics of what you can do with the Tor-like anonymity of iCloud Privacy Relay.
However, if you look more closely, the CSAM service and its implementation are crafted exactly around the introduction of the relay.
It's better to be forthright, or you risk your valuation on the whims of a single employee.
On the other hand, once Apple has written a backdoor enthusiastically themselves, it's a lot easier to force someone to change how it can be used. The changes are small and compliance can be immediately verified and refusal punished. To take it to its logical extreme: you cannot really fire or execute people who delay something (especially if you lake the expertise to tell how long it should take). But you can fire or execute people who refuse to flip a switch.
This technology deeply erodes Apple and its engineers' ability to resist future pressure. And the important bit here is there adversary isn't all powerful. It can coerce you to do things in secret, but its power isn't unlimited. See what happened with yahoo.[0]
https://www.reuters.com/article/us-yahoo-nsa-exclusive/exclu...
And if your phone has the capability to upload to the cloud, then you have to trust your OS vendor to respect your wish if you disable it, etc.
It's curious that this is the particular breaking point on the slope for people.
The "on device" aspect just makes it more immediate feeling, I guess?
In other words, extracting the code and analysing it to determine that it does do what you expect is, although not easy, still legal. But the source, the CSAM itself, is illegal to possess, so you can't do that verification much less publish the results. It is this effective legal moat around those questioning the ultimate targets of this system which people are worried about.
So in a world where Apple pushes you to set up icloud photos by default, and can do whatever they want there, and other platforms have been doing this sort of of thing for years, it's a bit startling that "on device before you upload" vs "on uploaded content" triggers far more discontent?
Maybe it's that Apple announced it at all, vs doing it relatively silently like the others? Apple has always had access to every photo on your device, after all.
There's certainly a slippery-slope argument, where some future update might change that scanning behavior. But the system-as-currently-presented seems similarly trustable.
And that's assuming they weren't actively hiding anything by e.g. splitting them up into chunks that could be slipped into legitimate traffic with Apple's servers.
The most-optimistic take on this I can see is that this program could be the prelude to needing to trust less people. If Apple can turn on e2e encryption for photos, using this program as the PR shield from law enforcement to be able to do it, that'd leave us having to only trust the OS vendor.
What I find interesting is that so many people find it worse to do it on device, because of the risk that they do it to photos you don't intend to upload. This is clearly where Apple got caught off-guard, because to them, on-device = private.
It seems like the issue is really the mixing of on-device and off. People seem to be fine with on-device data that stays on-device, and relatively fine with the idea that Apple gets your content if you upload it to them. But when they analyze the data on-device, and then upload the results to the cloud, that really gets people.
But that ship has long sailed, right?
Every packet that leaves a device potentially incriminates its owner. Every access point and router is a potential capture point.
A device I own should not be allowed to collect and scan my data without my permission.
It's not scanning; it's creating a cryptographic safety voucher for each photo you upload to iCloud Photos. And unless you reach a threshold of 30 CSAM images, Apple knows nothing about any of your photos.
I don't think anyone's necessarily being too upset or paranoid about THIS, but maybe everyone should also be a little less trusting of every closed OS - macOS, Windows, Android as provided by Google - that has root access too.
Apple has built iOS off user trust & goodwill, unlike most other OSes.
Where does that fit in your set intersection?
...except you can't? Not sure where these assumptions come from.
Well they could not do it.
I gave a pretty obvious and clear answer to that, and apparently you didn't care about the question in the first place, and have now misdirected to something else.
I am also not sure what possible definition of "privacy" that you could be using, that would not include things such as on device photo scanning, for the purpose of reporting people to the police.
Like, lets say it wasn't Apple doing this. Lets say it was the government. As in, the government required every computer that you own, to be monitored for certain photos, at which point the info would be sent to them, and they would arrest you.
Without a warrant.
Surely, you'd agree that this violates people's privacy? The only difference in this case, is that the government now gets to side step 4th amendment protections, by having a company do it instead.
And the answer is that they shouldn't implement things that violate people's privacy, such as things that would be illegal for the government to do without a warrant.
That is the answer. If it is something that the government would need a warrant for, then they shouldn't do it, and doing it would violate people's privacy.
If the OS was open source and supported reproducible builds, you would not have to trust them, you could verify what it actually does & make sure the signed binaries they ship you actually correspond to the source code.
Once kinda wonders what they want to hide if they talks so much about user privacy yet don't provide any means for users to verify their claims.
Changes to binding contractual terms that allow broad readings and provide legal justification for future overreach are dangerous. If they really are serious that they are going to use these new features in a highly limited way then they can put their money where their mouth is and add legally binding contractual terms that limit what they can do with serious consequences if they are found to be in breach. Non-binding marketing PR assurances that they will not abuse their contractually justified powers are no substitute for the iron fist of legal penalty clause.
I guess it comes down to that I don't trust an OS vendor that ships an A.I. based snitch program that they promise will be dormant.
This being an area people are paying attention to makes it less likely they'll do unpopular things involving it, from a pure "we like good PR and profits" standpoint. They might sneak these things in elsewhere, but this specific on-device-scanning program has been shown to be a risk even at its current anodyne level.
Yes, and we were trusting Apple. And now this trust is going away.
Is it really? There are some very loud voices making their discontent felt. But what does the Venn diagram look like between 'people who are loudly condemning Apple for this' and 'people who were vehemently anti-Apple to begin with'?
My trust was shaken a bit, but the more I hear about the technology they've implemented, the more comfortable I am with it. And frankly, I'm far more worried about gov't policy than I am about the technical details. We can't fix policy with tech.
Yeah. I don't really understand the tech utopia feeling that Apple could simply turn on e2ee and ignore any future legislation to ban e2ee. The policy winds are clearly blowing towards limiting encryption in some fashion. Maybe this whole event will get people to pay more attention to policy...maybe.