Interview: Apple’s Head of Privacy details child abuse detection
techcrunch.com
techcrunch.com
> This is an area we’ve been looking at for some time, including current state of the art techniques which mostly involves scanning through entire contents of users libraries on cloud services that — as you point out — isn’t something that we’ve ever done; to look through user’s iCloud Photos.
There were some reports that Apple has been scanning iCloud Photos server-side already; apparently that hasn't been the case ever.
Also, on client-side scanning still running if the user has turned off iCloud Photos:
> If users are not using iCloud Photos, NeuralHash will not run and will not generate any vouchers. CSAM detection is a neural hash being compared against a database of the known CSAM hashes that are part of the operating system image. None of that piece, nor any of the additional parts including the creation of the safety vouchers or the uploading of vouchers to iCloud Photos is functioning if you’re not using iCloud Photos.
Seems pretty clear now.
This article from Forbes notes otherwise, citing a warrant filed in Seattle for a case.
https://www.forbes.com/sites/thomasbrewster/2020/02/11/how-a...
I get the sense the interview is saying that they haven't scanned photos - like looking at actual contents - but they've always checked hashes. It is super confusing though, and I'd like to see Apple clarify this further.
> Well first, that is launching only for US, iCloud accounts, and so the hypotheticals seem to bring up generic countries or other countries that aren’t the US when they speak in that way, and the therefore it seems to be the case that people agree US law doesn’t offer these kinds of capabilities to our government.
Apple's already said they're going to launch this in other countries - it doesn't seem hypothetical. https://9to5mac.com/2021/08/06/apple-says-any-expansion-of-c...
I sense Apple's scrambling to respond to people's valid concerns. They should have just spent a few more weeks working on the PR before announcing this, anticipating people's attachment to their privacy, and maybe offering users a benefit to this more invasive system - like true E2EE for photos, backups, and drive files.
> And the idea that the most valuable thing that an attacker — who’s undergone such an extremely difficult action as breaching someone’s device — was that they would want to trigger a manual review of an account doesn’t make much sense. [...] and that is something that we want to only occur in cases where it’s a legitimate high value report. We’ve designed the system in that way, but if we consider the attack scenario you brought up, I think that’s not a very compelling outcome to an attacker.
It seems that Apple considers that scenario unlikely, although I think it would make a lot of sense for an attacker to choose that avenue, since they get Apple to do the reporting for them.
However, Apple also claims that even if such scenario would happen, their threshold limits and manual review would make such attacks worthless, but they do not elaborate as to how those checks would actually filter out those attacks.
However, one would expect that an anonymous tip trick is common enough to make someone skeptical at some point.
With this new system it will look like “you” self-reported yourself, no one else to question, plus Apple build a reporting pipeline with express expectation of “high-quality” actionable reports.
So it will drag on for much longer simply because “computer said so” and no real way to disprove any of that.
That’s not how it works.
If a device has CSAM on it and it reaches a particular threshold, Apple gets notified and reviews it before any other entity gets involved, like NCMEC or law enforcement.
It's also extremely difficult for someone to break into an iPhone and plant CSAM on it without the user knowing.
Yes, a determined state-sponsored attacker could possibly do it but that means you got an entirely different set of problems anyway.
The point is that it's highly unlikely that a disgruntled employee or mentally disturbed roommate could pull this off, which is what 99.999% of us have to worry about.
Many people will fall victim to scumbag hackers that will do it just for fun.
Your fake image also has to match the visual derivative, which isn’t possible since you don’t have access to the original image.
No technical or explanatory PR can remove the actual problem: In the name of "$something good" Apple will scan on user devices. Are we so addicted to "shiny toys and dopamine spikes" that cannot stand for personal freedom?
How about due process? Is "innocent until proven guilty" dead?
How about transparency? If a digital system is designed to "police" everyone, is it not in a public interest to be scrutinized and open source?
This tools are invasive and create a precedent (on device scanning) for others to follow. How much time we have to prepare ourselves that we will live in "Global China"? Normalization of this technology and social acceptance of invasive forms of surveillance will lead to global social rating system. And historically systems like this had been abused and used against the public.
What "magic spell" is over us? We are trading fundamental freedoms and handing moral and legal judgement over automated systems on the premise that "Big Apple" and "Big Google" know better and can solve every problem with software?
Are we so stupid? How long until using Linux and VPN is illegal?
Apple Social Goodness = China Social Rating System. Period. How long until your thoughts are directly monitored by "AI" system? What is next? Think Different? No. I am thinking critically. F* Apple. F* Google. F* Microsoft. F* Facebook. And big F*U for all politicians who are behind this "idea".
This is not for CSAM. This is the tech for Digital ID and AI Police of the near future.
I have lived under communistic regime, trust me I know the signs. The big difference is that this incarnation will be brutally efficient. You don't need people to spy for you, you will have global automation for this.
Apple doesn't scan your device. And none of the machinery runs at all if you don't use iCloud Photos.
On the other hand, there's no way to opt out if you use OneDrive, Dropbox, since they do scan your photos if you store them there.
So far, everything we know makes sense and checks out on a policy and technology level.
The rest of the industry scans for the same CSAM in the cloud, which is less privacy preserving. It appears Apple has gone out of their way to create a system that preserves privacy unless you surpass a certain threshold of CSAM while and use iCloud Photos.
Even the feature for minor children on a Family sharing plan is opt-in by the parents of those children; the machine learning to identify potentially dangerous content is on device.
>I would be more positively disposed to believing they're doing this for the greater good had they not circulated an internal memo calling the opposition to this the "screeching voices of minority". I will admit, I am extremely cynical when it comes to the topic of privacy and tech but I lean towards the belief that a company's internal communication is more honest about what their attitude is to these topics and features than the words of their PR department.
Your comment is worded in such a way that it represents the internal memo to be Apple's position, which is misleading to read.