Hacker News Reader

Comment by mindwok | Hacker News Reader

It's my understanding that Apple simply can't operate in China without playing by those rules. So really, the onus is on the CCP.

In this case, is Apple being compelled to do this by the US government? Or is it a choice Apple has made purely internally? I think that makes a difference.

View on HN

I agree the question of whether Apple was compelled by whatever government (or if they did this voluntarily) has implications on the ethics of these decisions. They may genuinely have no choice.

But I don’t see how it affects the question of whether Apple’s privacy assertions are trustworthy.

>> They may genuinely have no choice.

There is a choice. Don't comply and have the CCP make you stop selling there.

Apple has no principles that can't be tossed aside in exchange for a large market - in other words a lot of money. This should not be unexpected.

Well put. Perhaps it’s cynical, I think that’s true of any corporation of Apple’s size: their only true principle is to maximize shareholder value.

The best we can do as privacy-concerned mere-mortals is to take our business where those profit incentives align with our values. Apple’s put a lot of work into advertising that their profits are aligned with values of privacy, but some signs say otherwise.

You shouldn't expect privacy from a third party. You'll be disappointed by definition.

The CCP would make Apple stop manufacturing in China, which would effectively mean that they couldn't sell anything anywhere. Apple does have very deep pockets, but that would be an existential threat.

Maybe similar pressure was placed on them here and we just aren't privy to it.

ding ding we have a correct answer

This is a claim which should require extraordinary evidence, since Apple has very publicly resisted pressure to build technology at the government’s behest in the past.

It’s now common knowledge (as mentioned in the article) that Apple refrained from adding a feature at the government’s behest in the past. It’s a fine line between not adding a feature they don’t like and adding a feature they do.

This claim is often repeated but the only source for it I’ve found is reporting in Reuters citing anonymous sources. The user experience challenges of end to end encryption are immense, especially since iPhone is many users’ only iCloud client, and I find it hard to believe Apple was moments away from announcing a solution to them but the FBI pressured them out of it. That is not the extraordinary evidence such a claim needs. For example, Bloomberg reported an even better sourced story about supply chain compromises to Apple’s cloud services which has been more or less entirely debunked.

In addition, the emphasis on the on-device portion of this scanning project is evidence that Apple views losing access to iCloud data as part of its roadmap.

The Bloomberg claims were explicitly denied by Apple and several other companies. To the best of my knowledge, Apple has never publicly denied the Reuters reporting, and explicitly declined to comment when given the chance by Reuters. It’s certainly one thing to extend the benefit of the doubt to a company in a dispute with a reputable news agency; it’s entirely another thing to take issue with the claim when even the affected company won’t do so.

So you are suggesting that FBI pressure is the primary reason Apple did not pursue plans to end to end encrypt iCloud Backups on the basis of one news article and lack of comment from Apple?

(There is other counter-evidence: Apple rarely comments on speculation. In an interview with the WSJ, Apple’s answer to why now was that they figured out how to do known CSAM detection in a way they felt met their privacy requirements. The omission is at least slightly informative, if you think FBI pressure is critical. Finally, people more familiar with the legal context have argued it would jeopardize the program for there to be evidence that Apple is doing this work in response to FBI pressure as suggested. Finally, Tim Cook offered a more straightforward explanation and vision for iCloud end to end encryption in an interview with a German newspaper:

SPIEGEL ONLINE: Is the data as secure on your iCloud online service as on the devices?

COOK: Our users have a key there, and we have one. We do this because some users lose or forget their key and then expect help from us to get their data back. It is difficult to estimate when we will change this practice. But I think that in the future it will be controlled like the devices. We will therefore no longer have a key for this in the future. )

If so, what information would change your mind? How confident are you that this is the full story?

I have specific reasons to believe that Apple has been subject to legal pressure. But if you didn’t believe six anonymous sources in a story by a reputable reporter, you’re not going to believe my secondhand reports either. Skepticism is fine: stubborn unfounded skepticism in the absence of a direct confirmatory statement from Apple isn’t possible to argue with.

Apple being legally pressured is not the full story. It is absolutely true that they have been pressured by the FBI and others, and simultaneously that they also have real concerns about user experience with lost backups. If you read the Reuters story, it doesn’t draw a straight line from the FBI to the backup situation, it just points out that legal pressure is a factor in Apple’s reasoning. Apple spent a lot of money building an E2EE key vault based on HSMs several years ago, and it’s also fairly obvious that they had bigger plans than securing passwords and browser history. Yet they have not made full E2EE backup available even as an option for advanced users, despite the fact that even Android now supports E2EE backups. And prior to enabling E2EE backups (one assumes that’s coming this year) they paused to build exactly the on-device scanning system that law enforcement has been exhorting cryptographers to build since William Barr’s letter in 2018. It does not take a great deal of imagination to see the pattern, but obviously only Tim Cook can prove it to your satisfaction.

ETA: Just to take this a step beyond “someone is arguing on HN”: this argument matters because I think we all intuitively understand how dangerous this system would be in a world where Apple’s engineering is responsive to government pressure. Your skepticism makes perfect sense if you want to believe this system is secure. I wish I could live in a world where I was able to share that skepticism, it would be a more relaxing place.

I’m not sure why it’s obvious to you that Tim Cook must personally whisper into my ears otherwise. The FBI and every other intelligence agency is probably pressuring Apple all the time. Elsewhere in the thread, I even say that I think law enforcement pressure is one reason Messenger has not turned on E2E by default. I understand how this works.

What you haven’t convinced me of is whether Apple’s priorities are being driven by the pressure. Apple can believe keeping known CSAM off their services is important, and just because someone else agrees doesn’t mean the outside party was critical or the cause of the decision. We live in a society where there are lots of non-government reasons to not be the world’s #1 CSAM host, especially as the famously anti-porn company.

To what extent Apple’s intentions are sincere or coerced is important to suss out because it changes the likelihood that Apple, in the long term, will build different features that endanger its users. I agree that the platform vendor is “intuitively” a source of risk, but I don’t think what they’ve announced is any more (technically) dangerous than anything else my device already did. Even if Apple is outright lying about the contents of the hash database and what their human reviewers will flag, they could’ve been outright lying about whether they slurp my iCloud Photo Library straight out off iCloud with the keys they escrow. Besides that, there is no other possibly untoward behavior that I can’t verify locally. In fact, if Apple built iCloud scanning, I’d be at least as concerned about future features, because there I have no audit rights.

I don’t “want to believe” the system is secure - I have the tools to confirm that the system exposes me to no risk that I’m not already comfortable with as an (for sake of argument) iCloud Photo Library user, and almost all of the other risks are hypothetical. I’m even open to believing that the other risks are more probable today than a month ago, but the evidence isn’t very strong. Some evidence that would change my mind: any information about NCMEC being compromised by nation states and Apple ignoring that evidence, any evidence from Apple sources stating that they worked with the FBI on this system design, any evidence that Apple is expanding the system beyond CSAM.

Which brings me back to a question you never answered: how confident are you that the system presages generalized full device content scanning, and what evidence would change your mind?

I never said the device presages full-device content scanning. All I’ve said (including in this NYT op-ed [0]) is that it enables full-device scanning. Apple’s decision to condition scanning on a toggle switch is a policy decision and not a technical restriction as it was in the past with server-side scanning. Server-side scanning cannot scan data you don’t upload, nor can it scan E2EE files. Most people agree that Apple will likely enable E2EE for iCloud in the reasonably-near future, so this isn’t some radical hypothetical — and the new system is manifestly different from server-side scanning in such a regime.

Regarding which content governments want Apple to scan for, we already have some idea of that. The original open letter from US AG William Barr and peers in 2018 [1] that started this debate (and more than arguably led to Apple’s announcement of this system) does not only reference CSAM. It also references terrorist content and “foreign adversaries’ attempts to undermine democratic values and institutions.” A number of providers already scan for “extremist content” [2], so while I can’t prove statements about Apple’s intentions in the future I can only point you to the working systems operating today as evidence that such applications exist and are being used. Governments have asked for these, will continue to ask for them, and Apple has already partially capitulated by building this client-side CSAM system. That should be an important data point, but you have to be open to considering such evidence as an indication of risk rather than intentionally rejecting it and demanding proof of the worst future outcomes.

Apple has also made an opinionated decision not only to scan shared photos, but also to scan entire photo libraries that are unshared with users. This isn’t entirely without precedent, but it’s a specific deployment decision that is inconsistent with existing deployments at other providers such as Dropbox [3] where scanning is (allegedly, according to scanning advocates) not done on upload, but on sharing. Law enforcement and advocates have consistently asked for broader scanning access, including unshared files. Apple’s deployment responds to that request in a way that their existing detection systems (and many industry standard systems) did not. Apple could easily have restricted their scans to shared albums and photos as a means to block distribution of CSAM: they did not. This is yet another difference.

I’m not sure how to respond to your requests for certainty and proof around future actions that might be taken by a secretive company. This demand for an unobtainable standard of evidence seems like an excellent way to “win” an argument on HN, but it is not an effective or reasonable standard to apply to an unprecedented new system that will instantly affect ~1 billion customers of the most popular device manufacturer in the world. There is context here that you are missing, and I think suggesting more reasonable standards of evidence would be more convincing than your demands for unobtainable proof of Apple’s future intentions.

[0] https://www.google.com/amp/s/www.nytimes.com/2021/08/11/opin...

[1] https://www.justice.gov/opa/press-release/file/1207081/downl...

[2] https://www.google.com/amp/s/amp.theguardian.com/technology/...

[3] see p.8: https://www.europarl.europa.eu/RegData/etudes/BRIE/2020/6593...

> Apple’s decision to condition scanning on a toggle switch is a policy decision and not a technical restriction as it was in the past with server-side scanning.

This is not a meaningful distinction. There are many security and privacy protections of iOS that are equivalently "policy" decisions: letting iCloud Backups be turned off; not sending a copy of your device passcode to Apple servers; not MITMing iMessage which has no key transparency; not using existing Photos intelligence to detect terrorist content etc. In technical terms, there are many paths to full device scanning, and some of those paths were well trodden even a month ago (iCloud Backup and Spotlight, for starters, and Photos Intelligence as a direct comparison).

Making this claim also requires showing that the likelihood of Apple making one of many undesirable policy decisions has changed.

> Regarding which content governments want Apple to scan for, we already have some idea of that.

I asked about what Apple will scan for, not what governments want them to scan for. Again, I see a pattern in your argument where you state what government wants and then don't state how that desideratum translates into what Apple builds. The latter is entirely the source of ambiguity for me.

> Apple’s deployment responds to that request in a way that their existing detection systems (and industry standard systems) did not.

I could see that. If that's what Apple had built, would you have a different take on the system? It seems like no -- most of the risks you care about are unchanged since you are operating in a world of "policy decision" equivalence classes.

> I’m not sure how to respond to your requests for certainty and proof around future actions that might be taken by a secretive company.

You seem to misunderstand what I said. I'm asking for an estimate of your certainty, not absolute certainty. Furthermore, I'm asking you to provide examples of what information would change your mind for the same reason you keep repeatedly calling me stubborn and accusing me of bad faith: without it, I have no idea whether we are engaged in discussion or a shouting match.

Therein lies a problem. Most people would agree that a good predictor what people will do is what they have done in the past. If you read through some of the stories ( those of Snowden come to mind ), some declassified information over the course of the past few decades, a pattern emerges.

There is no evidence, either because it does not exist or because it is hidden. The best we have is inference and whistleblowers.

That said, I genuinely think we are not being tinfoil enough these days. And that is based only on what we know ( or at least avg. citizen should know ) was already done in the past.

The PRISM revelations, to this day, are very ambiguous about their implications for cooperation. When they came out, most involved companies flat out denied cooperation. The types of data the NSA claimed to get were available by tapping into network backbones. Unless you are aware of a theory or evidence I’m not, I think it’s just as likely that the program described in the leaked slides involved unilateral or covert intrusion by the NSA rather than cooperation.

It is reasonable to be conservative about data stored in someone else’s cloud, and there is undeniable value to end to end encryption that gives you control over who can access it. That said, especially if you read Apple’s letter in response to the PRISM allegations, Apple’s behavior seems quite consistent and sincere over time: https://www.apple.com/apples-commitment-to-customer-privacy/.

I don’t think it’s likely they designed this feature under pressure from the government or with the intention to expand it to local data on your device.

Ok. If that is not the reason, then the question becomes what is the real reason.

Some analysts seem to think Apple should be getting into advertising business, which would partially explain some of the proposed updates. Naturally, if that were the case, it would render Apple's commitment to privacy about as useful as T-Mobile's. Then again, I might be giving Apple too much crap. Most companies don't even pretend to care.

https://www.marketwatch.com/story/this-could-be-apples-next-...

I think Apple (i) genuinely believes scanning on device is better for privacy because it lets users (theoretically) confirm the behavior of the system and (ii) is learning from others’ mistakes of deploying scanning server side and having that become a blocker to moving to end to end encryption (e.g. Facebook Messenger). My guess is they announce expansions to end to end iCloud behavior soon. Features like this are building blocks to controlling who escrows your iCloud private keys: https://gadgettendency.com/apple-allowed-to-bequeath-and-inh...

Interesting. That did not occur to me. Thank you for sharing this. Lets see where this goes.

You’re welcome, and I’m happy to share alternative theories. I think some of the issues of control that have been raised are legitimate. For example, you brought up the idea that Apple is doing this as part of a move to advertising, perhaps because advertisers want to be reassured their ads don’t appear next to distasteful content. I think there’s an interesting idea there: lots of companies are trying to move towards “on-device” ads; are they really private? Is privacy the right framework to evaluate them, or is it a different type of control?

> since Apple has very publicly resisted pressure to build technology at the government’s behest in the past.

And they've also not done that. When Jobs died Apple promptly bent the knee and joined up to PRISM.

Very publicly resisted? Nope. They went along, very quietly (it was Yahoo that tried to fight back). We only found out about Apple having joined up thanks to the man that wrote this article.

How many human rights atrocities does Apple get to partake in before their credibility is shot, such that the burden is put on them instead, of proving - in such circumstances as this one - that they're not commiting more atrocities.

Perhaps. However Apple has since released two security updates to iOS but has not patched the iMessage flaw that allows Pegasus software to spy on thousands (perhaps millions) of iphones.

What are they waiting for? Hmm perhaps getting something else in place first.

You are saying known CSAM detection for iCloud Photo Library will launch before the Pegasus 0-days are fixed? The two are entirely different. If Apple was working on behalf of the government, they could’ve already shipped over the contents of iCloud for all the users targeted.

In any case, I’d happily take the other side of the bet at even odds that the security issue is patched after the child safety program goes live.

> is Apple being compelled to do this by the US government? Or is it a choice Apple has made purely internally? I think that makes a difference.

You're being downvoted but it's a critical issue.

If Apple is currently being compelled to do this, it likely means the US Government has a massive new privacy obliterating program underway and Apple probably isn't the only tech giant joining the human rights violation parade. It's important to find out if that's going on. We can be certain they didn't stop with PRISM.

If it turns out to be the case, that Apple has joined up to another vast human rights violating program (they already did it at least once before, remember), the US needs to move forward toward Nuremberg-style trials for all involved Apple management and all involved Apple employees (and not only them). That's the only way it stops.

Such human rights violations should not be allowed to continue. How many tech employees at these companies got away with extraordinary human rights violations related to PRISM? Employees at these companies were responsible in part and critical to helping to make it happen. Who are these enablers? Why aren't they in prison? Why is this so rarely discussed on HN? (yeah we all know why)

HN is pretty amusing about this topic. Privacy is a human right? Yeah? Also universally HN: but let's not talk about the people actually responsible for the human rights violations; let's not talk about all the techies being paid princely sums to commit human rights atrocities. Let's not talk about prison sentences for what they've done to their fellow humans. Let's not hold tech employees responsible.

The Jan 6 commission is using the riots as a pretext to collect and inspect private speech/communications from a huge list of people who had absolutely nothing to do with January 6 [0] but were politically active for Trump or his campaign or just posted memes on Twitter etc.

The massive list of people they're demanding records for is shocking. Privacy doesn't mean anything to the current establishment, if people could just take off their partisan blinders for two seconds they would realize this and we could probably form a plea to congress as unified voice.

Give this CSAM system another few years and they won't even have to subpoena for most of the private communication they're already going after today.

https://www.forbes.com/sites/andrewsolender/2021/08/25/jan-6...

Reply on news.ycombinator.com