Hacker News Reader

Comment by scosman | Hacker News Reader

I think I'm saying: you're not sending "your data" off device. You are sending a homomorphically encrypted locally differentially private vector (through an anonymous proxy). No consumer can really understand what that means, what the risks are, and how it would compare to the risk of sending someone like Facebook/Google raw data.

I'm asking: what does an opt in for that really look like? You're not going to be able to give the user enough info to make an educated decision. There's ton of risk of "privacy washing" ("we use DP" but at very poor epsilon, or "we use E2E encryption" with side channel data gathering).

There's no easy answer. "ask the user", when the question requires a phd level understanding of stats to evaluate the risk isn't a great answer. But I don't have another one.

View on HN

In response your second question, opt in would look exactly like this: don't have the box checked by default, with an option to enable it: "use this to improve local search, we will create an encrypted index of your data to send securely to our servers, etc..." A PhD is not necessary to understand the distinction between storing data locally on a machine vs. on the internet.

Even here with HN crowd: it's not an index, it's not stored on a server, and it's not typical send-securely encryption (not PK or symmetric "encrypted in transit", but homomorphic "encrypted processing"). Users will think that's all gibberish (ask a user if they want to send an index or vector representation? no clue).

Sure, you can ask users "do you want to use this". But why do we ask that? Historically it's user consent (knowingly opting in), and legal requirements around privacy. We don't have that pop up on any random new feature, it's gated to ones with some risk. There are questions to ask: does this technical method have any privacy risk? Can the user make informed consent? Again: I'm not pitching we ditch opt-in (I really don't have a fix in mind), but I feel like we're defaulting too quickly to "old tools for new problems". The old way is services=collection=consent. These are new privacy technologies which use a service, but the privacy is applied locally before leaving your device, and you don't need to trust the service (if you trust the DP/HE research).

End of the day: I'd really like to see more systems like this. I think there were technically flawed statements in the original blog article under discussion. I think new design methods might be needed when new technologies come into play. I don't have any magic answers.

> I think there were technically flawed statements in the original blog article under discussion.

Such as?

The third choice, after opt-in and opt-out is to force the user to choose on upgrade before they can use their device again. "Can we use an encrypted, low-resolution copy of your photos that even we ourselves can't see?"

Okay except "encrypted, low-resolution copy of your photos" is an incredibly bad explanation of how this feature works. If nobody on HN so far has managed to find an explanation that is both accurate and understandable to the average consumer, any "hey can we do this" prompt for this feature is essentially useless anyways. And, IMO, unnecessary since it is theoretically 100% cryptographically secure.

I think it's sufficiently accurate, why don't you think it is? I don't think the vector vs low-res aspect is particularly material to understanding the key fact that "even we ourselves can't see?"

I Think the best response is make it how iCloud storage works. The option is keep my stuff on the local device or use iCloud.

Exactly. It's the height of arrogance to insist that normal users just can't understand such complex words and math, and therefore the company should not have to obtain consent from the user. As a normal lay user, I don't want anything to leave my device or computer without my consent. Period. That includes personal information, user data, metadata, private vectors, homomorphic this or locally differential that. I don't care how private Poindexter assures me it is. Ask. For. Consent.

Don't do things without my consent!!! How hard is it for Silicon Valley to understand this very simple concept?

Every TCP session leaks some PRNG state for the ISN. That might leak information about key material.

Every NTP session leaks time desync information, which reveals—on modern hardware—relativistic travel, including long airplane trips.

Every software update leaks a fortune about what you run and when you connect.

I don’t think it’s reasonable to ask that people consent to these; I don’t think they can. I absolutely agree that photo metadata is different and at a way higher level of the stack.

This, 1000x. Thank you for voicing the absurdness of their approach to 'consent'.

The average smartphone is probably doing a hundred things you didn’t knowingly consent to every second.

Should Apple insist that every end user consents to the user agent string sent on every HTTP request?

> The average smartphone is probably doing a hundred things you didn’t knowingly consent to every second.

You've succinctly identified a (maybe the) huge problem in the computing world today. Computers should not do anything without the user's command/consent. This seems like a hopeless and unachievable ideal only because of how far we've already strayed from the light.

Even Linux, supposedly the last bastion of user control... it's a mess. Do a fresh install and type ps ax at a shell. You'll see dozens of processes in the background doing god knows what. I didn't consent to any of this! The distribution's maintainer simply decided on my behalf that I want the computer to be running all these processes. This is totally normalized!

I don't expect my computer to ask for consent again and again for every byte sent over the network, but I do expect it to obtain my consent before generally accessing the network and sending bytes over the network.

"The light" you claim is that users should have the knowledge and discernment to consent to what a computer does.

To me, there's never been a case, except maybe in the first decade or so of the hobby/tinkering PC movement, where most users had this ability.

Should we just not use computers?

> Should we just not use computers?

I don't think "should we just give up?" is a reasonable question to anything.

> I do expect it to obtain my consent before generally accessing the network and sending bytes over the network.

How would that make any difference in this case? Presumably, you'll have long-ago checked the "allow general access to the network" setting, so you've given consent to the "send my photo data" action. Heck, surely connecting to the internet in the first place is implicit consent that you want to send stuff over the network?

If I were actually given the choice, I would not check any checkbox allowing an application broad, unfettered access to the network. But, in most cases I'm not even given that choice!

> I didn't consent to any of this!

Yes you did. You purchased a computer, put this software on it and executed it. If you didn't want it to do whatever it's doing you should have determined what it would do beforehand and chose not to do it.

> whatever it's doing

Even assuming that running the software implies my consent (which I would dispute), how do I make the decision about whether I should execute the software if I don't know what it is doing?

This all-or-nothing approach is also problematic. I should not have to allow the developer free rein to do whatever he wants, as a condition of using the software. This is why operating systems are slowly building granular permissions and consent checks.

Installing and booting Linux absolutely implies consent to let it do what it does. It's open source, you can evaluate what it does before booting it. You know it's comprised of many processes, you know it has a networking stack, you connected it to a network. You can't then ask OMG why didn't it ask before sending something?

I agree that all-or-nothing is problematic but even with a flexible permission system the best you can hope for is for all the things apps do to be itemized and set to sane defaults. But even then sanity is subjective. For every person like you (and me fwiw) who values privacy there are 1000 people who will never find the settings, don't care about privacy, and will wonder why stuff isn't working.

Ultimately privacy is similar to security in that it comes down to trust. If you don't trust your OS you're screwed. Your choices are try to exert as much control over it as possible, or don't use it.

That's not how informed consent works.

> You've succinctly identified a (maybe the) huge problem in the computing world today.

And getting downvoted for saying it, which is a fascinating incongruity.

> incongruity

Or signal of non-named stakeholders.

It’s amazing how hostile Silicon Valley (and HN commenters) are to the basic idea of consent. It’s as if simply asking the user for permission is a grave insult to these technologists. “I shouldn’t have to ask permission! It implies I’m doing something bad!” they might be thinking.

If the world was a nightclub, “Silicon Valley” would be a creepy guy who walks up to every woman and says “You’re now dating me. To stop, you need to opt out using a form that I will do my best to make sure you can’t read.”

You're inverting morality and infantilising the consumer. Apple is a corporation. Corporations don't owe you moral anything, except as required by law.

Choosing an Apple product is consent to trusting Apple. Continued use their products represents ongoing consent. This is an objective fact about all complex connected devices and it cannot possibly be otherwise.

Corporation are driven by people. They’re not a separate entity that decides to do things while their owners are sleeping. Every actions have someone that suggested it and someone that gave the green light.

Corporations are driven by shareholders, through the board of directors, through the c-suite, which have a fiduciary obligation to maximise profits.

There is significant middle ground between "do it without asking" and "ask about every single thing". A reasonable option would be "ask if the device can send anonymized data to Apple to enable such and such features". This setting can apply to this specific case, as well as other similar cases for other apps.

Asking the user is perfectly reasonable. Apple themselves used to understand and champion that approach.

https://www.youtube.com/watch?v=39iKLwlUqBo

If you can't meaningfully explain what you're doing then you can't obtain informed consent. If you can't obtain informed consent then that's not a sign to go ahead anyway, it's a sign that you shouldn't do it.

This isn't rocket surgery.

+100 for "rocket surgery".

I mostly agree. I'm just annoyed "this new privacy tech is too hard to explain" leads to "you shouldn't do it". This new privacy tech is a huge net positive for users.

Also: from other comments sounds like it might have been opt-in the whole time. Someone said a fresh install has it off.

> This new privacy tech is a huge net positive for users.

It's a positive compared to doing the same "feature" without the privacy tech. It's not necessarily a positive compared to not forcing the "feature" on the user at all.

The privacy tech isn't necessarily a positive as a whole if it leads companies to take more liberties in the name of "hey you don't need to be able to turn it off because we have this magical privacy tech (that nobody understands and may or may not actually work please don't look into it too hard)".

I don't care if all they collect is the bottom right pixel of the image and blur it up before sending it, the sending part is the problem. I don't want anything sent from MY device without my consent, whether it's plaintext or quantum proof.

You're presenting it as if you have to explain elliptic curve cryptography in order to toggle a "show password" dialogue but that's disingenuous framing, all you have to say is "Allow Apple to process your images", simple as that. Otherwise you can argue many things can't possibly be made into options. Should location data always be sent, because satellites are complicated and hard to explain? Should we let them choose whether they can turn wifi on or off, because you have to explain IEEE 802.11 to them?

> I don't want anything sent from MY device without my consent

Then don’t run someone else’s software on your device. It’s not your software, you are merely a licensee. Don’t delude yourself that you are morally entitled to absolute control over it.

The only way to have absolute control over software is with an RMS style obsession with Free software.

They might not be legally entitled to it, but that's just because of our shitty "intellectual property" laws. Morally speaking, OP is absolutely entitled to have a device that they own not spying on them.

Regardless of one's opinion of intellectual property laws, nobody is morally entitled to demand that someone else build the exact oroduct they want. In fact it is immoral to demand that of other people — and you certainly wouldn’t like it if other people could demand that of you.

Want a phone that doesn’t spy on you? Make it yourself. If you can’t, find some like-minded people and incentivise them (with money or otherwise) to make it for you. If they can’t (or won’t) perhaps contemplate the possibility that large capitalist enterprises might be the only practical way to develop some products.

This is just "might makes right" bullshit with slightly prettier framing.

This has absolutely nothing to do with "might makes right". If a fast food store decides to offer a Vietnamese Peanut Burger and Sugar Cane Juice combo, nut allergy suffers are not "morally entitled" to a nut-free option and diabetics are not "morally entitled" to a sugar-free juice option. This applies whether the fast food store is a small family run business, or McDonalds.

To suggest that customers are "morally entitled" to a Samsung phone with zero tracking and zero telemetry is similarly absurd. If you don't like Samsung's product, don't buy it.

> If a fast food store decides to offer a Vietnamese Peanut Burger and Sugar Cane Juice combo, nut allergy suffers are not "morally entitled" to a nut-free option and diabetics are not "morally entitled" to a sugar-free juice option.

Why not? What gives McD the right to make such a decision unilaterally, other than might?

In fact, this is how disability legislation (for example) already tends to work. You don't get to tell disabled people to just go somewhere else, you have to make reasonable accomodations for them.

> What gives McD the right to make such a decision unilaterally

This cannot be a serious question.

> nut allergy suffers are not "morally entitled" to a nut-free option

Restaurant have a legal obligation to warn the customers. AKA "opt-in" which is NOT what Apple is doing. And it's the whole issue with their behavior.

Apple's food scientists have verified the food safety of their new recipe, and they are sufficiently confident that nobody will suffer any allergic reaction. Nobody has disputed their assessment.

That doesn't stop consumers from engaging in Info Wars style paranoia, and grandstanding about the aforementioned paranoia.

That's absurd.

We can regulate these problems.

If the EU can regulate away the lightning connector they can regulate away this kind of stuff.

You're seriously arguing that it's absurd for customers to have "absolute control" over all software?

No EU regulation could regulate away all "moral" concerns over software. More specifically, they EU could regulate, but the overwhelming majority of software companies would either strip significant features out for EU customers, or exit the market altogether.

Lol, they keep threatening that but they still like the money of the europeans.

The EU hasn't threatened granting consumers "absolute control" over all software.

I'd vote for a party that said the only legal license is AGPL :D

The “moral entitlement” has nothing to do with this. The software is legally required to abide by its license agreement (which, by the way, you are supposed to have read, understood, and accepted prior to using said software).

I honestly can’t tell if you’re being sarcastic. A license grants the end user permission to use the software. It is not a series of obligations for how the software operates. This would be excruciatingly obvious if you read any software license.

A license agreement is, well, an agreement between the manufacturer and the consumer which may include a requirement to acknowledge certain aspects of how the software operates (e.g. the user may be required to agree to “share” some data).

Some commercial software licenses may include various disclaimers which exist to ward away litigious assholes. They only serve to protect the vendor against legal complaints, and do not impart responsibilities upon the vendor. Such disclaimers are not necessary but corporate lawyers have a raison d'être, and at a certain scale assholes become inevitable.

Reply on news.ycombinator.com