Comment by willvarfar | Hacker News Reader

Comment by willvarfar | Hacker News Reader

Parent Full thread

I'm not understanding this as homomorphic privacy.

They take pains to say:

> your device downloads the current model, improves it by learning from data on your phone, and then summarizes the changes as a small focused update. Only this update to the model is sent to the cloud, using encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud.

(my emphasis on the word stored)

Now there are lots of scholarly articles on reverse-engineering and rule-extraction from neural nets.

So Google, having the diff can actually get some idea what it is you are trying to teach the net.

They just promise not to.

>They just promise not to.

Google makes the OS and the keyboard. If they wanted to run a keylogger on every device against the express wish of users they could.

So I think the more important question is if someone else could steal or "legally" request that data from Google and recover my keystrokes.

In the secure aggregation paper (linked in the post: http://eprint.iacr.org/2017/281 ) they indeed hint at the possibility of extracting information from the diffs. So the protocol they propose cryptographically ensures that the individual diffs can not be learned. They do this by having each pair of clients generate a secret the size of the diff, and have one add it to their diff and the other subtract it. The clients then send that result to the server where they are summed, which cancels out all the secrets. The bulk of the protocol then deals with setting up the secrets and dealing with client drop outs, which appears to be the real challenge.

Well, that's my fallible summary anyway, go read the paper. :-)

And having worked at Google, I trust them on that.

They say they have a technique called Secure Aggregation as well so that it can only decrypt the update if many users have sent their updates. I don't know how possible that would be to reverse engineer.

Plus this would seem a computationally expensive way of conducting mass surveillance

Reply on news.ycombinator.com