An open letter to Netflix from the authors of the de-anonymization paper
33bits.org
33bits.org
* For the longest time we mostly stuck to doing the math. We certainly didn't call for the contest to be cancelled, and we had nothing to do with the lawsuit. But when some people implied that we were responsible for the mess that ensued, we were kind of pulled into it. We posted this as a way of explaining our point of view and reaching out to see if there's a possibility of collaboration.
* The sadness we expressed is genuine. The reason we brought up Netflix's response to our paper wasn't "snark" or "gloating." Rather, we were pointing out that the cancellation of this contest was rather needless, because if they had acknowledged the privacy risks back when we published the paper, they would have had more than enough time to deploy an opt-in system for this contest. I think it is really unfortunate that that didn't happen.
* Someone wanted to know exactly what I thought of the "greater good" argument. Well, I'll tell ya. I'm vehemently opposed to it and I think it's a dangerous slippery slope. I think this point of view is enshrined in the ethos of this country -- "better let ten guilty men walk free than to convict one innocent man," etc. I don't think anyone has the moral authority to decide that the privacy concerns of a few can be sacrificed.
* There is a specific reason we chose the open letter format rather than communicating with Netflix directly. Acutally, two reasons. First, there are many data privacy researchers who are at least as qualified as we are for this role. We wanted to make sure the community had the opportunity to participate in whatever ensures, rather than just us.
Second, I'm sure there are many companies other than Netflix who have a similar need for privacy preserving data mining. If Netflix doesn't take us up, perhaps one of the others will. Bottom line, since there are multiple parties on both sides, and we don't really know who they are, we felt it is better to have this dialog in public.
* Finally, it is understandable when something like this happens to want to find someone to blame. But think twice before shooting the messenger.
I'll bet money the most likely result of this lawsuits -- and your actions are a big piece of it -- is Netflix, et al, will release no more datasets to the public. Instead, only researchers under NDA will be allowed to work with the data, as was basically the tradition before this.
Good job.
In fact in using this dataset you've kind of done exactly what you set out to stop - used peoples data in ways they haven't consented to.
And get 0.1% participation.
Furthermore, even if the algorithm finds the "wrong"
record, with high probability this record is very similar
to the right record, so it still tells us a lot about the
person we are looking for.
Could someone who shares the author's worldview try to explain it? I've tried, but I just don't see it.
As for that part of the FAQ, it is intended as an explanation of some of the theorems proved in the paper and is a response to some of the theoretical objections we face from the data privacy community. It is not an issue that arises in practice.
Instead, you brushed off our claims, calling them “absolutely without merit,” among other things.
Now even if you are all emotionless, 100% objective researchers with no interest other than the greater good, this very sentence will make it impossible for humans to read your letter without implying a certain level of snark.
If you want to project pure motives for the letter then it would have been best to leave their reaction to your original research out of it, or, probably more appropriately, don't publish an open letter at all—contact them directly.
Analysis of motives is of course valuable. But it's not an argument against the matter at hand. It may not have been clear that I was being facetious.
I don't think anyone is refusing to accept anything he says. They're just disapproving of his motives, which is fine.
Still, if you can overlook my misreading, I would love your response. I understand that the example from the FAQ is not central to your argument, but I do think it's important in understanding your worldview. If I provide a tool that can be used, erroneously or not, to create or enable prejudice, at what point have I crossed the line into 'violation of privacy'?
That FAQ question is a big red herring. Some of the objections to our paper during peer-review were along the lines of "if Netflix had esentially duplicated every record in the database, how could you be sure you found the right record? What does right record even mean?" No really, it was that silly. So it was meant to be a way to justify the fact that de-anonymization can happen even if you didn't find "the right record."
As for the possibility of netflix running a contest like this in an online fashion, well, maybe, but the benefits of having access to the data are enormous, plus you've now moved to a model where only the privileged few are allowed access via NDA, or Netflix has to provide computing resources to all researchers, etc. I don't see it happening.
- Do you view this as a breach of privacy?
- What do you consider private?
- Do you view this as a breach of privacy, but just
don't care?
- Do you feel that the utility of the data out-weighs
the privacy concerns?
- What about the people that view this as an invasion
of privacy and have their Netflix user data in that
set? Should they be thrown under the bus in the pursuit
of progress because *you* feel that the data has more
utility than the privacy concerns do?
Also, you're a retard for comparing movie predictions, and the possibility of matching a person to their movie viewing history, as even remotely comparable to human test subjects. Please.
As for invasion of privacy, I'm unsure -- I'm not sure of the probability of matching, the quantity of information necessary to get a good, for various values of good, match, etc. What is clear is the authors had a major hand in stifling a nontrivial nonacademic dataset and damaging the community around it. They further have aided the lawyers suing netflix, and have helped poison the well for any company in the future that decides they might want to do something like this. So I say congratulations! For the author to pretend this didn't happen as a result of his actions is disingenuous.
As for your questions, well, they're just stupid. We live in a world where the fbi/police get access to your PHYSICAL LOCATION 24x7 without a warrant just by asking, where your emails and telephone calls are scanned by the nsa with plans to open this data set to the police at large, where google/yahoo/et al see turning your emails and access patterns over to the police as a revenue opportunity, etc. If you care about privacy, this is such small potatoes as to be a waste of time. BTW, anyone can still spend roughly $100 to access your phone call history. G has, in subpoenable form, your entire search history -- and don't think that clearing cookies prevents stapling that history together.
So you're saying that since government agencies have access to a lot of my private information, I shouldn't care about any of my private information remaining private? Sounds like you're creating a false dichotomy. You're presenting things as if you can only care about all of your private data or none of your private data; since the government has access to large portions of your private data and you don't have much (or any) control over that, you should therefore care about none of your private data. Isn't it possible for me to care about all of my private data, but to choose the battles that I fight?
In other words, "we showed them!" -- seem an awful lot like gloating to me.
It also doesn't appear that you ever seriously considered the "greater good" argument, beyond asserting that it's "dangerous." This leaves me with the suspicion that you chose to take the easy way out instead of weighting the actual costs and benefits of what you were doing.
You're shooting the messenger here. If a dataset can be de-anonymized, it's better to know that and be able to make informed decisions.
Well, I can give you my worldview. Did you see the article "70% of HR managers turn down job candidate based on online reputation?" Here's the HN discussion: http://news.ycombinator.com/item?id=1192996
So. I rent a bunch of movies with strongly opinionated themes. I dunno, maybe every Michael Moore movie plus travel documentaries about Cuba. I also rent the entire Star Wars sextilogy and The Godfather Trilogy, which I review on IMDB.com. Some enterprising hacker ties my reviews on IMDB to my anonymized rental record. How many 47 year-old males are there in my neighborhood who watched those exact movies?
The hacker publishes his "findings," of course. Now I go looking for a job and someone in HR decides that my political views are too risky, so I don't even get an interview.
The author's views are that Netflix can provide the benefits you desire without compromising the privacy I desire. I think the debate should be around whether the authors are correct.
If the authors are correct, then the problem isn't the authors preventing the release, the problem is Netflix failing to learn from their prior mistake.
If the authors are incorrect, we should simply point that out.
That kind of data should not be released without consent.
Now I go looking for a job and someone in HR decides that
my political views are too risky, so I don't even get an
interview.
The author's views are that Netflix can provide the
benefits you desire without compromising the privacy I
desire. I think the debate should be around whether the
authors are correct.
I also think that for my needs 'anonymize and release' is the only feasible solution. While online systems might be better than nothing, the approaches I'm interested in (kNN on GPU's, massively parallel cross-validations) really require local data and full control over data layout. But perhaps there is indeed a solution that works for everyone.
>> It's not ideal, but basing hiring decisions on real information (even if 'private') strikes me as better than using uninformed prejudice. In the example you give, is it better or worse to have the HR person say "Homoiconic? Sounds gay to me!"? I don't see much increased risk in offering a deeper pool of inherently unverifiable information, even if this information might be used badly.
Paraphrased: "They have poor information sources for their prejudice now, I don't see what harm better information sources driving their prejudice could do."
>> This is a good debate, but for me it's a secondary one. Even if a privacy-safe solution is theoretically possible, I fear the only likely outcome will be that Netflix buries a 'release authorization' somewhere deep within their terms of service and is dissuaded from offering similar data sets in the future.
Paraphrased: "The debate on privacy dangers is secondary, if Netflix refuses to release useful data in the future then that's the primary danger"
>> I also think that for my needs 'anonymize and release' is the only feasible solution. While online systems might be better than nothing, the approaches I'm interested in (kNN on GPU's, massively parallel cross-validations) really require local data and full control over data layout. But perhaps there is indeed a solution that works for everyone.
Paraphrased: "I was working on a project with the data, and the FTC ruling appears to have put a stop to that."
They have poor information sources for their prejudice
now
I don't see what harm better information sources
driving their prejudice could do.
The debate on privacy dangers is secondary, if Netflix
refuses to release useful data in the future then that's
the primary danger
I was working on a project with the data, and the FTC
ruling appears to have put a stop to that.
I feel strongly about the primacy of privacy because the problems of prejudice are so severe. But the transparent world approach has a lot of depth to it, and I'm nervous and excited to see where it leads us, since like it or not it the technical and social changes we're seeing are creating so much data.
Here are the issues I think we have with this particular data release. I understand they may be resolvable though, and while I argue against accepting privacy problems like this right now, I would definitely love to find out that we can shift to a more transparency society without reservation and without the level of abuse that I fear:
-- Information asymmetry, since most people don't typically keep the type of statistical skills, computing power, and even secondary data sets for cross joins and comparisons around. Information asymmetry is a widely recognized concern in social behavior and commercial law (i.e. insider trading & real estate contracts)
-- Ethical concerns seem to constrain us to taking people's own privacy expectations into account. Originally privacy loss didn't seem to be a problem, but the situation changed when evidence emerged to the contrary
-- History shows us that governments frequently overstep their bounds. Rather than argue recent events which are more clouded by emotion and the narrative of the day, I think most people would agree that the FBI in the 60s was quite abusive in their gathering and use of data
That said, transparent world all the way. How do we get there?
"I'm going to just rewrite your argument" is not a valid form of debate. Please don't do that, ever.
When you're on the Internet it's hard to know where people are coming from. I can see now how I misread him, I wish I didn't, but I did. How else do I find out politely what the terms of a debate are in a public forum? Is it truly better to not attempt at true comprehension of a person's words? Is it all just throw-away, a comment on an aggregation site?
And this was a tough thing to ask politely, simply because I had a presumption which was wrong. But I tried to do it right. I think the response I got back was pretty clear, the OP knows the fundamentals of the privacy debate, he/she has an opinion on the privacy issue itself, and the OP is not too offended by my best take on a gentle nudge at the bias question, a pretty safe guess from that is that he understands the questions of bias as a safety net for intellectual thought rather than as an offensive gesture.
I didn't know all of that, and so I asked. That's all it takes.
It's hard to communicate well online. It's this new thing for the human mind, without a 100,000 years of conditioning. Every day I log on to Hacker News for two things: It's shockingly educational in many disciplines; and I want to learn to communicate so I can better participate in society in a positive way. What a challenging place to do it. In the process I've found out that my communication needs a lot of work. I've been lurking since '88 and lurking didn't teach me communicating, what gives ;-).
But I'm pretty sure I took a reasonable tack here. Listening comprehension, interest in knowing what people really mean in short form communication, and trying to find out why people say things that we disagree with seem to be critical needs in this complex medium.
When you rewrote the argument as "I'm going to just rewrite your argument", you ignored the same principle that you were proposing.
Apologies, if indeed it was meant to be a joke.
I'd encourage people that usually skim just the comments to read the post, which was not gloating at locking up the dataset. The thrust of the post is about what a shame it is that the contest was canceled and how to make sure future contests can work.
[Note one: I have nothing to do with either side. Note two: I guess there is a gloating interpretation, with the paraphrase "you ignored us, but the FTC said we were right, nah nah nah" — but this isn't a useful or constructive way to continue the conversation.]
That how I read their open letter. And my interpretation of "you should have worked with us" is as "you should have hired us as consultants"
Yes. The difference between 'John Elks of 7 Arborview is a rapist' and 'there is a 0.5 correlation between living in Pleasantville and being a rapist' or '38 year old males commit 10% more rapes than average' etc. is solely one of degree and not kind.
Suppose I have a set of datapoints like that. And let's say each datapoint applies to only half the population. How many datapoints before I have broken your privacy and linked you to the furry porn you like to rent? Well, I'm guessing you're an American male. The US population is ~300 million, and roughly half of that is male, so 150 million. The first datapoint pins you down to within 75 million. The second, down to 33 million. The third, down to 16 million, the fourth, 8 million, the fifth 4 million, the sixth 2 million, the seventh 1 million, the eighth 500,000 (starting to feel nervous yet?), the ninth 250,000, the tenth 125,000, the eleventh 75,000, the twelfth 30k, etc. until the 25th or 26th specifies just 1 - you.
Now, tell me: Where in this slippery slope did it suddenly flip from not being a privacy violation of some degree, to being a privacy violation?
Was it at the 5th bit of information? Are you damaged at the 12th bit of information? Or did it take until the 24th or 25th bit of information before it magically flips from being good science to bad privacy violation?
Is it fine just so long as it might also be your neighbor down the street, even though most people would shun you based on far less than a 50-50 chance of things like being a child rapist? (An employer on the bench might regard a 10% chance of you being objectionable as being too much; that only requires, what, 18 bits of information?)
Predictions embody a great deal of information. That's how Bayesian statistics and statistics in general work, after all.
And according to another article by them, all you need is zip code, gender and birthday to identify someone with a high degree of certainty.
http://www.eff.org/deeplinks/2009/09/what-information-person...
'The first piece cuts it down to 2.3 million people...'
It's not surprising that those two pieces of information can pretty easily narrow down an identity.
log2(3*10^8) = 28.2It comes back to the contemporary problem of statistics: We, programmers included as evidenced by many discussions of the sort, have a hard time getting it. We are only at the cusp of understanding what statistics can do with large data sets.
Now, we can either work with researchers and organizations to deal with the very real concerns, or we can simply refuse to believe that giraffes don't exist because we never saw an animal with such a long neck before. It's the basic problem of progress.
Essentially they are saying that if you know what rating someone gave 8 movies and the date that they gave those ratings, you can find a sample of their rating list (or something very similar to it) with 99% accuracy. So freaking what?
"Evidence" that flimsy wouldn't stand up in a local bar argument, much less a court of law. The records are still completely and totally anonymous. No names, no addresses, no way to identify anyone...nothing but a strong statistical correlation to a set of ratings in a database.
It sounds like they have a problem with the power of predictive modeling and not with the handling of anonymous data. Esstentially what they are saying is "if we know a little about you, we find can out things that we didn't know with a very high degree of accuracy, but no certainty.". Duh. That's what the whole netflix prize was about...using known data to make strong predictions about unknowns.
They had some interesting methods (especially in their similarity calculations) but this has nothing to do with privacy.
I am completely unsatisfied with the submit-and-run model. Feature engineering seems to rely on knowing your data really intimately, and that does not seem possible in a submit-and-run model.
Perhaps this is a partial solution for the Netflix dataset.
http://freakonomics.blogs.nytimes.com/2010/02/02/can-you-tru...
(off topic rant, I know. goodbye karma!)