Studying how Firefox can collect additional data in a privacy-preserving way

Tracking down regressions, crashes, and perf issues without good telemetry about how often it's happening and in what context. Issues that might have otherwise taken a few days to resolve with good info, become multi-week efforts at reproduction-of-the-issue with little information.

It simply boils down to the fact that we can't build a better browser without good information on how it's behaving in the wild.

That's the pain point anyway. Mozilla's general mission, however, makes it very difficult to collect detailed data - user privacy is paramount. So we have two major issues that conflict: the need to get better information about how the product is serving users, and the need for users to be secure in their browsing habits.

We also know from history that benevolent intent is not that significant. Organizations change, and intents change, and data that's collected now with good intent can be used with bad intent in the future. So we need to be careful about whatever compromise we choose, to ensure that a change of intent in the future doesn't compromise our original guarantees to the user.

This is a proposed compromise that is being floated. Don't collect URLs, but only top-level+1 domains (e.g. images.google.com), and associate information with that. That lets us know broadly what sites we are seeing problems on, hopefully without compromising the user's privacy too much. Also, the information associated with the site is performance data: the time spent by the longest garbage-collection, paint janks.

This is a difficult compromise to make, which is why I assume it took so long for Mozilla to come around to proposing this. These public outreaches are almost always the last stage of a length internal discussion on whether proposals fit within our mission or not.

I'm not directly involved in this proposal, but I personally think it's necessary, and strikes a reasonable balance between the privacy-for-users and actionable-information-for-developers requirements.

Also interesting: the method they plan on using for anonymising this: https://en.wikipedia.org/wiki/Differential_privacy#Principle...

If that is not sufficiently anonymous, then please submit the reasoning why to Mozilla.

1. Run an opt-out SHIELD study to answer the question: "how many people can find an 'opt-out' button?". That's all. You launch this at people with as much notice as you would plan on doing for RAPPOR, and see if you get a 100% response rate. If you do not, then 100% - whatever you get are going to be collateral damage should you launch DP as opt-out, and you need to own up to saying "well !@#$ them".

2. Implement RAPPOR and then do it OPT-IN. Run three levels of telemetry: (i) default: none, (ii) opt-in: RAPPOR, (iii) opt-in: full reports. Make people want to contribute, rather than trying to yank what they (quite clearly) feel is theirs to keep. Explain how their contribution helps, and that opting-in could be a great non-financial way to contribute. If you give a shit about privacy, work the carrot rather than the stick.

3. Name some technical experts you have consulted. Like, on anything about DP. The tweet stream your intern sent out had several historical and technical errors, and it would scare the shit out of me if they were the one doing this.

4. Name the lifetime epsilon you are considering. If it is 0.1, put in plain language that failing to opt out could disadvantage anyone by 10% on any future transaction in their life.

I think the better experiment that is going on here is the trial run of "we would like to take advantage of privacy tech, but we don't know how". I think there are a lot of people who might like to help you on that (not me), and I hope you have learned about how to do it better.

If they start opt-out tracking using the same approach as Google I do not see any reason to use it nor install it for my friends and family. That's some data for you, Mozilla.

  For Firefox we want to better understand how people use our 
  product to improve their experience. 

No phoning home. No telemetry, no data collection. No "light" version of the same, no "privacy-respecting" what-have-you. No means No. Nada. Zilch. Try and shovel any of that down people's throats and the idea of Firefox as a user's browser will die.

IMHO, this is a bad idea. Many people I know already use Firefox because they're weary to give Google (Chrome) all their data.

Firefox should make this feature opt-in only.

I think not having perfect information about the users is a trade off that should be made in order stay an alternative to most other browsers. There are still ways to get more data by other means, though. When it comes to most visited websites, for instance, the alexa ranking should give a good, if not perfect, idea.

Data is both highly alluring and addictive as evinced here by Mozilla potentially willing to shoot itself in the foot to get some. What's to keep this from becoming a frog in a boiling water kind of situation? How can I trust that Mozilla is going to adhere to their own stated standards? The easiest answer is that I won't have to because I can just use something else. Personally, the only reason I use Firefox is because it's slightly less convenient to set up a secruity-patched version of Chromium.

Other people in this thread have made the excellent points of the fact that not enough people opting in to data collection is in itself a critical piece of data. Moreover, things such as "Which top sites are users visiting?" can be answered by looking at data from page ranking services and then they can go to those sites on their own testing equipment to answer their other questions. A little investment in acquiring this data by not spying and maybe getting a wider array of testing equipment is probably less costly than the potential for loss in market share that they're already struggling to hold.

So we will toothlessly complain but then the changes will be shoved in our throats, because obviously why would one care what the non-targeted demographics whines about. And of course it will be framed as being 'for our own good' and half of the people complaining with just deal with it, just like the majority already does.

How does seeing which sites users use that need Flash drive their decision-making. Either they support Flash, or they don't.

And- ditto for "Jank" (not sure I understand that term, frankly- why is it capitalized?). Some developers don't optimize well- how is Mozilla going to use this? I think they do a good job over on MDN.

I guess I'd like to be sure I understand what problem they are trying to solve. Maybe they feel like without understanding their users they can't keep up with Chrome. I see people talking about how good Chrome is. And I must admit- it is sweet for me too. But that may be because (1) I don't have it loaded up with add-ons like I do Mozilla and (2) they have optimized for certain sites like youtube and gmail and I just can't get Firefox to work all that well on those sites.

But I'm not convinced that they need my data to fix that.

EDIT: On the other hand, Chrome seems to lose my passwords on every upgrade so it won't be my main browser until if fixes that little issue, which is going on, what, 5 years now?

I am starting to think that they just don't want people to use Firefox.

Yeah, I know it's free software, so I have no right to complain. I just wonder why?

In essence, Firefox will ask itself whether it visited website X and flip a coin and if it's heads, it will lie to the server and send a random boolean. If it's tail, it will not. This way there is no way for anyone (including Mozilla) to know whether you actually visited the website. But the statistics will work out such that the collective data from everyone will give a good representation of all users. I find this a neat technology to collect data in a privacy-preserving way. And there's an opt-out (opt-in won't work because it creates bias and provides messy results).

I really, honestly don't understand why people are so upset.

People are insulting the developers, saying Chinese owned, VPN-operating Opera would be better for privacy... there is a lot of nonsense here.

IMO this is not the most needed feature, and I would be happy for Firefox to keep in mind its reputation as a product focused on user privacy.

2. Frankly even opt-out is not acceptable. I can't recommend any software that peridically asks users for data access, since there exist non-technical users who have a nonzero chance of clicking yes to everything. If they are related to me in some way this compromises my privacy also.

It seems they've convinced themselves that the only way to improve the product is to collect data on their users, rather than continuing to push the idea of privacy - which, in my opinion, if marketed correctly, could win over a lot of users. The browser is still fundamentally awesome.

This seems like the kind of thing they could push through their TestPilot program and just market it, rather than pushing it to everyone by default. But I imagine they want to push it to everyone specifically so they can take advantage of those who are ignorant to the ability to opt-out.

Otherwise I might as well just use Chrome. Hopefully some PR guy will pour some water on this before it turns into a dumpster fire.

I would say that is none of the browser vendors business.

Please stay away with your opt-out stuff - it bothers me. Make it opt-in, always and forever.

First it was killing customization.

Now they are killing Privacy.

Why should I use this browser again?

Who runs Mozilla, do they understand why anyone would choose Firefox over Chrome?

Maybe it's time to put a spotlight on the management and decision making structures of increasingly important open source projects like Firefox to ensure they are being run in the public interest.

Could someone explains to me how this information is useful to a browser vendor? It's not as if they are optimizing on a site by site basis.

How many people here use website/app analytics to improve products they work on?

By now people should be aware that it is not just the content that is important, but also the metadata. A browser that phones home with information on users' browsing habits is not acceptable to many of us, who will move to forks or a different browser altogether. This from one of the people who "doesn't complain, but just never goes back."

For those interested in understand more about this project and why we're doing it, here you can find an introduction of Differential Privacy and what we're trying to do. https://twitter.com/Alexrs95/status/896366072240144385

Meeting the standard of true differential privacy is one of the strongest known unconditional privacy guarantees. It will prevent Mozilla from being able to answer any user specific questions. For example, they might have an accurate count of how many people visit Google.com (say 60% of their user base), but they will be mathematically unable to point to exactly which 60% visited the site.

Differential privacy in the RAPPOR implementation is peer reviewed and well understood. We can also review the actual code that ships in Firefox, which is a big plus over the Chrome implementation. There are some caveats -- what epsilon are they setting, are they adding an appropriate amount of noise, how do they protect against repeated queries, etc. but all of these can and will be reviewed by the differential privacy community.

I am not affiliated with Mozilla or Google, though I do work in the field of differential privacy. On mobile now, but I am happy to provide links or answer questions to people who might have any when I am back at a laptop.

I design optimization algorithms and software professionally and the majority of that software is released open source. Now, does my software likely run terribly on some problems that my users give it? Absolutely. That probably costs me business because they get frustrated, give up, and go somewhere else. And, to combat that, I could absolutely engineering my libraries to send anonymized information about their problem structure back to my company. Certainly, it would help me improve my software and algorithms. I also view it as horribly unethical, a breach of my customers trust, and an unacceptable course of action. Look, I want my software to work well for everyone, but it's part of my job to figure out when things don't well and fix that beyond scraping information about my customers uses automatically.

I contend this is a terrible idea and very much would like Mozilla to abandon it.

If it is opt-out, then Mozilla would have to be extremely open about how to opt-out and exactly what is tracked.

The real issue here is with ethos and perspective. I use Firefox because the ethos of the company and its employees, and their general "take" on issues like this allows (or has allowed) me a general sense of trust in them.

Even the very existence of this discussion erodes that trust. This says to me "the people making this browser don't understand the importance of consent, and have a vastly different perspective on the value of privacy to mine".

If your developers need more data from Telemetry, get consent and collect more data. Establish trust in users in what you do with that data.

I really don't understand this of the user, while previous sentence he writes:

"but I will say that I believe Opt-in is pro-privacy, while Opt-out is anti-privacy."

FHR is also opt-out, i.e., enabled by default. If you do not like this, you may want to disable this as well.

The real problem seems to just be marketing though. Regular people either don't see any reason to consider looking for an alternative browser or don't understand the differences. Years ago Firefox had a larger market share because the internet as a whole had a larger share of tech-savvy people and they had IE as a competitor.

Greed and monetization of user's data - this is the only real business model which brings profit. There is, probably, already a long queue of customers willing to pay for the data.

And, of course, "anonymity" is nonsense. The whole point of collecting data is to classify users into target groups and model user's behavior. In other words - to collect the data for machine learning algorithms and sell access to the datasets and other services.

With the 300+ upvotes and active discussion still occurring at hour 7 since posting, care to explain how the algorithm has relegated this discussion to the fourth page and is still dropping? Is the much less active, day old posting of a SF author's death more heavily weighted than an inconvenient topic that is important to several more factors of readers?

One of the last truly shiny examples of open source is losing the plot. Not only that it requires pulseaudio (alsa?), it is getting harder and harder to use it normally with FreeBSD. Now this.

I've had enough, testing links -g and it works well for most of my browsing needs.

In theory can be done, but in practice they are competing with Chrome and their team has waaay more data to use. And th is data gives them an edge at least on the decision which parts are worth improving.

So they can either start collecting some data and really piss off their most vocal privacy minded users and try to use this data to improve FF and steer it away from the death spiral it's on. Or they can keep the vocal privacy minded people happy continue to work in the dark and pretty much ensure that FF will become one of the insignificant .3% market share browsers.

Because somehow I kind of feel that multimilion fundraisers to make FF popular again aren't gonna happen second time.

> They don't plan on collecting URLs, just (eTLD+1).

This is true as of right now, but can change at any time in the future. From the post:

> What we plan to do now is run an opt-out SHIELD study [6] to validate our implementation of RAPPOR. This study will collect the value for users’ home page (eTLD+1) for a randomly selected group of our release population

This test consists of collecting domains, indeed, but that doesn't say anything about what will happen in the future.

> Note: "planning" means "reaching out for feedback about".

Planning means planning. Today they're reaching for feedback, and the plans might change or not.

> Hello, Redditors...

This is my fault, I suppose, for posting the link here :). Many of the angry comments are uninformed, but the users, educated or not, are stakeholders here and Mozilla should be prepared for the fallout. There have been situations in the past (Pocket, Google Analytics) where well-formulated feedback from users was raedily dismissed.

> One recurring ask from the Firefox product teams is the ability to collect more sensitive data, like top sites users visit and how features perform on specific sites. Currently we can collect this data when the user opts in [...].

Does anyone know what this is about? Telemetry? Because I will disable it if so.

> Allow Firefox to install and run studies

This is from the Nightly settings page but is pointing to https://support.mozilla.org/en-US/kb/shield, which doesn't exist (yet?). For anyone interested, there's a wiki page about them https://wiki.mozilla.org/Firefox/Shield/Shield_Studies.

> What we plan to do now is run an opt-out SHIELD study [6] to validate our implementation of RAPPOR.

This still sounds bad enough to forever poison "SHIELD" for me. It's also terribly named because it doesn't "protect" anyone.

> No telemetry, no data collection.

Without telemetry it would be almost impossible for the developers to figure out what works or not, and what's fast or not in Firefox. There's a whole spectrum here from "no telemetry" to "creepy". Please don't ignore this.

> Now they are killing Privacy.

Please try to get informed. A Mozilla employee in this thread (alexrs95) posted a series of tweets about what's being proposed: https://twitter.com/Alexrs95/status/896366072240144385. It's short enough, so please read at least that before complaining.

> What's your favorite open-source browser?

Firefox :).

> I've removed all URLs from about:config and replaced them with localhost (search for "http"). This should help with privacy-related issues as long as no API endpoint is hardcoded.

Beware of SHIELD, as Mozilla may still have the ability to push extensions to the browser.

> He said there are math theorem to prove that it's sufficiently anonymize.

I've not dug deep enough into the RAPPOR paper, but they do consider in passing the possibility of an attacker that has access to all of the collected data (think https://en.wikipedia.org/wiki/National_security_letter).

> Everyone else

Please be kind.

EDIT: Looks like this post might have been pushed back from the front page by a moderator. I'm not sure I'm fine with that.

It's like describing to a spouse a system of sex with strangers that includes blindfolds and a hazmat suit. Such a system could be a great way for a person to learn more about their sexual tastes and improve coitus overall. If the spouse is anxious about the system, then all they have to do is find a problem with the hazmat suit that would endanger the...

Wait, spouse, you haven't even studied the system that I so carefully designed to protect you from the possibility of...

Spouse?

Has anyone seen my spouse?

> DRM

EME is not DRM, it's a fully open source spec to support third-party DRM modules. If you don't actively choose to install a DRM module, there is no DRM in your Firefox.

> 3-rd party apps

Like what? Pocket is fully owned by Mozilla.

> analytics, tracking

So far this has been 100% opt-in. It might change with this new thing, but even that's not for certain.

    firefox.exe	3588	TCP	pc-name	49172	ec2-35-167-184-4.us-west-2.compute.amazonaws.com	https	ESTABLISHED	3	667	5	3,334