Hacker News Reader

Comment by Nullabillity | Hacker News Reader

Yes, running experiments on unknowning and unconsenting users is unethical, especially when the metrics being optimized for usually aren't in the users' best interests. The widespread acceptance for this behaviour could definitely be classified as brain damage.

View on HN

> the metrics being optimized for usually aren't in the users' best interests

Do you have any source for that? If you truly believe that Firefox is optimizing metrics that aren't in the users' best interest in general, why are you even using said software in the first place?

What other browser can you use if you are interested in some sort of privacy?

Is Chrome compiled from source safe from tracking?

>What other browser can you use if you are interested in some sort of privacy?

Waterfox, Safari. Some others too but they're a pain to use (ungoogled-chromium for example). Chrome itself has a bunch of binary blobs so I would stay away from that.

I think this can be compared to McDonald's slightly altering the recipe of something for part of the country. I don't find that unethical at all, and frankly the things we put in our body are a lot more personal than which web browser we use.

Yes, but it isn't (well should not be if they want to stay within the law) adding completely random drugs an garbage to the food. It may be seeking cheaper recipes with no sales losses, better recipes to improve sales, or something else, but it all has to be within the context of proper regulation, i.e.,, FDA. (we can argue elsewhere about the effectiveness of that system).

This unlimited testing can easily be seeking to better maximize psychological exploits with no regulation.

So, yes, I'm happy to see someone putting in a bit of control.

That is dismissive of how much value people place on their selves and digital aspects thereof. My identity, private conversations, and sexuality are infinitely more personal than the food I consume.

I agree that it’s appropriate to A/B test changes, within reasonable bounds. I wish the debate would focus more on what the reasonable bounds are to each objector.

> especially when the metrics being optimized for usually aren't in the users' best interests

I don't think so. Mozilla is not google, and apart from the mr. Robot error, that they apologized for, I am sure Mozilla is using it for making the browser better, and not to use you for targeted ads like the other browser maker does.

When has Chrome used A/B testing for ad targeting in the past? Can you link to an example?

Google Chrome uses the RLZ tracking identifier for some installations.

"RLZ gives us the ability to accurately measure the success of marketing promotions and distribution partnerships in order to meet our contractual and financial obligations."

https://blog.chromium.org/2010/06/in-open-for-rlz.html

While not exactly A/B testing as discussed above, that's still a relevant link. Thank you for sharing.

No problem. There's more information in the Google Chrome Privacy Whitepaper:

https://www.google.com/intl/en/chrome/privacy/whitepaper.htm...

So when deploying changes it should just go to every single user with no incremental rollout?

Yes. The testing should be done on users that opt in (likely less than 0.1% but that should be OK if it's done properly) and it should be rolled out by releasing an update to everyone.

These so-called "best practices" often aren't.

By relying on opt-in you won't test whether a feature actually works for the average user, so if you don't test it, you might roll something useless/confusing/actively harmful out to all of them instead of just a few.

Exactly. The recent Windows 10 update fiascos demonstrate this perfectly: they are opt-in (Insider program) and there were too few users experiencing the bugs to overcome the noise of those who weren't. You need a truly random sample.

I think they updated too many parts of Windows at once, not too many computers at once. And I'm not certain that a delayed rollout would have fixed it. Is it fair that a random sampling of people are given the problem?

So instead of having enough voluntary or paid testers to detect catastrophic, file deleting bugs in a new version, your solution is to just roll out the new changes to random users expecting a stable system?

Something went terrible wrong somewhere, that we got to this.

And that's OK. Being honest to your users is much more important that having the most efficient deploy system.

I, for one, as a matter of principle, opt-in to all user tracking systems that are disabled by default, and try to opt-out whenever they are enabled by default. In the case of firefox, it was never clear to me that the tracking/remote install capability was enabled by default.

I don't see the purpose. There seems to be an assumption that A/B testing is inherently bad, therefor should be opt-in.

A/B testing is necessary for stable rollouts, that much seems self evident.

So what is the evidence that A/B testing is harmful to users?

It isn't A/B testing that is bad, it is running and/or modifying stuff in the user's computer without their knowledge that is bad. If a site modifies some part of their layout or whatever isn't bad, but a browser installing addons or changing settings to see if it will crash is bad.

Well, it's not like they're hoping it will crash. Presumably they're hoping it won't.

Wouldn't it be more irresponsible to deploy to every user without first testing it out?

that's what the stable/experimental channels are for - a user who don't want to be tested on (such as a business user) can easily be sure that they are getting a stable release and their settings won't update under them.

Experimental/beta channels can have these changes pushed to it, since that's implied by the name, and those people who like living on the edge of new features do so knowingly.

Let the users control their destiny, let the users decide when. That's the hallmark of a trustworthy company.

Users who intentionally run beta / nightly are not normal users. How many of them run on cheap "desktop" computers they bought on sale at Target, do you think?

Wait, so running and modifying stuff on all users' computers is good, but a random sample isn't? That's what A/B testing deals with.

The discussion isn't "modify stuff" vs. "don't modify stuff". The discussion is "some users" vs. "all users".

And obviously if you have an auto-update mechanism you're modifying stuff on the user's computer. Chrome updates are near invisible on modern machines. That's a good thing. It keeps the browser evergreen and protects users. If everything were opt-in because "Google shouldn't modify stuff on my computer" that would be an absolute disaster for everyone. No way.

This is a technicality, users do not care about technicalities, they have their own expectations and if you are on -say- a stable branch with autoupdates enabled your expectation is that you'll get the latest version that everyone else is using.

A/B testing through that branch breaks that expectation (and no, mentioning it somewhere hidden in some EULA or equally obfuscated place does not mean people will not have the expectation - hell, even mentioning it right below the download wont make a difference).

Isn't that the entire point of the nightly and beta branches?

Those populations are not representative of the release population.

Reply on news.ycombinator.com