Hacker News Reader

Comment by notRobot | Hacker News Reader

Also posted this as a child comment elsewhere in the thread, but:

---

I don't know about Ubuntu, but while installing Debian the installer makes it very clear what popularity-contest does and asks the user for explicit consent.

It also makes it very clear that this setting can be changed later at any time.

The default option is "I do not wish to participate", so if someone isn't paying attention during the install process and just hits 'Enter', their privacy remains intact.

This, IMO, is the perfect way to do telemetry.

View on HN

> This, IMO, is the perfect way to do telemetry.

I understand @notRobot's sentiment, but do not agree with it.

If the default is "no telemetry", then very little telemetry will be collected.

This is an advantage to the user, but a strong DISadvantage to the collective, because the distro manager now cannot make informed decisions as to what packages are installed where and how often.

It really is a balance of "individuals vs the collective" and I don't think that answering both needs is simple, nor is it cut-and-dried.

Why would a collective like Debian or Ubuntu need to know what packages are the most installed? I know Ubuntu has their own App Store, is it perhaps to show which apps are the most popular?

If it’s for caching, can they not see which packages are most frequently accessed over http and implement caching that way? Or is that against their privacy policy? I’m not very knowledgeable on this subject but would love to know more!

It helps deciding on priority (bug fixes) or whether a package is worth maintaining or keeping in the distribution even if it costs time or prevent some (library) upgrades, improvements or refactorings that would break it.

As a software writer, it motivates me to know that my software is used and that putting effort in maintaining it will help people. For instance, I made a small Firefox extension that requires regular updates by design. Seeing that it was installed on a hundred browsers has motivated me to maintain it. If it was just for me, I probably wouldn't have bothered so much even though this extension scratched a personal itch.

As a user, I am a bit reluctant to being tracked so a balance must be found.

Edit: the amount of downloads from the archives is not enough. Some packages are downloaded again and again by automated systems like continuous integration systems and a downloaded package can be uninstalled immediately and this would be imperfectly detected.

Perhaps when making the choice you could tell the user their problems will get more priority if they opt in.

Also using differential privacy you can gather statistics on package use while limiting knowledge about a specific user.

They used to put the most popular 650mb of packages on a CD.

But the maintainer can still make decisions based on the telemetry which is available. It's not unreasonable to assume that the the smaller sample of uses will still be a representative sample.

Yes it is. The average user will change very few default settings, so people who opt in will have fundamentally different usage patterns because they fiddle with stuff.

Opt-out telemetry does not create informed decisions, it creates decisions based on the subset of people who don't care or don't know enough to opt out, and I have seen real instances it adds a dangerous overconfidence to the applicability of the decisions made from that data. And it's important to note that little telemetry is often enough to do a decent amount of statistical analysis–in fact, one of the first things you'd learn in a course is that you don't really need to collect data from everyone, and most sample sizes don't go above 10% of the population unless you're doing some kind of census.

"A little telemetry is often enough" -- Assuming a representative sample, right? If only a few opt in, they might climb above 10% but may not represent the population distribution.

(I'm agreeing with your first statement, but questioning your second one.)

Yes, just saying that in general if it is possible to have a representative sample (which I argue is not a thing even when you are doing opt-out, and would be willing to content is probably even less representative).

Do you really need samples from that many users? Wouldn't a sampling of a few thousand users give you more or less the same results as a sampling from a few million? And if not, is it worth the bad will you're earning from your users by treating them like this?

For websites and online businesses yes you need as much data as possible. For free software I agree, I don't see a use case

A random sample – sure, if you have enough traffic to still get decent sample size, but a sample obtained by users opting in is often close to worthless because it's not random and so not representative of your userbase.

I don't think people who opt-in for such a telemetry is as equally distributed as totally random people who didn't opt-out.

The average users that you want to target don't really change those settings.

We really need a DoNotTrack-like standard for application telemetry so that sysadmins and the privacy conscious can just set some environment variable like NO_TELEMETRY to a nonempty string and not worry about it. Packages would still have to be audited and shamed for not respecting NO_TELEMETRY but thats no worse an obstacle that has faced homedirs.

Developers can get that sweet sweet telemetry by default and those that care can just set a var in ~/.profile and be done with it.

>We really need a DoNotTrack-like

Ahh...that thing that is mostly ignored because its based on goodwill?

"Mostly ignored" is [DIVIDE BY ZERO TRAP] times better than "does not exist." Would you prefer nothing at all?

That's an interesting environment variable... Wonder if laws like GDPR would get that deep enough to have effect on command line users too.

Is it such an advantage?

> Canonical’s Michael Hudson Doyle says “…the package and backend have both been broken since 18.04 LTS without being much missed.”

If you ask politely, I will often say yes to opting in! People don’t seem to understand this and think that somehow I’ll feel exactly the same if it’s opt-out because “it’s the same data…”

Same here! I usually opt in to popularity-contest on all Debian installs of mine that are on PCs, though I leave it turned off on servers.

I have it turned off on a laptop I use for red teaming because you don't want to be accidentally noisy on the network, but other than that it's turned on.

I really wish popcon asked two questions right up front:

1. Is this machine primarily used as a laptop, desktop or server?

2. Is this machine owned by a business or by a person?

All sorts of popcon results are taken as being authoritative, when (I think) the overwhelming majority of responders are personal owners with laptops or desktops.

Similar. But I also set it off on vm's if I'm testing something.

It'll quickly and destroyed, so I just keep thinking I'll pollute their stats.

It makes sense: if a service asks politely and does not use dark patterns, it's a signal that they take your privacy seriously. So it feels good to reward them and give the permission.

Without making a value judgment about whether it is the right thing to do, it has been well established that automatic opt-in vs default opt-out numbers favors automatic opt-in ( usually discussed around organ transplant issue ). I agree with you in theory ( as in I know I almost automatically do the opposite once I find someone tries to opt me in without my say so ), but I also know inertia is a powerful force.

Yes, but with statistics you don't need everyone to opt in–just some portion. You can't really extrapolate a kidney ;)

Usually what happens with opt-in versus opt-out is that the people who are the heaviest users of your software, the ones who care about it, the ones who might know its details or care about privacy will turn it off and you'll just be getting analytics from people who are casual users–and it's really easy to end up interpreting this as "hey, nobody uses $OBSCURE_FEATURE, let's remove it!". Whereas if you make it opt-in, yes you will generally have smaller numbers, but the distribution might be more useful to you, and you're less likely to think that just because you have numbers on 70% of people you're going to make the right decision.

Unless you think they want your data at the expense of the majority, this doesn't seem relevant. The difference between opt out and opt in is massive, 80% of people don't care or don't read. Your irrational preference here doesn't even register on their charts

> 80% of people don't care or don't read

I don't think it's "irrational" to suggest that you shouldn't be collecting data from those people as they have clearly not given consent to you doing so.

I wasn't referring to that, thanks. I was referring to your own irrational willingness to hand over the same data based on how they ask for it. Maybe it is rational; you know they're not getting any useful data with an opt-in, so you'll play along with their incompetency by handing them a single datapoint?

It's the same reason why I would rather want someone who approaches me on the street to ask me for money (which I might give!) rather than just stealing it from me silently and and then offering the option to return it when I ask for it back–of course only if I notice the theft. I take issue with you calling consensual data collection "incompetent".

Making a decision based on principle does not strike me as being irrational.

it's not the same data though. if you default to on, you get the data from people who tend to leave the defaults the way they are. if you default to off, you only get data from people who will read the text in the installer and decide to change an option away from the defaults.

Arch operates like Debian, the package is called `pkgstats` and it's an opt-in to install. The results are public: https://pkgstats.archlinux.de/ (like Debian, the results are only as good as those who choose to opt-in)

Thanks. I wish this were more visible in the install process / guide, I had never heard of this before.

nod It's mentioned briefly on the "General Recommendations" wiki page in one sentence, not exactly easy to find. The Installation Guide wiki page is protected from edits, so an official wiki Admin would need to approve and make the change (see the History tab for recent editors you could contact).

Thanks for the reminder. I actually read about this somewhere (probably on ArchWiki), while installing Arch a few weeks ago, but I totally forgot to actually opt-in.

Really nice distribution, btw! My favorite, so far.

"This, IMO, is the perfect way to do telemtry."

Maybe someday this will be the only legal way to do telemetry.

They could also track it at server side which would not need any form of consent.

Probably not, because of caching and mirrors. There are many, many mirrors, including global CDNs. Debian makes no attempt at controlling package distribution. Also packages are signed, but sent over HTTP to allow easy caching, and there are no-fuss caching packages in the repos. If you have a site with even a handful of debian or ubuntu machines, you would be well served to setup a cache to save bandwidth and improve update download times. This all results in making it impossible to get even remotely accurate statistics server side.

The popularity contest package is checking which packages are actually actively in use[1]. It keeps track of file access time with a 12 hour resolution and any package used in the last thirty days is considered active.

I think this might be interesting for packages that are part of the default install, server side statistics would always show those even when they are not used.

[1]https://popcon.debian.org/FAQ

Ha Ha! I've got an apt-cache server. Suckers!

These behaviours are precisely the reason why I enabled popularity-contest, and have telemetry completely disabled on most other software.

Ask beforehand or it'll be denied; mandate or lie and you'll never get a single byte, ever.

Fully agree. The only downside is that the data they receive is probably representative of a small segment of users (my guess is free software enthusiast desktop users, but I may be wrong).

Reply on news.ycombinator.com