Making Go telemetry opt-in is a mistake
twi.github.io
twi.github.io
As I see it, relying on excessive telemetry, surveys, focus-groups, etc is a bit of an indication that nobody at the top has a strong idea of where to take the project.
Edit: fighting the markup trying to get strikethrough to work
They already do regular surveys.
I disagree, doing what you feel like is a great approach when you have no users, your building a tool that solves your needs and basically saying "Try this it, it works great for me, maybe it'll be good for you too!"
Once you have millions of users and billions of lines of code you need to think about how your changes affect those users. If you get it wrong there's a huge cost to that (c.f. early forays in to go packaging and dependency management). This is why go's comparability promise is such a big deal and a big part of why go is a very popular language.
just for the sake of argument, wouldn't telemetry be useful to know which of these issues are most likely to benefit the majority of users?
whether that's worth the cost of telemetry is another issue.
Inbefore I go to a new job and find out that they are using outdated, custom patched go compiler.
> I mean, there are 5000+ issues and 330 open PRs on the go github right now
How do they know which ones are affecting the most users?
> forced google spyware in a compiler
go is open source, feel free to compile it yourself without the telemetry. Which distros will do if any major promises would be broken
In terms of the open GH issues, people are pretty vocal about which ones they think are most important to fix, as is the case for most popular projects. It's simply not true that the Go team have no way of knowing which of the open issues are most important to the community.
They did stop. "Go 1" is complete.
Since 2017, the project renewed itself by moving to what is dubbed "Go 2"[1], being built around community feedback. Decisions are made based on data collected from the community. Adding telemetry to increase the amount of data available is in line with the new project scope. Go 1.13 was the first "Go 2" release[2].
Indeed, "Go 1" is still there if you want to use the tool that was built for what was needed at Google and nothing more. You don't have to move into the "Go 2" ecosystem. But if others want to put their time into that, why not? That's their prerogative.
> keep making the damn tool however you want.
But then you follow up with this:
> If the Go team at google no longer has any ideas for what to work on (as must be the case if they're wasting their time on dumb crap like forced google spyware in a compiler) then they should just stop.
They want to add telemetry. So they should just add it. If you dislike it, then patch it out yourself. As you say:
>That's how open source software is always supposed to work.
"Users are liars, so let's spy them directly to know what we want to know".
It is mind blowing how, as an user/the target, you can support that.
Nothing is really anonymous and your anonymous data can say a lot about you.
Telemetry coming from this IP, so company x is using go. A pattern of data coming every 2 days, so their build nodes rebuild every 2 days. That kind of build pattern is there, so they are using the xxx crypto library...
And when they say, let's trust Google, I would propose to Google to accept the opposite:
Now they will transmit to the public telemetry of their internal systems: how many users, what do they do, how many users they block, for what reason, how many build nodes they have, how many commits, how long the go team is spending looking at telemetry reports, which website are the more visited by Google employees,...
And let's see if they will accept. It's for the good of the world, why they would refuse?
Privacy used to be kind of sacrosanct in the industry. What happened?
I guess when surveillance driven advertising pays all the bills it has an extreme warping effect on everyone’s mentality. It’s hard to care about privacy when violating it is how you get paid.
I’m increasingly wondering if we might vastly improve our society by banning or taxing the hell out of all advertising. It’s an industry that makes the world a worse place in deep and pervasive ways that go much further than just making everything ugly and obnoxious. The industry pushes a toxic relationship between people and businesses at every level.
It is difficult to get a man to understand something when his salary depends upon his not understanding it.
It is gross. It's also very ignorant. The author's argument basically boils down to, "I can't see an obvious major downside to this proposal, so that must mean there isn't one nor will there ever be one". How many times to we have to get burned by large companies invading our privacy and after promising not to before we wise up?
> Telemetry coming from this IP, so company x is using go. A pattern of data coming every 2 days, so their build nodes rebuild every 2 days.
Not true. Even if Google lied about collecting IPs, the data would be sent only every ~ year with aggregated counts so no real time usage data. And even if one could see the patterns from the data, everyone will be able to, not just Google.
Let's not trust Google. But let's not shoot ourselves in the foot by refusing any automated cooperation.
Additionally, majority of distros use package managers, so if any of the major promises of the upstream would be broken, distro packages could patch it out. This isn't forced, there are several points where anyone can stop the telemetry.
If they don't want to do that, why exactly not?
Paranoia is imagining something. This is posing a question that they or you or anyone is free to simply answer, or fail to.
> Without knowing what ports of Go are used, the Go team can’t make sure that the right time is spent on maintaining those ports.
Am I being overly pragmatic if not selfish for thinking this is absolutely fine? If you use a free tool behind closed doors to make money and don't want to opt-in to telemetry, then I couldn't care less if the go team doesn't make go work better for your use case. Meanwhile I'm developing my open source project with telemetry turned on, ?????, PROFIT.
People really need to accept that they can't and shouldn't have full introspection into everything they're ever interested in. Even if it's a thing they care about (like their own homepage or programming language). Sure it's nice to get feedback. Sure it's nice to know that somebody uses it. But putting a tracker on everything with the argument of "will help me provide a better service", forgoing informed consent.. It's wrong when it's done on an institutional level, whether it's companies or the goverment, and it's wrong when individuals do that.
But the creators (someone trying to be "owners" of sowmth5 that is just a name, lol) has the foresight to make it free and open source, so we have Tencity, the original software with a new name.
After that, they resolved to only add "Don't Send"/"Send" crash reporting and opt-out update checking, which only sends the OS version, Audacity version, and IP address. (They claim to immediately anonymize the IP address.) The user is notified about the update checking at first startup.
Which part of this do you consider "doing this and gaslighting the community about it"?
There's your answer.
> Which part of this do you consider "doing this and gaslighting the community about it"?
Telling long time users that the new owners knew better. Everyone was able to go and check out how they ruined their other products too.
They had an army of people telling them "no" and they did it anyway.
You seem to be familiar with the fiasco so I'm not sure how we aren't seeing the same thing....
So you're saying that opt-out automatic update checking ruined their product by any reasonable definition, and they were entirely aware of this, so by claiming that automatic update checking is useful they were gaslighting their users?
If so, then from what source are we certain that the "army of people telling them 'no'" in the relevant GitHub issues is representative of how their broader user base feels about automatic update checking? And how do we know that they ought to have been similarly certain?
> Unfortunately, the proposal came from a Google employee. Google has a reputation of taking user data and feeding it into piles of linear algebra in order to skew its search results to match your existing biases or whatever the hell else people do with linear algebra.
If we lived in any other age, people might tolerate this, but we don't, and therefore everyone is well within their right to mutiny, regardless of how interesting a technical solution this is.
There's also a philosophical problem in going too deep into "customers don't know what they want" and A/B testing everything to death: you end up using it as a substitute for dialogue. Ultimately customers are making conscious decisions. You have to make the case rather than assert that it's "better" from behind your metrics dashboard.
So if someone says that opt-in data is useless, I'd say that out-out data is just as useless.
And then we have to pretend that implementation, collection and analysis and then further actions would make it worthwhile.
Googler 2: The data says 100% of users are opted in, so deprecating the obsolete and unpopular opt-out functionality is the evidence-based choice. Ignore the shrill demagogues on the mailing list saying otherwise, their anecdotes are meaningless.
The entire idea is bad, the defaulting reduces the impact to many but the very existence of this telemetry is enough to take more significant security defence against the tool. Once you start doing that as an organisation Go becomes legacy with a strong desire to replace it. Its definitely a mistake to make it opt in, the data will be lower quality and it will still drive security concerns.
Not true. The code is open source and everyone can trivially with minimal effort check that they are sending only the data they said they would send.
The consensus was that opt-in was the best solution, and thankfully Google went with the best solution.
You could argue that it's ultimately the users who suffer. Telemetry helps guide development, so it's harder for developers to know what to focus their efforts on.
And Golang also has the yearly survey that gives much more valuable inputs than telemetry could track IMHO.
They can, but do they? Russ' very first blog post gave several examples where major issues slipped under the radar because they went unreported. The example that sticks in my mind is that they accidentally introduced a dependency on a c compiler on Mac that was not reported for over a year simply because users assumed it was meant to be a dependency.
People were using it, they took it away (IIRC, could be wrong about that), people whined, they brought it back as an explicit operator, people really whined and the BDFL walked away.
Telemetry would have told them what exactly?
I’m not familiar with how it works now but the example I remember was the variables escaping from list comprehension statements, like:
x = [y for y in z]
if y != z[-1]:
bad_stuff_happened()
if (m := re.search("regex is(n't)? fun")):
print("there was a match")
# do something with m
[y for x in xs if (y := f(x)) == 5]
while data := fh.read(131072):
# do stuff
while True:
data = fh.read()
if not data: break
# do stuffIt's just a programming language not organ donation.
It will pretty much become useless (how many people are going to opt-in, realistically?) - and that's fine if it's what the community wants.
Not really. People hate opt-out telemetry regardless of who runs it.
So, they don’t test go in CI on clean Mac OS installs? If this sort of issue is the rationale for the telemetry system, it sounds like they’re trying for a older, slightly-less-evil Microsoft approach where you fire QA, and just treat user machines as your test environment.
"Telemetry shows people aren't using this (privacy / internet freedom related) feature, so we can just remove it"
There are bugs reports with 100s of people complaining, about simple features, or broken code, and the attitude is WONTFIX.
If you won't listen to users, seething, upset, annoyed over inane issues you created? If you won't fix bugs for years, because it's your pet project/feature?
Then how are you even paying attention to telemetry correctly?
Opt-in telemetry is only useful if you will, no matter what, implement and fix what it tells you. Mozilla, predicated upon their behaviour, clearly only uses data when it agrees with prior opinions.
opt-in is useless to such an org, and opt-out isn't why it is happening.
It's them. They are broken as an org.
If the Go team wants to see what features are used, go audit open source Go code.
Since Go is also a Google thing, they could likely audit the presumably large sums of Google Go.
Yes, you will miss unknowable amounts of code this way. Who cares?
Just trying to add this telemetry with opt-out in the first place shown one thing: They don't know their audience. Because if they would, if someone would have come up with that idea, everybody in the team should have screamed: "No, hell no!". You don't have the trust of your users.
I get it, getting trust back is much more effort than collecting more data (even if it's with good intent). But hey, the industry put itself into this.
Even better if the go tool were to have a `go update` with a `telemetry-off` flag or something. (and possibly a prompt to remind people otherwise they will complain again)
Problem solved.
Without an explicit agreement, it could even run afoul of wiretapping laws (unresolved in courts, as far as I’m aware).
When you 'go upgrade' your compiler, if telemetry was off it should stay off. No need to further specify in most cases.
If telemetry was on, you might want to turn it off, however.
Knowing the goal allowed rsc to make this judgement.
This was years ago. https://github.com/rust-lang/rustup/pull/1642