Hacker News Reader

Comment by fancl20 | Hacker News Reader

Asking nicely doesn't work in pratice because people willing to answer the question is not a randomly sampled group of people and usually highly biased. You can check how polling works and why it doesn't work.

The problem is always how to randomly sample people and AFAIK there is no good way except randomly sample people (a.k.a telemetry). You can argue that developers shouldn't care that much about their software are being used and whether they are performing as expected, but asking nicely really is not a solution.

APPENDED: PLEASE NOTE I'm saying asking nicely is not a solution to get correct data from statistical standpoint. You can argure the data accuracy is not important in this context.

View on HN

> Asking nicely doesn't work in pratice

LLVM, GCC, rust, zig, D, etc don't have telementry and they seem to be doing fine. What makes go special? They aren't trying to be more efficient than rust. It isn't more widely used than GCC. I would argue they don't need this telemetry at all.

No one claimed it's essential to have telemetry or that a language literally can't do without it. After all, Go is doing fine without telemetry too.

But that doesn't mean it's not useful, for Go but also gcc, or Zig, or D, or any other language.

A long list of use cases is given here: https://research.swtch.com/telemetry-uses

Useful doesn't mean it should be done. There are a ton of useful things for me I don't do because it's not right to do them.

Apparently as soon as a company becomes big enough, the line between convenience and ethical becomes very blurry.

> Useful doesn't mean it should be done.

I never said anything about what should be done. I'm just saying it is useful, as the previous commenter said it's not.

When did I say it isn't useful? I said they didn't need it. They also don't need to install a kernel module that records every keystroke and memory access, but it would certainly be useful data.

Have you asked core developers of those languages how they feel about this? I'd bet that you'd have near unanimous agreement that this class of information would be useful for informing support decisions.

Where I'd expect difference is precisely over the question of whether it's worth the cost & trust issues. That makes me wonder whether there might be some middle-ground here where some third-party like the Linux Foundation could run a telemetry service which is highly public in both its design and collected data, and whether that would be enough that many people would be comfortable with this kind of service without Google's business reputation entering into the equation.

> useful for informing support decisions

Useful does not equal ethical or legal. A lot of things would be useful to me.

Please enumerate. Go on.

Hence the second paragraph in the comment you're replying to. There's no question that this could be implemented legally but the ethical considerations are legitimate and warrant extensive discussion for an open source community. I think that the Go proposal has been derailed by a lot of hot takes based on scenarios far in excess of what's actually described, but I also think it's quite likely that many languages' core developers would look at that and agree that they don't want that information badly enough to spend time trying to negotiate those hazards.

What if your plumber installed a camera in your bathroom for informed support decisions? Sure would be useful for him to see where you put those "biodegradable" baby wipes after the deed.

A more apt analogy would be a sensor which anonymously reports if a pipe starts leaking so the manufacturer has a sense of their failure rate.

Do you really think that's anything like the scenario being proposed?

Here's what I think would be a closer analogy to what's actually being proposed: the water company installs a smart meter allowing them to read usage data more frequently. They randomly sample 2% of customers' data and prepare a histogram showing the top daily usage hours and total usage over a monthly period.

The water company already has a connection to you (the pipe) and the smart meater measures that connection. The smart meter does not collect information how you use that water inside your house. This analogy would match a website collecting anonymous usage statistics for connections but would not match an otherwise offline program making connections to collect data.

Yes, that’s why I think it’s a close fit for what Russ proposed rather than the more outlandish conspiracies circulating in social media.

I always wondered how much intelligence they gather from the Go playground.

Probably the data is too wild to make any use of, but then all the folks who have a clue using the playground as a kind of proof-of-concept play area... would be great for analysis.

Anyways, I'm not sure how well it would work, regardless of if it should be a thing at all. The majority of stuff I do, and the folks at work, we all have build pipelines that control internet access, use cache systems, etc... so I'm not sure it even matters.

I was actually wondering how much this would affect many build pipelines — if you're doing your builds in a container, the 7 day threshold would never even be reached unless you enabled caching for that file.

The main thing I think this would do is help them flesh out the non-mainstream configurations. For example, I've used Go on AIX but I'd be surprised if bug reports on that platform didn't hit basically every maintainer by surprise since statistically nobody tests on it regularly or even has access to do so. I'd think most of the value of systems like this would be finding out that the change they tested exhaustively on everything Google normally uses is also going to break 100% of some niche community.

FWIW, there is a proposal to add telemetry to LLVM [0] and Rust used to have telemetry [1], both off by default. Some things in the node.js world have telemetry enabled by default, like Next.js [3].

Some people are posting here as if this is already decided -- AFAICT, that's not the case. It's not even a formal proposal yet, and the stated intent was to start a conversation around something concrete. (For context, this is standard for how the Go project approaches large topics, including for example I think there were something like ~8 very detailed generics design drafts from the core Go team over ~10 years).

It sounds like the Go team is going to take some time to look into some of the alternative approaches suggested in the feedback collected so far.

In any event, this is obviously a topic people are very passionate about, especially opt-in vs. opt-out, but I guess I would suggest not giving up hope quite yet.

[0] https://discourse.llvm.org/t/rfc-lldb-telemetry-metrics/6458...

[1] https://github.com/rust-lang/rustup/issues/341

[2] https://nextjs.org/telemetry

C# ( net core ) has telemetry, as well as Java so it's not that it does not exists in other very popular languages.

https://www.oracle.com/java/technologies/javase/terms-java-u...

https://learn.microsoft.com/en-us/dotnet/core/tools/telemetr...

Also the idea of GCC is more used than Go and GCC does not have telemetry so GO doesn't need it is completly unrelated.

Most dev working on core tools want telemetry to improve those tools, it's invaluable data to make the right decisions.

> C# ( net core ) has telemetry, as well as Java

Which Java distribution? Maybe the Oracle JDK/JRE? I highly doubt the Oracle JDK/JRE has serious usage numbers because of it's (quite commercial) license.

> dotnet, vscode (Microsoft), golang (Google), Oracle JDK/JRE, Nextjs (Vercel), ...

I wonder if there's some trend where corporate backed, large "open source/source available" projects being more open to telemetry compared to more "open community" projects.

For non-corporate projects, the general thought process might be it's fine to leave efficiency gains and improvements on the table if it means violating some foundational principles. While in a business setting, those efficiency gains could be very tempting since it can translate to more money, market share or promotions and that way of thinking gets applied to the open source projects as well.

I think your analysis is correct, and would also add that corporate projects probably have both the culture of monitoring (they use it for everything else) and the expectation that they have to deadlines so they prioritize the idea of being able to ship something with the confidence that it either won't break anyone or they'll be able to know & react quickly if it does. Open source didn't used to have that same focus since it was rarely someone's job and there was more understanding that time was in short supply (“if it breaks, let me know or send a patch”).

It probably also helps there's more easy access to infrastructure for all of this: servers and sysadmins. Ignoring everything else, that's much harder for your more classic volunteer effort open source project: servers cost money, and maintaining them costs time (usually better spent on actually writing code and such).

> GCC is more used than Go and GCC does not have telemetry so GO doesn't need it is completly unrelated.

I mentioned that because an argument could be made that they're the most popular, or they're trying to be the most efficient language and that's why they might "need" this telemetry data.

> Most dev working on core tools want telemetry

Yet somehow, almost all open source, popular, efficient tools don't have telemetry. Like the ones I listed + python + Linux + coreutils + bash.

Of course they want this. But there are mountains of successful projects that work very well without it, and every one serves as evidence that this Isn't as important as they let on. We shouldn't subject our personal computers to google tracking because they want more tracking data.

In case anyone wants to mention that it's Google's language and they can do what they want with it, this is a perfect example of why people aren't happy with Google's control of the language

> What makes go special?

it's owned by an ad company

> Asking nicely doesn't work in pratice

Well, to bad. Because if you don't, you've built a piece of malware, by definition. You need to ask, live with bias, and draw your conclusions accordingly.

I'm confident that you're not jumping from "asking nicely is not a solution" to "using force or deception is fine."

It's just that, if the argument in my last paragraph did become widely accepted, it would lead to a pretty brutal society. (And, of course, it is becoming pretty widely accepted where software is concerned.)

While true in general, it is really not that necessary to have a representative sample for checking whether your software works. With large enough sample size a convenience sample will find almost all of the bugs that telemetry would have found.

The bigger question is why do they suddenly need this to such a degree that they need to do a backhand tactic to get the data? Most other programming languages have no such data and have survived and thrived for years, decades even before Go existed.

It seems like it’s only a proposal at this point?

Generics were also controversial and it took multiple iterations and years to get them done.

I wouldn’t call it “suddenly”.

Also, why do they need the data “suddenly”? What do you suggest?

It's neither sudden nor backhanded — if either of those characterizations were accurate, we'd have learned about it from something shipping. Announcing it publicly before implementing anything and having a number of privacy protections built-in suggests neither is a fair description.

Reading through the use-cases at https://research.swtch.com/telemetry-uses one thing to consider is that these are the kinds of things people care about as a project matures — once you have a large number of people depending on your project, you have to worry more about unintended consequences as you evolve and since the user community has grown larger and the project more stable it's also harder to know whether the people you hear from regularly are representative. When a project is small or mostly used by enthusiasts you can be more confident that your understanding of what people are doing is relatively accurate but that just isn't true over time. For example, how many millions of Java developers never contact Oracle or the OpenJDK developers throughout their entire careers?

The same argument could be made by many other entities include government bureaucrats and LEOs. Justifying data collection with "it makes my job easier" is a very slippery slope IMO.

My code is on GitHub as is a large number of other peoples. They can try to compile it themselves with all the profiling and probes they want, on their own machines.

That's not the problem they talking about. It would be useful in some cases (and I'd be surprised if they haven't already considered experiments along those lines) but what you have on GitHub doesn't easily tell them about your build or runtime environments.

Here's a long list of questions Russ Cox was interested in - notice how many of them would really want something like “what is your CI configuration?”, and consider that while they could sample GitHub Actions configuration that doesn't tell them anything about the much more varied set of projects who don't use that service:

https://research.swtch.com/telemetry-uses

That's a very forced, contrived, un-impressive list. It reads like someone told them "hey, we want telemetry, please make up a list of uses that will sound plausible to programmers to justify it. Be sure to use big words like histogram a lot to shut down objections by sounding real smart."

Most of these things are in the category of bug prioritization. Why don't you... you know... talk to users?

If you want some statistics (oh sorry, histograms), maybe pull down the gigabytes and gigabytes of Go code residing on public repositories and places like GitHub and analyze all that. That'll tell you a ton about how people structure their projects, what architectures they target, etc.

Biased sampling only matters if the bias has some relevance to what you’re investigating. There is zero reason for software telemetry to care about privacy bias.

> Asking nicely doesn't work in pratice

Who cares?! It's about respect for individuals and respect for their privacy. Any kind of telemetry should be opt-in no matter how you spin the PR.

By using telemetry you are losing the statistics from people who block it, i.e., the most tech-savy users.

This may be by design, as it punishes those who dare to value their privacy.

eg, some options:

a) No telemetry. All users of the Go language get equal treatment/consideration by the developers ("best guess").

b) With telemetry. The users who opt out are thereby excluded from consideration ("punished").

It seems like over time, option b) would lead to problems for the Go language as the developers base their decisions on incomplete information, whilst believing that information is somehow more complete than option a).

I'm not entirely sure how biased the data would be; there needs to be a correlation between "frobs with telemetry" (whether it's opt-in or opt-out doesn't even matter) and any other metric. I'm not so sure there is one, although perhaps there could be – we need telemetry to demonstrate that :-)

Questions about bias in reported data are legitimate and a genuine topic of concern, but this:

> This may be by design, as it punishes those who dare to value their privacy.

Is just veering off in the conspiratorial.

> Questions about bias in reported data are legitimate and a genuine topic of concern

The problem is an obvious one but doesn't seem to be considered. :(

> Is just veering off in the conspiratorial.

Punishing people who value their privacy seems like it'd be strategically useful to Google, an ad company with demonstrated intelligent but sometimes ethically challenged people. ;)

Isn't it contradictory to "care" about how privacy objectors use Go, while denigrating those people by suppressing their voices?

You seem to have a problem understanding the concept of consent.

Allow me to help you. You ask first. If the answer is no, you don't do it. If you cannot master this skill, or refuse to, you are embarking on becoming one of the most problematic demographics on Earth.

There is no excuse for not asking, and honoring the answer. Sending data is not free, and I guarantee your telemetry is not that important, neither is your vaunted sampling. All you want is to know who your Users are, and the fact is, that is fundamentally gated behind them wanting to even bother telling you.

It’s a simple value proposition: want priorities for your problems and use cases? Turn on the telemetry.

Just like answering polls makes more of a difference than voting.

Reply on news.ycombinator.com