Audacity: Actions we propose to take on PR 835 | Hacker News Reader

Audacity: Actions we propose to take on PR 835

github.com

190 pointsjraph175 comments

I often think, especially if you have 'technical' users, better than telemetry is just making it easy to tell you things.

I frequently want to be able to just quickly say 'gah, that was annoying, I clicked that instead of that because I thought it would do that, not seeing this covered by those' or whatever, just vent a little bit, and vaguely hope it might be improved.

Not a support chat, happy to have fa follow-up email if more information needed and it's actually being worked on I suppose, but generally just a quick easy fire and forget.

Like telemetry, but where the user provides all the content; not a glorified yes/no to content you've guessed ahead of time.

Subnautica had this in early access: a "feedback" function where you chose either a smiley or sad face, filled in a free text field, and one click sent it to the devs. From what I hear this worked pretty well.

Yup. And Visual Studio has Send a Smile/Frown, firefox had something similar plus a "Report site issue" button that takes you to a simple web form.

Those things are not super hard to put in place, and seems like a great way to get feedback that people wouldn't travel all the way to the Github issue tracker for =)

Visula studio is also teeming with telemetry, though.

Yup, this has been best advice I've listened last year, and subsequently built on-site community chatbox. It's been amazing source of QA, feature ideas, discovering who are the real people using the service... We even hired from it.

My only reservation was that it won't scale but even for userbase of 100s of thousands registered and millions of uniques a month (in the parlance of dumb telemetry), it has! I love it.

To anticipate "why not just discord/telegram/whatever channel/server" - doesn't work as well, there's certain immediacy of it being purpose-built and being right there on site, that's apparently very valuable.

> My only reservation was that it won't scale

If you get too much feedback, can't you just ignore part of it?

Some basic sorting/filtering can put common feedback topics into groups, letting you see the rare and unusual ones more easily. Much better than ignoring it.

Sure, you can improve it. But the point is that as you scale, even the naive approach doesn't get any worse.

Talking to users can be cheap and easy - easier today with the ease of running remote usability sessions for example.

Telemetry is the lazy, unthinking approach developers typically take. They think the telemetry data tells you what users are doing and how. But it doesn't really. It does not tell you how easy, difficult or frustrating the user finds using an app. It does not tell you whether the user accomplished their goal or gave up in frustration.

If anyone still thinks running a usability study (to spot usability problems in an app) is too complicated, read this 20+ year old article from Jakob Nielsen (of NNGroup):

Why You Only Need to Test with 5 Users: (published in 2000)

https://www.nngroup.com/articles/why-you-only-need-to-test-w...

Tantacrul, the project lead for both MuseScore and Audacity, is a professional UI/UX designer and has once conducted a significant first-time user testing for Dorico [1], so I'm pretty sure that he already knows what you are saying. Given his latest video about Audacity [2] though, I guess that Audacity has a very diverse user base and it might be hard to do the user testing while accounting for that diversity (the article you've linked specifically mention this issue).

[1] https://www.youtube.com/watch?v=S-3wEC6Fj_8

[2] https://www.youtube.com/watch?v=RMWNvwLiXIQ (includes an interview with original Audacity developers)

So he's already flubbed this once before. How many flubs until we can assume malice?

Im getting a Miguel de Icaza vibe from him. Wolf in sheeps clothing.

It's even worse than I imagined. He's refusing to say who owns and how they came to own Audacity 'for fear of getting it wrong'...

Something shady is going on for sure and this guy is their PR wall.

And he's made it clear repeatedly now that he's not actually in control of the project or empowered to make any significant decisions on it, despite what his youtube video implied. He appears to be just a glorified UI/UX contractor, albeit one with a huge youtube audience. Whoever IS in control clearly has no understanding of the community or history of Audacity, and no interest in gaining that understanding, given that he apparently had to beg them just to get them to step away from sabotaging the software within a week of buying it.

>So he’s already flubbed this once before

To my understanding he has never worked for Dorico in any capacity. He did the usability testing independently for the sake of his video essay, not in order to actually guide work.

"It does not tell you how easy, difficult or frustrating the user finds using an app. It does not tell you whether the user accomplished their goal or gave up in frustration."

You actually can, it just takes a more sophisticated analysis. For example by tracking mousemovemt you can quite easily tell wheter the user finds what he wants or gets more and more stressed (faster and more undirected mouse movement)

But I agree that you should only do so when explicitly testing your design with consenting (and paid) testers and not your regular users. They will indeed be happy, if they find a form where they can give you precise feedback.

(depending of course on the software - you will get better feedback from CAD software users, than from the typical online game)

> quite easily tell wheter the user finds what he wants or gets more and more stressed (faster and more undirected mouse movement)

Could be just me but I don't make frantic mouse movements when I'm stressed or can't find something. Also sometimes I just let the mouse drift around as I scroll or read that probably looks like undirected movement.

My point being that I think these metrics have a lot of inherent noise. With mountains of data you hopefully see the averages shake out but my gut tells me you shouldn't put too much weight on the analysis of these kinds of metrics and they should be supported by other sources of user feedback with better SNR.

User testing and analytics are completely different tools and their purposes should not be conflated.

Analytics are for quantitative analysis of the entire UI. For example, if I want to see what tools are and are not being used, only observing 5 workflows would be massively insufficient. Sure, it doesn’t tell me how people feel - but that’s what user testing is for. Analytics can tell me if nobody is using the shiny new button that we put into the menu (at which point I’d do user testing to examine why this is happening), or that many users make heavy use of an item buried three levels deep in sub menus that we thought wasn’t important (but maybe turns out to be essential to some specialized workflow).

We were told to use the term “usability testing” since after all were weren’t studying the users.

Slack provides this in a slick way through the /feedback command

Slack is a program where the natural interaction with the user is a text field, so a /feedback is up there with about: in the browser's address bar. The user's hands are already on the keyboard. Something like Audacity where the rare time a user enters text is primarily in a Save As dialog, so the user typically isn't in the mode of thinking about text entry and their hands may be no where near the keyboard.

It's a bad idea:

1) people never answer those

2) they're bad at telling what's happening

3) this does not scale at all

Telemetry is an automated process and it's just the way to go.

There is lots of evidence in a variety of contexts that humans are very bad at self reporting this kind of thing. You might get bug reports by soliciting feedback but you wouldn’t find out if people discovered a new feature or UI shortcut. How could they complain about a thing they didn’t know about?

Hence the "technical users" requirement.

If anything I would wager “technical” users are worse at this sort of accurate usability feedback than average.

If they don't know about it, then it isn't bothering them. Why would you want to change that situation?

> Not a support chat, happy to have fa follow-up email if more information needed and it's actually being worked on I suppose, but generally just a quick easy fire and forget.

Yes! Many indie early access games do this great, having a button in the pause menu to just send title + body + optional email with any feedback. Click the button, message sent.

Raycast (Mac app launcher) has a similar take: https://raycast.com/blog/feedback/

Building a “send feedback” action into their app was a smart move and I’ve used it quite a bit.

> We assumed that making it opt-in would allay privacy concerns

That seems like a reasonable assumption to me?

That's not primarily what it was about.

"...the convenience of using Yandex and Google is at odds with the public perception of trustworthiness..."

> We assumed that making it opt-in would allay privacy concerns but since this isn't the case, we are dropping it.

Especially after dropping Google & Yandex, I guess they could've stood up for their telemetry needs, and flat-out say that "if you have privacy concerns about our anonymous opt-in telemetry, you can just use the default settings."

Once the feature and toggle exist, reversing the default is a much lower barrier than adding the feature. Keeping it expensive helps reassure users that they'd notice the change.

Absolutely agreed. A program without telemetry being possible at all is not one that I need to worry about enabling it behind my back. Imagine if /bin/bash had optional telemetry. Suddenly I'd need a way to ensure that telemetry is never enabled. In order to make sure that no confidential data are leaked, I'd need to have that setting always be visible, perhaps by adding it to my $PS1. I'd need to make a mental habit of checking that setting before executing any command that contains confidential data. All that additional overhead as a result of telemetry being an option at all.

There are a couple different stages on the sliding scale of telemetry, which have huge effects in the amount of privacy being sacrificed on the part of the user. However, the biggest technical hurdle is in that very first step, from "No telemetry is possible" to "telemetry is possible". Every single step after that is small and incremental, easily enabled with minimal effort.

* No telemetry is possible.

* Telemetry is possible, but requires searching in a menu to enable.

* Telemetry is possible, and has a single opt-in popup, path of least resistance is to disable telemetry. (e.g. An unchecked box that enables telemetry, and a "Continue" button. The easiest path is to ignore the box and just click continue.)

* Telemetry is possible, and has a single yes/no popup, both accept/decline are equally easy to select.

* Telemetry is possible, and has a single opt-out popup, path of lease resistance is to enable telemetry. (e.g. Same as the opt-in popup, but with the box defaulting to being checked.) This is the level that starts to raise red flags for me, and to make me feel uncomfortable with it.

* Telemetry is possible, and has several nag screens. These may re-occur each time you start the software, or re-occur sporadically. These may have the euphemistic option "Not Now", a newspeak phrase that prevents me from saying "No" entirely.

* Telemetry is mandatory, but can be blocked at the router level. Any software here and above is spyware, and should be treated as such.

* Telemetry is mandatory, and software refuses to run without a server connection.

> Imagine if /bin/bash had optional telemetry.

That immediately brought PowerShell to mind -- it boggles the mind how much opt-out telemetry Microsoft has in their core offerings.

Holy wat? I had to look it up to make sure you weren't joking, but yes, PowerShell has telemetry [0]. I hadn't ever looked into it when developing on Windows because it hadn't even crossed my mind that that would be something that was done. I think I'd insert "Silently enables telemetry without telling you and with no indication that telemetry can be disabled" just below the mandatory telemetry stage, and would extend the "spyware" description to include it.

[0] https://docs.microsoft.com/en-us/powershell/module/microsoft...

Not just PowerShell by the way. Most of their CLI offerings (compilers, shells, ...) that I've used have opt-out telemetry.

> Silently enables telemetry

Iirc most of them tell you about that the first time you use them, or they did back when I still used Windows.

The default was opt-in (telemetry off).

It is a reasonable assumption. They just totally flubbed the original incident. Someone just dropped a PR saying they're merging in telemetry without any issue discussion, or forum post, or anything, and all the initial review discussion on it was able how they should vendor their dependencies - no discussion about telemetry.

Both yes and no imo. I strongly believe that they have good intentions and that the reception has been a bit in bad faith.

I think it's a reasonable assumption to make, but as a user I'd want insight and granular choices, rather than just "allow telemetry? y/N"

If it's opt-in and there are privacy concerns about where this data goes to after you opt into it, then probably fewer people will choose to opt in. The option outlined in the submission may attract more people to opt in, thus making the data more conclusive and valuable.

If I where in their shoes, these would the multiple dimensions I'd think about when taking that decision:

- legal (living in Germany I am legally only allowed to share personal data from my users with entities with whom I have a data processing contract ("Datenverarbeitungsvertrag") that specifies how they treat said data)

- practical (using a provider which people trust, will make more people switch on telemetry. People who use your software usually trust you – so it makes sense to collect telemetry yourself)

- personal (I want to know what I am running where, feels better to me)

- political (no need to feed big monopolists yet another shovel of user data. After all I make open source software because I want the world to become a better place)

- ethical (is it ethical to use a provider which could do god knows what with your users data? Users trust you, which means you have to choose providers which you can trust)

- symbolical (whom does a open source project trust with their users data? What does this in itself communicate?)

That being said this is opt-in, which leaves the decision about many of these things to the users. However to me this episode revealed something about their thinking on that specific issue – or rather the lack of it. I wouldn't have raised an eyebrow if they managed to express a good reason why they want to rely specifically on those services, why they trust them with their user data etc. What raised my eyebrows is, that they didn't think about that at all, which means the privacy of their users isn't valued as high as I would have assumed prior.

Their reaction gives some hope they learn from this.

The PR had the opt-in choice button set as default, which is not very GDPR compatible.

It definitely is but unfortunately the original PR didn't say that it was opt-in, only that it was "optional" which usually means opt-out.

Obviously people on Reddit and HN assumed the worst and by then it was too late to get the truth out.

Bit of a shame; I feel like telemetry probably would have been quite useful to them (e.g. for optimising which things are in the toolbar). Though, on the other hand I think Audacity's usability problems are obvious enough that they easily have a decade of work before they get to the point where they run out of things to improve.

Yeah, opt-in is totally fine. I'm straight up militant about opt-out spyware bullshit in apps, but opt-in should be acceptable to everyone.

So we've gone from "Spy on the users in an open-ended way that can grow ever larger" to "If it crashes, you can push a button to send us a report" and an automatic update check (not automatic download!). Both of which can be turned off. That seems reasonable.

That said, this has vaporized any trust I had in in Muse group, and it's not coming back any time soon. When big companies "acquire" high quality software or sites, the result is almost always either a. Destruction of the property in an attempt to monetize or b. They mostly forget about it and it gradually rusts away from neglect. Here's hoping Audacity is somehow an exception.

It was opt in. There's barely any difference, except crash reports have far more personally identifiable data like hardware configuration.

That means nothing. The issue is having it at all. Changing it from opt in to opt out is a tiny change that can sneak past people checking

I recently wanted to split a long mp3 file into individual tracks. I used Audacity for it. As someone with zero experience in sound editing, I expected it to be more complicated than it turned out to be. We need to have such open source applications available to us without fear of telemetry sharing with ad companies. Glad to see the decision.

Based on your stance it seems that you have not seen the intentions declared in the original PR, including opt-in approaches and the use of data solely for development and UX improvements?

I just don't want to take the chance about telemetry data being used for monetization, now or in future. Also, the original PR's setup meant that even if I wanted to share telemetry, there was no way for me to share that without it going to Google/Yandex. And I definitely don't trust them.

So, first of all, as someone who works for Google, I don't think you should be worried about it. Using personal data is currently pretty strictly regulated. Using 3rd party analytics data in such a way would likely be illegal (IANAL and I'm not writing this comment on behalf of Google).

And even in the worst case, supposing that Google and Yandex are evil, what's your exact concern? That you presses on Play and Record buttons will be used to target ads?

Imo the last part is a bit too dismissive. It's not too hard to imagine how telemetry from software can be used for ad targeting, or for more nefarious purposes, since the telemetry would have access to track info, including metadata.

If we're imagining worst-case scenarios, my first thought is "use track info to discover unlicensed music usage". Or for ads.

I don't think the actual application-specific events would be used by Google - as every application has their own events and ways to use them there would be no generalized way to do it without having developers manually "reverse engineer" the event meanings and assign them to ad targeting signals.

What Google most likely does however is use persistent analytics IDs to track people and improve their on-site tracking. Let's say you clear your cookies and happen to change IPs (dynamic IP, etc) so you appear to Google's web properties as a new user - all they have to do is wait for some other piece of software on your machine to report analytics with a persistent ID and essentially bridge the gap between your old identity and your new one, so now just based on IP alone, Google's web properties can infer with good accuracy (and the more datapoints the higher it goes) that it's you.

This is EXACTLY what I, and I believe MANY other people

#DONOTWANT

I agree that this might be a concern, but since Audacity is open source sending any sensitive information like track metadata would likely be caught very quickly.

First, illegal obviously doesn't matter to Google. They are collecting GDPR fines but continuing as before. Responsible authorities in Ireland are asleep at the wheel, probably intentionally. As a user, "illegal" doesn't calm my fears, Google fearing to go bancrupt from fines would. But we aren't there yet. Also, Google will mostly avoid responsibility by dropping it all on the Audacity developers and their terms of service.

And ad targeting by button presses isn't the problem. Telemetry transmitting audio content, memory dumps, screen shots, home directory content is the problem. As an end user, I cannot distinguish between the harmless button-press telemetry and the harmful versions. Pinky swearing in the terms of service doesn't help, users have been lied to too often.

Oh, and btw, button presses can also be harmful, e.g. for an on-screen keyboard, a browser or any application where buttons reveal user data.

> First, illegal obviously doesn't matter to Google.

I'm just an engineer and I have good visibility only in the project that I'm working on, but at least from what I see, the privacy policy is taken very seriously. All product changes are going through legal review, and I'm not aware of any instances where any illegal or even "gray area" changes were knowingly rolled out in production. When GDPR came into law, a lot of work was spent on making all the systems compliant with it.

I'm not saying that everything in Google's billions of products is strictly legal, but "illegal obviously doesn't matter" is obviously wrong.

I'm afraid that the question answered by all that "legal review" is "what can we get away with", not "what is legal", and that that was the principle that informed your GDPR not-quite-compliance big initiative.

That the GDPR introduction required a lot of work is a sign that Google failed to comply with German data protection law in the time before. Google did business in Germany, so would have had to comply. GDPR is, in most aspects, a straight translation of what was necessary to do business in Germany for decades. So why should I trust that Google will comply now when it didn't before?

CNIL (the French DPA) could fine Google because Google failed at something as basic as having a Data Protection Officer in their supposed European headquarters in Ireland. If Google legal fails at a two-line appointment letter and an address entry in the privacy legalese, how should Google's legal review be any better?

As to taking things seriously, take for example youtube. For some time we have now been getting a cookie consent banner, but that was introduced only some time after GDPR coming into effect. And that banner is obviously illegal, because it clearly implements the "I agree" vs. "Customize" dark pattern. So I cannot reject as easily as I can agree. And even if I click customize and select "Off" at "Ad personalization", there is still the sentence below that says "We rely on cookies to remember your settings and other preferences. We also use cookies to [...] deliver, maintain, and improve our services and ads". Also, there is the plain lie that "You can change your browser settings to reject some or all cookies.". You can block cookies, after which youtube just won't work in Firefox. I suppose in Chrome it just uses some other hidden identifier.

So please don't try to tell me that things are taken seriously when I just need 30s to find blatant violations and gray areas. Name any other Google site and I would wager I could easily show you more obvious violations like that. I can accept that an engineer won't know or care about stuff like that, but claiming you have never used youtube and never seen what I described seems odd to me. I can accept you wanting to defend your employer, but everyone always says something like "it is fine in my project". That only serves to increase my mistrust in any claims by Google employees when things like the above are clearly not OK.

> Using 3rd party analytics data in such a way would likely be illegal (IANAL and I'm not writing this comment on behalf of Google).

Your company is breaching the GDPR with their current tracking consent flows (Google's consent flow does not pass muster according to the ICO's guidelines: https://ico.org.uk/for-organisations/guide-to-data-protectio...), so just because something is illegal doesn't mean Google won't do it, and their friends at Facebook did something similar and got caught using 2FA phone numbers for ad targeting even though they promise they wouldn't.

Are you aware of these Googley actions?

1) people's location data they weren't aware was being kept (dozens of stories and nuances now),

2) scraped SSIDs of WiFi routers from...the world...via Street View cars, said "Whoopsie"

3) collected MAC addresses via free terminals and hotspots in...NYC was it?

4) Allowed multiple cross-storage access bugs where users have accessed each others' shit in Drive....

Anyway, I fear you are not an authority on Google's effectiveness at preserving customer privacy.

I'm not claiming to be an authority, I'm just commenting on my experience.

> 1) people's location data they weren't aware was being kept (dozens of stories and nuances now),

Location history is currently off by default.

> 2) scraped SSIDs of WiFi routers from...the world...via Street View cars, said "Whoopsie"

I am quite sure this was an honest mistake.

> 3) collected MAC addresses via free terminals and hotspots in...NYC was it?

Not sure what this is about.

> 4) Allowed multiple cross-storage access bugs where users have accessed each others' shit in Drive...

Again, I don't remember this story, and anyway, bugs happen.

> and anyway, bugs happen.

This is an extraordinarily blasé who-gives-a-shit response to a critical security vulnerability. Hopefully your attitude isn’t representative of your employer. Maintaining the privacy of the data customers entrust to you should be your highest priority.

I'm sorry, but a publicly traded company's highest priority is to it's shareholder. To them money is the only priority.

To think otherwise is a bit naive.

User privacy is usually an afterthought, or a PR statement. Since you are the product being sold, your expectation of privacy should be somewhat lower.

For working at Google you seem to know very little about them.

1. Data was sent even though location history was turned off. [1]

2. It is pretty well-known that SSIDs can be used by location services to increase location accuracy, especially where GPS coverage is low or noisy [2], or slow to connect.

3. Would be useful for location tracking again.

4. That seems like a bug that wouldn't really benefit Google at all.

[1]: https://apnews.com/article/north-america-science-technology-...

[2]: https://slate.com/technology/2018/06/how-google-uses-wi-fi-n...

A good reputation is hard to earn, and easy to lose.

A bad reputation is even more difficult to lose, realistically it never goes away. Ever.

Google has a bad reputation these days. No amount of PR will help. Ever.

(Doubt my above logic? Imagine a person who donates his time to help the poor, then is caught stealing from the poor. Will anyone care about the donation time, or will they primarily only think about the theft? The betrayal?

Will that person ever prove they are "pure" again?)

It is not strictly regulated in the US. Google has internal policies that they aren't required to implement by law.

Google/Yandex aside, I don't want intelligence agencies around the world tracking which applications I run locally and when I run them.

Or GP has experience with such promises in the tech landscape, e.g. FB's promise about WhatsApp data.

Or they don't believe in telemetry ever bringing value to users, only to the ones collecting it.

What I'm about to say I've already said elsewhere in this post, but Audacity has been an incredibly loved and popular FOSS project for 22 years and it's done that without using telemetry.

It's hard for me to not be pessimistic and see the recent sale of Audacity and the introduction of telemetry as the beginning of the end.

FYI, if you ever have this same usecase again, look at mp3splt. It does just splitting of MP3 (and some other formats), with the significant benefit that it does not decode-and-reencode. It also works as a pure CLI tool by just giving the input file and timestamps for where to split.

> We need to have such open source applications available to us without fear of telemetry sharing with ad companies.

Do you pay for audacity?

The hysteria about this is absurd. To make product decisions you need to measure actual usage patterns. To do that you need telemetry.

No, you don't. You could talk to people, instead.

then you'll have opinion of vocal minority

Yes. Hopefully developers and power users, if they're chosen correctly and given the proper incentives (like having their ideas implemented).

Surveys and user interviews are also useful, but nothing beats having the data. And usually we use the telemetry data for the app to craft the surveys and interviews. Did you stick a cool feature on a dialog but nobody ever uses it? Telemetry tells us that's something to look into. The thing is that this is very easy to do with web apps, since you get a ton of telemetry for free through API requests. A desktop app is s black box without telemetry. Just release and pray

A 10-30 min walkthrough+interview of how 5 users use the software will elicit by who and why the dialog isn't being used.

When was the last time you reached to talk to some project maintainers of your favorite open-source software? You have the answer to your question then.

I give opinions in tickets all the time.

This is floss were talking about, so budgets are a factor.

That also biases data to a very narrow technical user.

I think we as a community need to come up with officially recognized names for such activities to lessen the cognitive burden on installing such software...

Telemetry can be useful but it's hardly the only way, and it doesn't have to be opt-out.

People made "product decisions" before the internet.

> People made "product decisions" before the internet.

People still pull numbers and assertions out their asses all the time. That doesn't mean it is, or was, good.

Telemetry doesn't guarantee that people won't pull numbers out of their asses. There are lies, damned lies, and statistics.

Designing good telemetry is hard, interpreting the data well is harder (and it's tempting to cherry-pick post facto to justify your decisions). Are users not using functionality X because it's useless or because there's some other issue preventing its use? Or maybe users just don't know that it exists?

Crash reports are more of an unmitigated good IMO however, since they can be used to pinpoint the source of crashes and identify regressions.

> Designing good telemetry is hard, interpreting the data well is harder

The biggest problem facing any use of data is quality of the collected data. For example, if you wanted to train a model but all you had were examples of a particular class... well, you excluded yourself from being able to predict any other class from the git-go. Data collection itself require a deep understanding of statistics and probability. Otherwise you’re just shooting yourself in the foot.

But let’s be honest, machines don’t “learn” things, people do. Machines have no teleology they determine for themselves.

People are both the problem and solution. Anyone who thinks big data and ML will be able to generate the answers are deluded and have already turned off or lost completely their critical thinking skills.

Data collection in the form of telemetry is only a mechanism for finding solutions and an incredibly INVASIVE one at that.

What exactly are you saying in regards to this case about Audacity though. It was opt-in and they had already been making product decisions yet they knew they could make better decisions with more information.

Sorry, I was making a general case. I think being opt-in definitely alleviates most concerns.

I just generally dislike this trend of putting telemetry in every singe application, it sets a bad precedent and can easily be abused by less principled developers (especially for closed source software) in order to syphon data for all sorts of purposes.

> they knew they could make better decisions with more information.

That's an unproven assumption. More information does not lead to better decisions, it just leads to more confidence in those decisions (whether justified or not).

"could make" not "will make" - unless you're arguing that having information about what you're doing will always fail to help you do it better in which case ¯\_(ツ)_/¯.

There are a ton of improvements to be done on any software project (especially venerable ones) by experienced devs before resorting to telemetry. Talking to users/customers should be first.

Software does not need to move at the speed of light, much less something like audacity which has a huge user base and hopes of staying useful for decades.

Telemetry has nothing to do with "moving at the speed of light", it has to do with discovering actual usage patterns. There's no requirement to move fast on telemetry. In fact you probably want to do the exact opposite, because you want large representative aggregates.

If you want to discover actual usage patterns, there is a much more efficient way: Sit down with 10 users and observe them while doing their everyday work or while performing a predetermined set of tasks. This is even easier nowadays thanks to screen sharing, so you can even put a focus on getting a geographically and demographically diverse sample of users.

From personal experience, I've always had more insights into actual usage patterns from sitting next to a random user for 30 minutes than I could have had from heaps and heaps of telemetry.

This is a strawman. The heat about this is not about whether a project needs telemetry or not. It is about how Audacity chose their service partners (without thinking about it) and it is about how they communicated it (a PR request without prior discussion).

That being said – even if they had communicated transparently and discussed openly – if their goal was to get good telemetry maybe self hosting allows more users to trust you, and therefore you receive more telemetry? Because the overlap within the venn diagram of users who trust Audacity and users who trust Google/Yandex will always be considerably smaller than the overlap of people who trust Audacity and people who trust Audacity.

What you call hysteria was caused in big part by the fact they were going to use Google and Yandex for this.

My pessimistic take is that Audacity has been one of the most popular and well-loved FOSS projects for 20+ years, without telemetry. That's 22 years of amazing software that didn't have to track users in order to be so good.

It's hard not to see the recent sale of Audacity and the introduction of telemetry as the beginning of the end of an otherwise stellar piece of software.

Audacity is very rough and buggy. Its popularity came from being free and featureful, not polish or focus.

IMO: the UI is pretty good the way it is, I was even thinking that the last time I used it (last fall, I don't use it all the time which shows just how intuitive it is to people who don't use it often.)

Not really. There's lots of other ways to learn about how your product is used, and without them telemetry will lead to some bad decisions. I've personally had to clean up the mess when telemetry driven decisions go sideways so many times:

* No one seems to use American express, so let's remove it from payment options. Turns out, the site had a bug that made sure 100% of AMEX transactions would fail. By fixing, we picked up 6% in sales, and lost a UX associate who was (and probably still is) arguing to drop AMEX because analytics.

* Since very few people actually click the apply button on the job board, let's remove the apply feature. The idea was we could send candidate profiles to employers based on views instead of apply clicks. Had to revert this one.

* Same company: let's take job posts out of our mobile job search app because most interactions end on a job post (and a click on the apply button). Best ticket the next day, "I can't find any jobs in this job search app?"

* No one uses the UI customizer (dashboard colors, fonts, and preferences). 100% of the tickets the next day were from people who were PISSED BEYOND BELIEF that we removed their favorite feater. Turns out the customizer basically was invisible to our app analytics. Had to revert this and re-do two sprints of work because the other new features wouldn't work with the customizer.

In 100% of the cases, talking to actual users would have been beyond wise before using analytics data to make decisions.

Totally agreed. Just because a feature is used by 2% of users doesn't mean it should be killed. That feature (plus others) could be critical to the workflow of those users.

It seems the telemetry-driven product development in the past decade has made significantly worse apps than in previous decades. There are definitely a higher quantity of apps on a wider variety of platforms, but it seems harder to find apps that are powerful enough to allow a wide variety of users to do all the things they need.

The long tail of features is where a system achieves its status as "polished." Is this why software never gets polished any more? (And no, all features don't need to be visible at once.)

Telemetry is just data. It can help you make good decisions, but can't prevent you from making bad ones.

The general lesson in your examples isn't that telemetry isn't useful, but that telemetry gives an incomplete picture.

Telemetry is part of a toxic pattern: It gives people a sense of control, which makes them feel more confident in their decisions, even though that confidence is usually not actually backed up by anything meaningful.

Telemetry isn't the problem. Making assumptions about what the telemetry means is where most of us go wrong.

I'm gathering you're not against telemetry, just against over relying on it?

Would the AMEX issue have been found without telemetry drawing attention to it?

Telemetry in the sense of automating the collection of useful data might be worth the effort, but there's no need to share that data with Google or Yandex.

Considering sharing user data with some cloud service is reckless enough to cast a serious shadow on a company's intent and morality, even without an actual GDPR breach.

1) It's not hysteria and calling it such is disrespectful and dismissive.

2) No you don't.

Did no one make product decisions when building software prior to the mid 00s?

Not related as such, but worth pointing out. The author of this discussion - and the project manager of Audacity - is Tantacrul, who is, in my opinion, one of the best music youtubers around. His video[0] on corporate music was fantastic, and his video on Shostakovich[1] made me totally reconsider his music. Plus, he's funny! So there's that. Highly recommend!

[0]: https://www.youtube.com/watch?v=G77ev9pks4I

[1]: https://www.youtube.com/watch?v=MCxzMYVvHBg

His reviews of the various scorewriters (Sibelius, Musescore, Dorico) are also fantastic, even if you are not into music. His review of Musescore lead him to become the project manager of Musescore, and probably of Audacity as a consequence.

His video Stock Music & Reality TV - How to Misrepresent the World [1] is also really good.

[1] https://youtube.com/watch?v=G77ev9pks4I

There is some fishy stuff going on with audacity and "muse" that nobody involved seems to be able to just say out loud simply.

https://github.com/audacity/audacity/discussions/880

Personally I would rather use closed source reaper or an audicity fork than deal with whatever is going on with audacity. Everyone involved seems to have been sworn to secrecy which generally tells me it's not in my best interest to get involved and pit myself in a position where this conspiracy has any influence over me.

Can you point out the problems shown by this discussion? Because I can't.

The concerns are legitimate but let's give Muse some time to answer properly. I haven't seen anything shady… other than the initial telemetry thing that has been widely fixed. The community has been listened, I see this as an actually very positive thing for Muse.

I'm very happy to see private companies take care of open source projects (if they do it properly, obviously)

The history of musescore is not really the brightest, which may explain why people do not trust them much ([0] and discussion). Yes, they shelved telemetry and listened to the community, which is good, but they still proposed an intrusive change without discussion and only turned back because the outrage was big enough.

[0] https://github.com/Xmader/musescore-downloader/issues/5#issu...

>he concerns are legitimate but let's give Muse some time to answer properly.

It's 6 days old, and they have some responses in the discussion but nothing that clarifies what was going on. It appears to be heavily NDA'd.

Musescore has done some interesting things

https://github.com/Xmader/musescore-downloader/issues/5

A bit off-topic, but in all the posts surrounding Muse Group acquiring Audacity, including the one you linked, I never understood why people keep saying things like "what does it even mean for a company to acquire a FOSS project?" or "a company can't acquire anything other than the trademark".

Well of course they can acquire a project. To acquire the project doesn't just mean the trademarked name, but also the rights to call their version the official version, the official website to distribute their builds on, "BDFL" status including write access to the canonical VCS repository and the right to accept or reject pull requests (unless, of course, the project doesn't have different governance policies in place).

They might also acquire the right to the previous owner's source code, which means that they could (assuming they re-wrote all the other parts that were written by other contributors) dual-license the project (without violating the GPL).

They should outright state what they bought, who they are, what their plans are, etc.

As of now, it's fishy. Uncertain. They should just state outright what's happening and why.

https://github.com/Xmader/musescore-downloader/issues/5

Here is some interesting activity by musescore threatening another developer over things musescore didn't have any rights over.

I dont trust them. They don't act like people worthy of trust.

I am also curious about this. The attitude in both of these cases suggests to me there's a either a cultural or leadership issue at the company.

> but also the rights to call their version the official version

I'm not entirely sure what this really means beyond the protections already afforded by the trademark.

This is one of the key open source strategies used by Red Hat, Microsoft, Intel, Google, etc. I am not implying anything nefarious here. (Cygwin, a product of Red Hat, specifically advertises on their commercial license page how many core contributors for various open source projects work for Red Hat. This allows them to quickly fix bugs for major customers.) In short, if there is a very important open source project, you hire a lead developer from the project who has commit access to the main repo.

The Muse Group website doesn't even have an impressum: https://mu.se/

Some comments on the GitHub thread claim that it's actually a Cypriot shell company. Having read a bit about the situation with MuseScore, it looks rather shady...

"log a non-reversible hash of the IP address to improve the accuracy of the daily statistics."

2**32 IP addresses seems not to be hard to reverse. Am I wrong?

For sure, unless they use a very computationally expensive hash.

Beyond that I'm not sure why they'd get from that, given that non-static IP addresses are very common on the internet and NATed networks even more so.

IP are considered PII and are regulated. So the easiest way is to avoid PII.

If you XOR the four octets into a single byte then it’s definitely irreversible :^) (and also pretty useless for telemetry purposes)

they could also just generate an "installation" id ? this also works more accurately when the computer switches ip addresses.

> works more accurately

That's what they want to avoid. An advantage is, that you can ask the user for the id an check the logs for debugging.

You could generate an 128-bit installation ID and then send [date, hash(id, date)]. Now when you're debugging with a user you can ask for their ID and see which traffic is theirs.

won't work because date = receive date and now you have a unique id and an IP.

Sorry, why won't it work? The idea is that as long as you don't know the ID you can't tell which traffic is coming from the same person across whatever time duration you choose to bucket at (finer bucketing gives more privacy but also more computation to decode the logs once given the ID).

And if I give you x and hash(x, y), this does not tell you what y is (it just lets you very likely confirm y if you have it)

It might become a bit harder if they salt the hash with some other data, but no. Especially because you can reduce the search set even more.

Even though in the specific case of audacity, I'm not sure what the negative implications of a disclosure "I use audacity" by access-log of a server would be. So to me it's fine.

The whole telemetry would be correlated to the IP. I'm not sure what they gather now.

There is no good reason to put any form of telemetry in a non commercial open source project, ever. It is just user hostile behavior. Yes, even if it is opt out. Good to see it got pushed back. Let's see for how long.

Yes, people will say it "helps make the product better" but open source software, especially the GPL flavor, is for serving the user first and foremost, not making the developers job easier and I strongly doubt it really helps with the later.

People these days are just horribly addicted to data gathering and believe quantitative methods for analyzing problems are the only right way. Probably for need to justify their job. Every good open source project is getting more than enough good qualitative feedback from the community that you should focus on. Improve the tool for the people that actually use it and are part of the community and not for some potential new users you hope to generate.

> There is no good reason to put any form of telemetry in a non commercial open source project, ever.

That's simply not true, and there are absolutely reasons to put telemetry in projects, including non-commercial open-source projects e.g. tracing performance issues, tracing crashes and other bugs, collecting stats on feature usage (to know what to focus on or what to surface better) or workflows (to know what to improve support for, or what to better document if there are better ways to do things in the project).

Getting useful feedback directly from users is difficult, and doubly so when you have no activity traces to go with it. The average Audacity user is never going to capture ETW traces when they have issues, and odds are they'll complain on audio tools forums rather than your own.

I'm absolutely sensitive to the argument that telemetry is commonly misused and offline software adding telemetry is dodgy and icky, but pretending telemetry is necessarily useless and bad is just fals.

> Every good open source project is getting more than enough good qualitative feedback from the community that you should focus on.

1. they really dont

2. especially for a project aimed mainly at non-dev users (which is what Audacity is), it's very unlikely that the feedback you get is relevant to the majority of your userbase

We disagree about the goals non-commercial open-source projects should have. I am focusing on serving the (established) users, you on things that might help the developers. I does not matter to me how much telemetry might help development at all.

Even if all those things you listed were substantially helping improve development, nope I don't care. Software should serve the user and only the user. As a user I don't have any benefit from the use of telemetry. Sure maybe it might help fix issues in futures version for me but again it does not benefit me as a user of the current version of the software.

I am well aware of the benefits of telemetry. If you develop commercial software it is absolutely critical to gather as much data as possible. But again the goal of open source non commercial projects should not be success at all cost.

But to argue about the benefits of telemetry for arguments sake, nope raw data is not really that useful, turning it into actionable insights is still a job on its own. Software has been developed without telemetry for decades. There is lot's of way. From getting testers to actual user surveys and so on.

Making it easier to develop the software helps users, because then users get better software

I already expected people to make that point that is why I wrote:

> it does not benefit me as a user of the current version of the software.

That is like the argument that ads are good for me because it allows companies to offer services for "free". Yeah, no thanks.

Also it is not true with Audacity. The reason they are thinking about telemetry and the like is because they plan to do a massive redesign of the UI. I don't want this because I don't want to relearn how to use the software.

I have no idea why you are being down voted. Everything you said had borne true in many other cases.

> There is no good reason to put any form of telemetry in a non commercial open source project, ever. It is just user hostile behavior.

So tell me, why do people use telemetry at all? Why would it only be beneficial for commercial projects? That doesn't make any sense.

I'm in the process of building an audio application myself. One of the things I worry about the most, when dealing with complex end-user software, is how do I as the developer discover bugs and issues my users are experiencing. Only a tiny fraction of users go our of their way to submit a bug report, let alone do it properly so that it's of any use to me. Good telemetry is key to improving my user's experience, and I'm putting a lot of effort into getting that stuff right.

Literally just used it (still open) to to create an audio performance for my kid's schoolwork. I use it once a year on average, all platforms, for a decade or more. It's awesome, though the whole import/export is not save/load thing seems superfluous. While this makes sense to programmers, for normal user UX I would recommend removing that and just normalize to save/save-as with warnings if data is being lost. Telemetry or advertising would make me use alternatives.

In high school, one of my teachers flubbed the recordings of an oral exam by having multiple student recordings in a single Audacity session and only saving a version with all tracks overlaid on each other. If anything, I think save/export need to be made more distinct.

Import/export has its own distinct meaning that is unrelated to save. It’s common to almost every content creation tool that has a distinction between an editing format (eg .psd, .reason, .fcpx) and a publishing format (.jpeg, .mp3, .mp4)

Sounds like some sort of plagiarism if you are doing your kids schoolwork. Let them do it themselves.

Looks like they're handling this (use of google analytics, yandex) properly. Promising.

OT but am I the only one always opting in on telemetry prompts? I want the developers to get as much info possible to improve the app and fix issues.

I think more people would if the telemetry was not using Google and Yandex to get the analytics and instead used their own servers and API that no third-parties have access to. Also if there had been more discussion in the first place before implementing this opt-in telemetry.

Yes - my problem with telemetry is never to hide from the app's developer, it's to hide from scum like Google and Facebook, or even "innocent" paid analytics providers like Mixpanel because while those don't profit off my data, they still concentrate it into a single place which would be a problem if leaked or misused.

    s/if/when/

By new management, broke developers, advertisers, data brokers, criminals in government, etc.

Can't argue with you there. I actually didn't know apps used Google for that. I always just presumed it went to their own servers.

Fantastic, that's your choice. Does this shed light on the subject?

Super non-obvious to a lot of people despite good intentions, so good on Audacity for following up.

We were surprised w something similar in Caddy 1: managed sw can monitor, self-hosted shouldn't unless manually added with real friction and alarm bells, vs default on, click through, etc. Otherwise, it's a time bomb that will burn your brand for a long time.

Messing up trust issues like violating privacy isn't just about the immediate incident, but the ones you already missed and the ones in the future. Telemetry is a recurring footgun here.

Well, that's nice to see. Let's hope both Muse and the community can have a productive exchange now that a sign of good faith (dropping Google Yandex, changing the source default to be telemetry free, making much clearer what is connected and why) has been offered.

Anyone here want to speak up for opt-out telemetry to help understand product usage patterns?

Surely "opt-in" destroys any validity the data would have (self selected groups are usually a poor proxy for typical users) even aside from the massive drop-off in participation.

Considering the people who would give out their zip-code in the supermarket at checkout, I would hazard the guess that the less technical crowd would opt in and that their feedback or telemetry on what makes a simple UI is more valuable.

But in the case of a checkout where you give info letting them track your purchases, you get discounts in return. Audacity isn't offering anything in return for opting in to telemetry.

What you're getting in return is improved UX and performance, albeit it is a more abstract and collective gain than discounts.

You also get (unless you are a contributor, I guess) a very capable and transparent music editing software for free.

In Germany I have never been offered a discount in that manner. Coupons are the only way to do this here.

I'm also German. Once, when I was asked for my zip code at checkout, I asked what they use it for. The cashier said it's being used to decide where to distribute physical advertisements (those store-branded magazines that land in your mailbox every week which list all the items that are on sale next week). I can't know how much truth there is to it, so take it with a grain of salt.

One of our local supermarkets could not check out until we gave them one, so we found some to give them.

For Belgium: https://www.bpost2.be/zipcodes/files/zipcodes_num_nl_new.htm...

Fun ones are 0612 (Saint Nicholas) or 1110 (NATO). We got great mileage out of 3 suisses until they went bankrupt.

Further random fact: The state and the postal services in Belgium don't always agree on the zip code. We found 1 location in Ghent where a monastery used to be, but it had been torn down and houses were built. The first inherited the address of the monastery, and hence had - on its own - a whole postal code, no street and no house number. But only for the postal services, not for the state (or the other way around, I dont remember). This location actually crashed the server for one geolocation provider.

Just to say, we are hackers, we're good at these games. They want a zip code? Find one for your country and feed it to them.

I remember a situation when after checkout an employee asked if I’d provide my zip code for statistical purposes. I politely declined and she froze, as if her brain didn’t know how to handle the exception.

> Surely "opt-in" destroys any validity the data would have (self selected groups are usually a poor proxy for typical users)

Is there actually any data to back this up? Personally I see no reason why an opted-in group would be using an application that differently from the masses - your hot paths are still hot paths, common errors and annoyances will still be triggered. Really the only major thing I see that could be different is that a certain hard to find feature might be used more than it would otherwise.

But disregarding that, the argument is not "opt-in" vs "opt-out". Opt-out is not be on the table at all - silently collecting information from users without their consent or any affirmative action on their part is immoral, and damages user trust. So when you take that into account, the options are opt-in data vs no data whatsoever, and opt-in, taking into account any potential bias you think this data might have, is still more infinitely useful then getting nothing whatsoever.

Putting discussions of deanonymization to one side for a second - how is collecting aggregate usage patterns "immoral"?

If your issue is about the potential for deanonymization then that's understandable but that's a discussion where we would need to talk about specifics.

> how is collecting aggregate usage patterns "immoral"?

What I do alone in my own home should be known only to me.

Do I read books? It doesn't matter if I do nor what books.

Do I listen to music? It doesn't matter if I do nor what songs.

Do I watch tv? It doesn't matter if I do nor what videos.

Do I sleep? It doesn't matter if I do nor for how long.

Do I play games? It doesn't matter if I do nor which games.

Do I use a computer? It doesn't matter if I do nor what software.

What I do in my own home is private. Anyone looking at that without my explicit consent is violating my privacy. Usage patterns of what I do in my own home are immoral whether in aggregate or not.

Nobody is talking about what books you read.

We're talking about "What percentage of users clicked this button or used this audio filter?".

No, you missed the point. Do I click buttons in my own home? It doesn't matter if I do or which ones.

Recording what buttons I press is a violation of my privacy.

You're entitled to think that and I'm entitled to think that at this point your opinion is unreasonable and slightly silly.

Is it immoral for utility companies to compute aggregate statistics about how much power/water/gas their customers are using?

Utility companies can collect truly aggregate statistics over an ensemble of users from realtime measurement of electric loads and water and gas flow through their pipes or over time from each user's meter (provided it actually records only total consumption, not events). Software doesn't have this luxury.

Those are streaming services, not applications, and they can't be made into applications.

I don’t think anybody is aiming for statistical-research-publication level rigour on the data here. Yeah, it’s not an ideal proxy for genpop, but it’s far better than nothing. The goal here isn’t to be able to say 1.454% of users try feature X, rather it’s to be able to say “only a few users have tried feature X, is it correctly explained in the UI?”.

I agree that it is bad for feature decisions (a filter rarely used can't be removed, maybe the few times it is being used, it is important)

However telemetry reporting crashes with context information can be useful to identify bugs.

And the combination can be useful "each time a filter is being used it crashes" vs "the filter is used often, but crashes rarely"

Wouldn't "opt-out" just self select in the other direction?

>These decisions took time to arrive at because - despite my role as the lead on the project - calls on this specific issue are not mine to make.

That's just sounds like huge red flag to me.

Only real suspect line in the entire OP, and retroactively justifies the angry user response. Whatever arguments one can make for developers and designers to use telemetry to improve the software are irrelevant. Instead, the owners were making a business decision to add telemetry, not the project lead, and that's the beginning of an attack on the users.

The only well-done telemetry I have seen in open source is what 0ad does. Everything is fully opt-in and very transparent.

>the convenience of using Yandex and Google is at odds with the public perception of trustworthiness What's the likelihood that this will percolate to non-technical services? If more services think like this, maybe there is hope.

Reply on news.ycombinator.com