Hacker News Reader

Comment by gcatalfamo | Hacker News Reader

> And it’s a malevolent spectre if you’re the 1% of users who actually use a niche feature in something, since developers will use that knowledge to take it away.

Yes, but also no. If my resources are not infinite, knowing that an "expensive-to-maintain" feature is used by just 1% of users, it is an incredibly important piece of information when I'm gonna have to decide where/how to recalibrate my (team's) effort.

Product managing is also this: I have to know what works and what doesn't. Maybe I will use that knowledge to understand how to increase the 1% to 5% with a specific business case. Maybe it won't make sense and I will have to decide if I can keep it or not.

I agree that vanity telemetry is bad for users, but killing ALL telemetry is bad for everybody.

View on HN

> Product managing is also this: I have to know what works and what doesn't.

You can't use a blunt usage metric to tell whether a feature works or not. It only tells you that the feature is popular for some reason. That reason might be that it works, but often the reason people use a feature a lot is it's shoved down their throats.

It's a vicious cycle. Say you are Product Manager for Feature X. Your bonus/promotion/career progression depends on some vanity usage metric for feature X. The more monthly actives or new users for feature X, the better it is for you. So, naturally, you're going to do whatever you can to funnel users at that feature. Stick it prominently on the main screen. Highlight it in flashing red. Spam the user with notifications begging them to use it. Make competing features low-contrast gray on gray. Whatever you can get away with. Then the next monthly metrics report is in: The feature is now being used by +15% of users since last month! Incredible! Your bonus buys you a Tesla. Management's conclusion is: "This feature really works and users must really like this feature." So, the product focuses more on developing that feature.

Now scale the above out to every feature in the app, each with their own competitive and motivated Product Manager. This is how we get apps that are all-telemetry dumpster fires and with more and more flashy features fighting each other to grab your attention and fewer and fewer useful 1%-features.

I've personally witnessed meetings with multiple furious product owners literally shouting at each other about where their feature's button gets placed on the home screen because that placement made the difference between 5% feature usage and 25% feature usage. Where was the end user in all these discussions? Not present. End users are to be milked for metrics gains.

> You can't use a blunt usage metric to tell whether a feature works or not. It only tells you that the feature is popular for some reason. That reason might be that it works, but often the reason people use a feature a lot is it's shoved down their throats.

I never said that was the only source of information, but indeed a useful one. Thinking a good PM only uses that one bit of info means you have known really bad PMs in your work.

The product managers know this, but the higher-ups (who control the money flow) tend to get tunnel-vision on a single metric, and that metric is often going to be the one most understandable to a layman. So that's the one that gets juiced.

Unfortunately that's the conclusion that most managers will draw, without asking more questions.

But knowing that only 1% of users use this feature is not enough to make that decision:

- maybe 1% of your users only use it, but they are the ones that pay the most money

- or it's the most prolific users so it generates half of your content

- or it's a niche feature used by power users, but those users are the ones bringing other users, introducing you to companies and doing free support

- or it needs to be used by only 1% of the users by design, and it's enough to be useful (E.G: bug report form)

- or your report it wrong, it's one percent of all accounts, but 10% of actually active users

etc.

Most telemetry will not tell you that of course.

Well, tracking users by cookies or fingerprinting or whatever can tell you that by combining data from different sources. Now we've looped back around to being in favor of the most intrusive methods.

That's my point, if you want really useful information, you need identifiable users, and for that, you need consent.

So ask for it.

All data will be biased and incomplete anyway, so at least acquire it in an ethical manner.

As I said that is a "piece of information", not the whole story. Making an axing decision with ONLY that information would be outright idiotic.

Telemetry has to be designed as well, to piece as much as you can, and then you complete the full picture with other sources. Maybe not everybody does this, but I don't think that this practice should be "shocking" for anybody.

yea you sound like someone who is competent and will actually thing things through and make a rational decision. Many people are not and will latch on to the information "only 1% of people use it!" and refuse to listen to why that may not matter"

> Most telemetry will not tell you that of course.

Why...? If that's the case, it sounds like the problem is too little telemetry, not too much.

Well, in Europe, if you want to know how much a user pays, you need to tie telemetry to their account containing personal information, which is forbidden by GPRD unless they give their explicit consent and they have a process to delete/rectify/download all of it.

Since most telemetry are optout and don't bother about giving users access to their data, they can't legally do it.

That's my point really: useful telemetry is really, really invasive. So just ask consent, and accept most users will tell you no.

If you chose what's convenient instead, you will not only have a mixed bag of data, but also you have decide to basically force yourself on people, in an age where a part of your userbase is getting upset over things like using the wrong pronoun on them.

I tell yes to Firefox and VLC to report on my usage. It tell no to Google and Microsoft. They want their cake and it it: behave badly, and then have we trust them.

> give their explicit consent

They already need to agree to Terms of Service when paying and starting to use the software.

And that is not explicit consent.

Explicit consent needs to be standalone and refusing it cannot stop the user from doing anything that doesn't actually really require said consent.

Right. A mention in the ToS does not equal real consent. I would only add that it also has to be informed consent. Saying something like "can we collect data from you to improve our products and services?" is also not soliciting real consent, because it tells the user basically nothing about what is being asked of them and how it will be used.

You think that keeping track of "This paying user uses this and that feature, and is thus on the correct price plan" is a violation of the GDPR, assuming it's mentioned in the privacy police and maybe ToS?

I don't know what does or does not violate the GDPR, and I'm not talking about metering services. I'm talking about telemetry for installed software.

I'm not even talking about what the law says, because the law is very inadequate on these matters (in the US, anyway). I'm talking about, from a common-sense point of view, what "consent" is.

In my view, it can only be counted as "consent" if I have been informed exactly what it is that I'm consenting to, and I am asked for that consent.

ToS don't count for a number of reasons, starting with the fact that nobody reads them, it's unrealistic to expect people to read them (because everything comes with lengthy ToS documents and if you really read them all, little time would be left to do anything else), they are generally difficult to properly understand if you're not a lawyer, and they are are usually intentionally vague of this issue -- which means the "informed" part of "informed consent" isn't satisfied even if you do read them.

I think the Principle of Least Surprise should apply here. If software is going to do something that users won't notice happening, is intrusive, and isn't pretty obvious from the nature of the software, the right thing to do is to tell the user about it and ask for permission.

Software that doesn't do this is adversarial to the user.

That seems interesting, when thinking about pro rated subscription fees -- if a feature isn't being used, it'd cost nothing. But with telemetry disabled by default, the company couldn't detect that the feature wasn't being used, and thus the customer would pay more. And if opting in to telemetry, the customer pays less and the company gets telemetry.

Ok, but if we focus on results, why has software usability and functionality declined with the rise of telemetry.

I personally think it is because program managers rely on telemetry instead of stakeholder research and industrial design, either because they don’t know better, or because management thinks it will cut costs.

Besides, telemetry is a feature that 0% of users use, so it should be the first to be cut.

Subjectively speaking, while software I use these days has a lot less settings, I also tend to need to change settings a lot less than back in the day, defaults tend to be just right a lot more often. So at least for me it seems to be working, at least somewhat.

> Subjectively speaking, while software I use these days has a lot less settings, I also tend to need to change settings a lot less than back in the day, defaults tend to be just right a lot more often. So at least for me it seems to be working, at least somewhat.

But there's a trade-off there—as, ideally speaking, most software has become easier for most people to use mostly satisfactorily right out of the box, it becomes all the harder to adapt if you're one of the people for whom it doesn't work mostly satisfactorily, since there's no real culture of design around the idea that a user might want to use software in the way that they, rather than the designer, intended. Even if it's fair to write off those niche users, there are still the most users for whom the software works mostly satisfactorily, but can't be modified to be perfect, again because the culture of user customisability just isn't there.

Right. There used to be a design philosophy of making the defaults fit the majority of users, but to expose plenty of settings so that more advanced users, including those who have grown into the software, can still do what they need to do, and can work around design decisions that are bad for them.

That seemed to be a reasonable compromise to me.

In fact, it's something the US Air Force figured out after spending substantial time and resources.

That is to say: There is no such thing as the Average Man.

Make something that will generally fit most people, but then also add enough customization features so each man can tune it to his liking. There is a reason car seats and steering columns are adjustable.

The dichotomy isn't between telemetry and no options vs. no telemetry and all options. Somehow a decision needs to be made what to include because everything has at least an opportunity cost, and historically that has usually been made by "common sense", i.e. what whoever is in charge feels like is the right set of options. That can work out great if you're very much like that person or able to influence them (and they want to listen), but if you aren't, too bad, and the vast majority of people out there is not in either position. Late 90s/2000s me was probably a huge lot less like those people than many here on HN, and I guess that explains some of the difference in experiences. I generally seem to be quite a lot like "most people", which makes an approach that targets the middle of the bell curve work well for me. I'm not a big fan of embedding surveillance technology in apps, but I'd say the people who do are on to something and what's missing may just be a more transparent and privacy-preserving way of doing it.

My subjective experience has been the opposite. Modern software tends to be inflexible and limited, and those limitations absolutely adversely affect me. There isn't a day that goes by where this sort of thing doesn't interfere with what I'm trying to accomplish.

If you only care about users for which your software works, then most users spend almost all of their time as “power users” (= ones that have used the software for more than a few days)

Focusing purely on out of the box experience looks good for quarterly metrics (subscriber growth with low initial churn) but you end up with a population of users that are desperate to switch away.

At that point, you are one viable competitor away from being screwed. Worse, as users defect, telemetry will tell you that usage of the features associated with switching is declining, so you’ll double down on stripping out the functionality that you should be investing in if you want to defend your market share.