Hacker News Reader

Comment by j_maffe | Hacker News Reader

> Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction?

I'm not sure if you're implying it's obvious but it's not obvious to me that it would be unhelpful.

View on HN

Just anecdotally, I get the feeling telemetry often does more harm than good, because it's too easy to misinterpret or lie with statistics. There needs to be proper statistical methodology and biases need to be considered, but this doesn't always happen. Maybe a contrived example, but someone wants to show high impact on their next performance review? Implement the new feature in such a way that everyone easily misclicks it, then show the extremely high engagement as demonstration that their work is a huge success. For Git, I'm not sure it would be widely adopted today if the development process was mainly telemetry-driven rather than Torvalds developing it based solely on his expertise and intuition.

Not to mention it's really hard to statistically tell the difference between people spending a lot of time with a feature because it's really useful or because it's really difficult to get to do what you want

Telemetry is a really poor substitute for actually observing a couple of your users. But it's cheap and feels scientific and inclusive/fair (after all you are looking at everyone)

That is just poor analytics IMO, if you have a good harness you can definitely tell if a feature is not well designed. You have to optimize for things like number of clicks to perform an operation not time spent in app.

I think the seeing the underutilized commands and flags (with real data not just a hunch) would have helped identify where users were not understanding why they should use it, and could have helped refine the interface and docs to make it gradually more usable.

I mean no solution is perfect, and some underused things are just only sometimes extremely useful, but data used smartly is not a waste of time.

Reply on news.ycombinator.com