Comment by colinchartier | Hacker News Reader

Comment by colinchartier | Hacker News Reader

Hey, this looks really cool.

In my line of work [1] I talk to a lot of developers about productivity, here are some questions about what you've built in the context of that experience:

1. I think (as you mentioned) "number of commits" or "number of pull requests" merged are proxies for productivity at best. Most teams that I've seen care more about "are the tasks for the epic being chipped down" - is that something your tool can help estimate as well? The progress bar for the epic? It'd be nice to get a table of the metrics and where they come from.

2. In the same vein, "scrum with epics" might not always be how teams decide on units of work, do you support other methodologies?

3. You mention deployment frequency a lot, but that seems tangent to developer productivity - Isn't getting things deployed quickly mostly for the benefit of the product folks? (getting feedback faster)

4. How does maintenance work fit into this? Does a developer who spends most of their time refactoring show up with "worse metrics" than someone pushing copy changes to the landing page?

[1] CEO @ https://layerci.com (YC S20)

Great question! I'll try to cover them :)

1. While we do use pull requests as a unit of measure, we don't actually consider # of commits and # of pull requests as great proxies for productivity. We mainly track Cycle Time, Deployment Frequency, Change Failure Rate, and Throughput which help encapsulate speed vs. quality of an engineering teams. It's important for engineering metrics to correlate to engineering work. Specifically, that the engineering team has full control over those metrics. When looking from an Epic, Story, or Task perspective - this is often controlled by other parties, making it more difficult to know if engineering itself is improving or not. In addition to this, Epics often don't reflect actual engineering work in the same way that pull requests in Github do making them problematic to focus on.

2. Because we focus on engineering work itself from Github. Haystack can be used with any methodology so long as the team is using Git!

3. From an engineering perspective, the main goal is '# of successful iterations'. Product's main goal is that those successful iterations are pointed in the right direction. Looking at # of successful iterations (aka speed, deployment frequency, and quality) allow us to see how often we deploy value to customers and what is the quality of those deployments. Focusing on these metrics allow engineering teams to improve 'how they deliver'.

4. We actively don't look at individual engineers and we especially don't compare them using any metric since we've seen that lead to some bad takeaways like the one you mentioned. We help measure at the team-level to simply highlight improvements and bottlenecks rather than cross-compare any team or individual which is like comparing apples to oranges

Reply on news.ycombinator.com