Comment by crazygringo | Hacker News Reader

Comment by crazygringo | Hacker News Reader

I was ready to dismiss all of this, but it's actually pretty interesting.

The big-picture view is given by Exhibit 1, and you can see it's really much more about analyzing all aspects of productivity (company-level, team-level, etc.) to identify bottlenecks around e.g. too much time spent testing over writing code, or senior developers stuck in endless cross-team alignment meetings which means management needs to identify a single person responsible for making final decisions quickly. So this is a document for engineering managers, not engineers.

But in terms of measuring actual individual developer productivity, if you read through all the verbiage it's really mostly just adding up the "planning poker" agile/sprint points that a developer delivers over the course of months. About which I have two main thoughts:

1) As long as planning poker is executed correctly -- i.e. always with unanimous consensus from engineers and without managers/PM's providing pressure -- this is actually probably going to be pretty decently objective, as a moving average. It's not terrible -- it's infinitely better than things like "lines of code". Although teams can also experience gradual "points drift" over the course of months, and points between teams can't be translated, so it can't be used company-wide or even year-to-year. But it will compare productivity between team members.

2) BUT, there's a huge risk involved because not all e.g. 5 point stories are the same, because of variance. Some are very straightforward where you know it's 5 points, while others are much more unknown, where the best guess is 5 points -- and it might wind up being 2 points but it also might wind up being 50 points. And so I could definitely see certain stories turning into "hot potatoes" nobody wants to touch because it's just too much of a risk -- it'll tank their average and they won't get promoted. And then everything just breaks down. Which means you either need post-sprint processes for "correcting" estimates and assigning a reasonable value for points done, or going deeper into "research" stories -- e.g. a 1-point pre-story to better nail down the actual size.

Ive always held the belief that story points were specific to a team, and a result of the work that specific team does and/or services, products and customers they support, which was a why comparing cross teams is a huge no no!

But story points themselves are actually guesses, and are never a true reflection of the work undertaken post mortem - and rarely if ever get reevaluated with 'correct values' mainly because it's really difficult to do that - it's another guess really.

I think any attempts to measure the accuracy of guesses on imaginary relative units, specific to a team, with deliberately obfuscated and difficult to reconcile units is futility by definition.

Instead companies have started to see the light of measurement using standard units of time and tracking the start to finish time including the states of tickets - Flow metrics.

Which is all fine, however you still have the original problem - unless you strictly compare very similar tasks perhaps done by similar engineers in terms of tenure, skill level and maturity, and maybe other factors, you still can't directly compare tasks across teams.

There still needs to be consideration of the make up of a team to get an understanding of skill level and experience.

Ive personally witnessed teams with graduates in them and people with 20 years experience completing very similar tasks and the difference is astonishing.

Planning poker

Step 1: We are going to estimate tasks in imaginary relative units

Step 2: We are now going to use these imaginary units to measure your velocity

Step 3: We are now going to compare your progress in imaginary units to other teams who use a different imaginary units, and complain your underperforming

Then you imaginary unit inflation and before you know everything is an XXXXXXXXXXXXXXXL t shirt size.

SV companies function as cults built on the same principles as pick-up artistry and other scams, but institutionalized and sanctified with language. Recruiters are professionally engaged in love-bombing tactics, and managers are professionally engaged in negging. It's all a game to get as much psychological leverage over people as possible. That's also why they like to get 'em young.

This is very accurate. And unlike love, there is no exiting this pick up artistry without losing income.

If you put a productivity metric in place, teams and individuals will optimise to increase the score of the metric. That’s what you want in general, but it is completely incompatible with planning poker. Why?

The same people are responsible for the estimate as the delivery. The obvious maximum solution for a team is to continually inflate all estimates to demonstrate improved velocity. At the individual level, it is to always take the most over-inflated story in order to demonstrate maximum individual contribution.

You get perverse outcomes with all metric based productivity measures (for example in retail sales having friends buy large volumes and return to another store). You can prevent this kind of thing to a degree through various sticks, but with “planning poker” it is the antithesis of the desired goals. Measure in a different way.

The main issue is when the developers inevitably find out that story points completed are how they're evaluated.

Hm, accidentally hit the submit button before the reply was fully done. Anyway, I think the story point completed, tickets done, pull requests submitted or even number of commits works somewhat well for measuring developer productivity, as long as nobody is aware of it. The least productive developers I've worked with did tend to have very few commits in comparison to more productive peers, but obviously that'd quickly change if they found out that it's a metric being watched, I see no reason for why story points wouldn't cause similar issues. Fight to get the "easy points", less collaboration etc.

If you catch the early submission within a short time frame, there is an edit button if you hadn't seen it previously.

Reply on news.ycombinator.com