Hacker News Reader

> At one of my employers we had a very large and highly skilled dev community and being quant-biased, we tracked code checkins over time, number of tests written, number of pushes to production etc.

Why are those objectively useful metrics for productivity though? I can think of a dozen reasons why they aren't, as opposed to why they might be. Which, I guess, makes their objective usefulness subjective.

Some people will be working on longer lived feature branches because they're more complex, so they end up with one merge to master in the time it took the dev updating REST APIs to accommodate a new optional parameter to merge four branches.

Someone writes four tests that another dev might cover in one because the four assertions were all closely related to the same invariants and code paths. Who was actually more productive?

One developer pushes to production three times in one day because they typoed a property name twice... ...etc. etc. ad nauseum.

If you seriously believe that these metrics are objective methods of measuring developer productivity, I strongly suggest you spin off a business, because you're about to make millions.

It seems to me that while any one of those metrics isn’t particularly useful in isolation, if the same person is consistently at the top of all of them, that’s telling you something.

Maybe. but what if their peer has far fewer commits to their name, far fewer written unit tests attributable to them, because they spend most of their time using their indepth domain knowledge to assist other developers, who then make commits based on that contribution?

Do they have to sit around and ensure that their coworker includes a "Co-Authored-By" in the commit message? One commit message? All commit messsages? etc. etc.

This guy was also top in terms of the number of FAQ entries written to help people. In fact he wrote the FAQ system itself and also wrote a right-click plugin so if you rightclicked on a question in chat it would auto-search the FAQ.

See now that might be a better metric.

_What_ it tells you, however, depends on whether or not the combined metrics are used to evaluate people (or more accurately, whether people _think_ they're being evaluated by those metrics). Goodhart's Law[1], ruining everything since 1975.

[1] https://www.lesswrong.com/posts/YtvZxRpZjcFNwJecS/the-import...

Number of check-ins, if the check-ins are high quality as he implied, are a pretty good rule of thumb to measure the productivity. The guy working on the most complex feature with only one merge to master should check-in even more often than the guys working on the simple features.

I don't understand that logic - the complex piece might require tons of research, experimenting with different approaches, benchmarking etc, before getting to something committable. And the successful result might only be a small number of lines.

I worked on project where I had dozens of commits every day. Now I work on project where I have one commit per few days. Sometimes I need to research for a week just to create one small commit.

And I am still the same person - only my position changed.

What is the point of checking in a horrendously incomplete feature that nobody else is going to work on, that probably cannot be tested by anyone until it's actually mostly done?

I'm working on such a thing, and I will check in ~100k lines of code when it is the right time to do so. That is not now.

In a CI/CD setup, you could make lots of atomic commits growing the feature behind a flag.

A side effect of atomic commits is that number of commits does better correlate to "work done", I suppose.

You could, but does that really solve any real problem? Now you're wasting more time on CI, commit messages, and trying to keep them atomic instead of delivering the feature. It's not even clear what makes up an atomic commit. If the feature does not work, it's arguably not very atomic. Stubs, mocks of parts that need to be mocked to push forward now and redone later properly, unimplemented branches that end up in abort(), these are arguably not very useful history either.

Also, I disagree that number of commits is a useful measure of work done. Not any more useful than butt in seat time. Nobody can say whether that feature is 200 or 2000 commits, it really depends on how you split it up. It could be 2 commits, plus the fixes that follow once it actually runs and gets tested and fleshed out.

If tracking progress is a priority, you break the job down to some reasonably small number (no more than a couple dozen) of items in a TODO list and update it with what's done and rough estimates of what's left to do. That's a hundred times more useful than "N commits, 123 hours of butt in seat."

Or maybe you can just demonstrate what works. Or tell, when it comes up in a meeting.

(I'd argue that butt in seat time is actually more useful than number of commits, because the latter can change by an order of magnitude depending on how you split it up but I've never estimated to take a month to deliver something that ends up taking 10 months.)

This is probably not the place to debate the pros and cons of CI/CD in general, but if a team is already doing it, isn't the list of atomic commits effectively the TODO list you're talking about?

Of course, over-engineered solutions will score you more commits (or more lines of code, or more TODO items, etc) but still be a bad outcome.

If you don't know how many commits it will be before you start, no, it is not a TODO list. I've never worked with such perfect knowledge. Usually, there's a high level understanding of what it takes to deliver a feature, which allows for the creation of a coarse TODO list that works for tracking progress & coordination. And then there's the devil, in the details. One commit is not one feature, and there are too many details for an a priori list of all the commits you're going to make. And very often you end up doing a bit of opportunistic refactoring & cleanups & little fixes to related code when fitting the new stuff in.

Even very small and simple features can end up being anywhere from 1 to 15 commits.

Yes, you're right; it's not a TODO list because it's not known ahead of time. Therefore, it can't be used to track progress on the particular project itself.

This all came up in the context of measuring productivity, though, not measuring progress on a specific project. Looking backward, it is a list of items completed. If the teams strives to keep them atomic in general, they are more likely (though by no means guaranteed) to be closer in size. So my guess is that this would correlate better to productivity than lines-of-code or time-in-seat, while still falling well short of the ideal.

By check-in, do you and the OP mean commit? I had interpreted it as a feature branch being merged to master - in the days of Git über alles, you can commit like crazy.

No developer metrics are great, especially not if devs are incentivized to game them. However, I wouldn't be surprised at all if there's a correlation between good developers and those metrics.

If you incentivize the metrics, I would be surprised if the correlation held.

Well I suppose the correlation would still hold for the good developers but it would then also hold for the not as good developers. Over time maybe the bad developers would show a higher correlation than the good developers, but maybe not.

> in fact a lot of what he wrote was very simple when you looked at it (and I think that's part of what made him so good

Genius is when you write a piece of code that's so simple everyone else thinks they could have done it. And yet they didn't.

Presumably someone, or some group, was writing all the requirements and acceptance criteria necessary for him to code all day, and they were managing the changes to those criteria as users gave feedback on what he was doing.

And then there were the other devs peer reviewing his code, happily accepting his PRs without needing to add comments or request changes.

And then there was the QA team doing the integration and end-to-end tests, user testing, training, etc.

He couldn't have been quite so productive without those all those people.

That's not to put his amazing work down. No doubt he was brilliant. It's just that we shouldn't ignore the efforts of the people who enable developers like him to get on with what they're good at. Development is a team sport.

You’d be surprised. Sometimes the level of support from others is basically zero but you still make it happen. That’s the real sign of 10x

I know the type

Used to be like that when I was young

I also know that there's the guy that never writes tests, is incapable of writing non reduntant-code, pushes everything to master as a blob of unrelated changes, using git as the backup server, comments are sparse and mostly useless, code is a mess and 'deploy to production' means starting a Jupiter notebook on the Dev machine.

The same guy saves to the company millions in expenses and gets the promotion.

10-20-100x Devs are just very good Devs, nothing that automatically changes the fate of a company

You can have the best guitar player in the World and your band goes nowhere or you could be the Ramones...

How did it work out practice? After each Sprint the numbers got revealed?

Did that not lead to a super competitive workplace? I cannot imagine that this was very motivating to the remaining 990 devs... Or was it?

The only relevant question is whether that person has a life outside programming and work. If yes, then it's truly a case of remarkable talent. If not, it's a case of sacrificing something in exchange for something else.

Moderation is not for everyone.

The world would be a much less vibrant place if not for the obsessives among us who devote themselves religiously to their craft. Whether that be programming, writing, art, or otherwise.

I think blub was just trying to distinguish whether the person delivered more value per hour, or just more hours. The outcome may be the same, but the former sounds more like "10x" than the latter.

If you have someone who is so obsessed with programming that they have time for little else, the lack of social contact has the potential to turn that person into a problem later on. Case in point: Richard Stallman.

Inappropriate behavior may have been well tolerated in the patriarchal past, but in 2020 we are expected to do better and our institutions can ill tolerate the risks presented by asocial white men.

I call BS, that's not even possible. Even the legendary "tourist" cannot do it, in his own words. He programs like 2-4 hours on good days.

What you are describing here is a code monkey. Did he had any opinions or took any design decisions or he was just paid to simplify things?