Hacker News Reader

Comment by vmateixeira | Hacker News Reader

> There’s no way to measure productivity in software

This is clearly not true

Edit: It seems obvious to me that OP has a problem with his team, other than with being able to measure pros/cons of his technologically choosen path

View on HN

Its a simplification of "there's no way to objectively measure productivity in software that doesn't introduce bad moral hazards".

If you thank thats not true the industry would love to see what you've got as its one of the central problems in software development.

Should we as an industry stop trying measure productivity, or should we measure and try to manage the moral hazards? Is this a case of "perfect is the enemy of good enough?"

(edit: It seems funny to me that many other industries are fine with less than perfect information, yet somehow the software industry can derive absolutely no value from it.)

I don't think we've stopped trying to measure productivity, but when the author says "theres no way to measure productivity" he's short circuiting a whole meta conversation about trying to measure if a monorepo is better than a multirepo approach because the merits of the measurements then become what the argument is about and you end up with the same place but with a level of abstraction on top of what you are trying to determine.

Lots and lots of other industries struggle with measuring productivity. The medical profession is rife with that problem. I think what might exaggerate the problem in software is we over value the objective decision making. Its our nature to want to quantify and we don't like making subjective decisions even when that is what we are being called to do.

You've made an assertion, but not provided any evidence for it. That doesn't seem, to me, to add much to the discussion.

Would you like to back your statement up?

They don't need to back it up. It is up to the person making the original claim to back it up.

There's half a century of research and practice into this topic. Anyone in the industry _should_ know about this. Anyone who doesn't can find it on a search engine within 30 seconds.

No, it's a well known problem. The old lines of code joke still applies. You can introduce metrics, but those will just be gamed because software is a multidimensional optimization problem with complex trade offs. So, you can't translate that to a single measurement and call it productivity.

Yes, it's a well known problem.

There's half a century of research - and yet, no clear way of measuring productivity, that I am aware of.

If you're aware of one then a link to an article about it, or even a wikipedia page, would be awesome.

In software, productivity can pretty much be measured through time spent on a task, while mantaining same quality.

Before which you have to figure out how to measure quality. There are countless ways of course, but choosing your metrics is subjective and pretty much arbitrary.

I will oversimplify my response.

Let's imagine that I wrote some test cases. To me, If my piece of code passes those tests, I will consider it of being of X quality. I won't care about any other quality than this X one.

If I write/build/test two different pieces of code, both passing my tests, and happens that I took less time to write the second one of them, then I would consider I was more productive writing the latest.

I think your example shows exactly why measuring software productivity is so problematic. With your scenario, probably the easiest way to "improve productivity" would be to not write any documentation comments, heck it would probably be faster to just stick it all in one giant method.

I think many of us have known software developers who are incredibly fast, but who write "rat's nest" code that's impossible for anyone else to decipher, or which is very difficult to modify when new requirements come along. Quantifying those attributes is incredibly difficult, but as most time in software is actually spent on maintenance, it usually ends up being more important than a simpler assessment of initial output.

One can assume writing documentation and comments is covered by both approaches. Still, the fastest scenario would be the most productive one.

I agree with your view regarding maintenance cost, but I don't think that's what's at stake here. We can go further and try to compare the amount of time needed to fix rat's nest with the amount of time needed to write the software all over again. Covering the rat's nest is in many occasions faster (on which I'm considering more productive) than rewriting.

You're never going to get around the fact that most of these things are qualitative assumptions. Is your documentation good? There's really no completely objective way to measure maintainability, because maintainability is often about how easy it is to handle unknown future change requests.

Years ago "function points" were bandied about as a truly objective measure of software output and value. It turned out to be trivially game-able and added 0 value to software development estimation.

I think you really did oversimplify it. Writing a piece of code once obviously affects your speed at writing it again.

As you clearly understood, that's not what I meant.

Ok, so now you "just" need to factor in long-term maintenance.

Besides, how often do you write code twice just to see which way is better? What you have is a comparative measure that is completely useless for measuring new code, i.e. code which implements a new feature.

Where is your metric? By how much will you be declared more productive? Is it just time to achieve a result? Because that is obviously wrong. How are you accounting for quality, or even measuring that X? Do you account for readability? maintenance? documentation? factoring? over-engineering?

There are so many visible as well as hidden factors that attempting to measure any (let alone all) of that is a pipe dream.

Also, by writing the second one you are not accounting for second version effects since it is a reimplementation of something you know, and if someone else develops it for you then you just changed a major variable.

I get that you said "oversimplified", but this is just like saying "thought experiment", yet as soon as you confront that to the real world, it just doesn't realistically fly.

> Why is time for achieving a result a wrong metric?

We're paid per hour of work.

Also, on today's market, the first to release wins. I know there are some exceptions but whoever wins the market first has clear advantages.

So why is time not a good metric?

And I didn't account for second version effects, that was not my aim.

Even it you just hardcode the tests cases in a giant lookup table? Tests can prove the presence of bugs, but paying doesn't mean the software is bug free.

Also, since many tasks are literally only done once, "task" is itself a variable, not a fixed data point. As for "time", that can be quite hard to measure, since unless it's a very menial task it'll span several hours, or days even, and it's quite elusive to total what really constitutes real time spent on a given task.

I believe productivity/efficiency in software engineering is something that is felt (i.e you can measure some of its effects but many are a matter of perception, not definitive metrics. like pain.) rather than measured (i.e you can synthesize a definitive metric from a set of known, well-defined data).

I don't know if I would agree that there are countless ways to measure quality. I think the standard one is: the ability of the software to meet its design specification. Depending on your spec this could be more or less objective.

I would say that's a good baseline, but I would extend that to "the ability of the software to meet its design specification well", where real quality lies in the more subjective subtleties between minimally functional and genuinely happy users.

I find Agile velocity to be a fairly useful measure:

1. Break your workload into small-ish units of work

2. Before starting a unit, assign it a number value based on how complex the task is. 1 is absolute easiest, 2 is twice the work, etc.

3. Compare how many points of work you complete over periods of time

(Author here.) When I wrote that, I was echoing what I believed to be a widespread view. For example: https://martinfowler.com/bliki/CannotMeasureProductivity.htm...

"Productivity, of course, is something you determine by looking at the input of an activity and its output. So to measure software productivity you have to measure the output of software development - the reason we can't measure productivity is because we can't measure output."

In an earlier post, I argue that in light of the fact that we can't measure productivity, we should measure job satisfaction instead. https://redfin.engineering/measure-job-satisfaction-instead-...

I think that productivity is a tough thing to measure...yes there are metrics, but do any of them give an accurate picture of productivity? What do you think is an accurate measure of productivity?

Time would be an accurate measure, considering the same level of quality is met.

Time spent working on the wrong thing is not productive. How do you measure quality in software? If you need to measure quality in order to measure productivity, than one cannot truly measure productivity.

Or to describe this in practice: you can write prefectly modular, documented, tested and well written code that gets top marks at cyclomatic complexity checks. And yet, if you think carefully, your program can be solved by a quick shell one liner. Were you productive at all?

You don't need to measure quality, unless you want to achieve perfection. If you set your quality according to goals, you can ensure that no matter different software you write, it will need to meet same goals (same level of quality).

Reply on news.ycombinator.com