Hacker News Reader

Comment by dberg | Hacker News Reader

> For example, an individual contributor software engineer s competency matrix may include a row for coding/feature output velocity. In a Level 1 to Level 5 system, the matrix would specify that for a Level 1 engineer, expectations on code velocity are X pull requests per week

I mean. No.

View on HN

It really is a ridiculous metric. So ridiculous that you'd think it would never be used in reality... and yet, I have in the past actually had the CTO of a large engineering firm insist upon it.

I was just thinking the other day about one of the issues I have working in this industry. You are expected to solve some hard problems, within a couple of days, usually. If you fail to do this satisfactorily, people will wonder what you're doing; if you're smart enough. And if you can't solve those problems fast enough, the "right" way, you are let go for not "performing."

In reality, you don't run into these kinds of problems often. Usually, the problems we're solving are relatively straightforward. However, if you accidentally pick up too many of these problems in a short span of time ... you can be screwed.

The thing is, for a junior engineer, this is not that bad of an idea. I don't think that it should be part of a formal performance process, but if you're green and aren't putting out enough PRs, it suggests you're either struggling to contribute, or putting out PRs that are far too large and need to be broken up to be easier to review & safer to deploy.

Think of it as a threshold. A high number doesn't suggest you're great. But a low number can often be a symptom of a greater problem.

I assume this suggestion only applies to big orgs where juniors can get a steady stream of small, well-scoped items to tackle.

At a lot of smaller orgs I assume it's harder to do this? Where I've worked, juniors often receive exploratory items that are lower-priority, and off the critical path, but they're expected to have reduced output, as well as asking for guidance as needed from more senior members.

Perhaps this is a failure of planning (which would have been my own failing as a lead at times), as well as management

> steady stream of small, well-scoped items to tackle

Yes. The part of the quote parent left out is

> or the ability to close Y story points per sprint

With that in mind, it makes sense and the metric is dependent on the entire org functioning well. So to the extent it does so, this is an objective, very quantifiable, metric that all junior engineers can be compared with.

That said, this is a guide for startups. So I do believe this metric is garbaj. Junior engineers will be unfairly held to task for org dysfunction.

I mean, if you want the # of PRs to skyrocket to an impossible to manage level, that's great advice!

What are you quoting? I don't see that sentence anywhere in parent's linked twitter thread

“If you can't measure it, you can't manage it”,do you have better ideas that also work in daily operations and practical? is there a way at all to measure software engineer or no?

"If you can't measure it, you can't manage it" is absolutely terrible leadership advice. Or to phrase it another way, if you could magically transport yourself to work on any team in history, which would be and why? And how did those teams run? How did the leadership function? How did the team think of its own performance? In this regard, I feel history is a much better guide than middle management claptrap admonishing us to all kneel before the false god of Measure Everything.

I don't buy it fully either, but, I could not figure out what's a better alternative, not in theory, but in practice.

Software projects are notoriously hard to manage indeed, but it has to be managed somehow, Agile approach is a nice try, we had CMMI as well.

Even a neurosurgeon can be measured, why can't software developers, why are software engineers so special? Maybe AI programmer is the way out.

So, interesting example: how much do you know about surgery? The best surgeons care about things that are not, in fact measured -- things that amount to the quality of the craft that very much correlate to patient outcomes, but are also hard to measure. Indeed, if you really get a surgeon going, they will complain about the things that are measured -- namely, the number of surgeries that they perform. I know of more than one surgeon who has left their craft because this quantification was essentially forcing them to perform unnecessary procedures!

In general, quantification of human endeavor is fraught with peril -- even in those domains where it would seem to be entirely non-controversial like professional athletics.

The alternative to measuring everything seems obvious. Don’t measure everything.

Why not measure everything? Because measurement has a cost. And some things cost more to measure than you gain from the measurement.

I was on a team once with a product owner who insisted on absolutely everything being in jira with story point estimations. Well, that’s fine and good but one day I sat down with the designer and walked through the product. We made a list of about 30 things that needed fixing - almost all of them trivial. “This is the wrong shade of blue”. “There is too much spacing here”, etc. Well, I made a list in notepad and got going but I got in trouble for not putting it in jira. Only, it would have taken longer to write up these issues than it would have taken to fix them. To say nothing of wasting everyone’s time doing story point estimation. In this case, measuring our work would have cost significantly more than doing the work! We argued back and forth and eventually he admitted he wanted things in jira to appease upper management, who I suppose wanted to know how busy we were by looking at a number on a spreadsheet.

(He ended up writing up all my points in jira himself, guessing costs and ticking them all off as “done”. What a waste of time.)

And no, software isn’t special. Teaching isn’t measured (except maybe with a student survey at the end of the year). Science is measured in citations per lifetime, not micropapers/hr. And you don’t quantitatively measure your spouse, your friends or your enjoyment of mum’s cooking.

Measurement is a lovely tool. But don’t make a religion out of it. I heard a story from a leadership summit recently. A young CEO got up and said “But there’s no way I’d make hiring and firing decisions by gut instinct!”. A bunch of the older people there disagreed immediately, and said that’s exactly what you should do. Train your instincts then trust them to do their job.

> But there’s no way I’d make ... firing decisions by gut instinct

Well the reason it's so important to hone your gut instinct for good hiring decisions is that you absolutely should have measurements for firing decisions (at least in the US); that decision needs be defensible with data under the scrutiny of a lawsuit

You don't need a reason to fire someone in the most of the US (AFAIK). You can just say that they walked in with a red shirt and you don't like red shirts. As long as the reasoning isn't because they are in a protected class, you can pick whatever reason you want.

Firing someone "because they walked in with a red shirt" would do damage to your company's reputation, and call into question the competence of your management team.

And you can still get sued.

I mean. You can get sued for wearing a red shirt. It would probably get tossed out in ten seconds (unless the judge just wants some entertainment). But you can sue anyone for anything. You can do tens of millions of dollars to a company, get fired for it, and sue them. Nothing is stopping you.

Secondly, no one is going to give a shit why you fired someone except the person you fired (if you even told them why), and HR. In fact, most companies don’t even give managers firing authority, but if you are the owner/CEO or someone who does have that authority, it’s possible even HR doesn’t know the reason. That person is just fired.

not interested in measuring everything, neither do I believe 'measuring nothing because I'm a software developer'

Agile has its merits, abuse it is obviously wrong.

There got be something that is practical and get the job done and benefit all parties in software project management, I'm searching for answers myself.

Measuring software engineers productivity has been a meme and the butt of jokes for decades.

However, I think there are things worth measuring that do help manage success and are not intrusive.

For example, giant pull requests or pull requests that go into production with no/minimal review are likely to create problems. We use LinearB to point this sort of thing out automatically.

Code with high cognitive complexity is likely to generate bugs (and be hard to maintain). There are many other similar issues that can be automatically detected. We use SonarCloud for this.

Asking an engineer "How do you feel about work overall", "your personal well being", "career growth", "work relationships", and "impact & productivity" on a scale of horrible to outstanding once a week prior to 1:1 meetings allows each engineer to tell you their story in numbers. If you have built a culture where people trust that you care about them, you will get honest answers. This helps discuss and correct things that bother your team members, and in aggregate shows whether you have a problem with leadership and/or culture that needs to be corrected.

Those are the numbers I gather, and it takes a lot of the gut feel and guesswork out of management.

It's incredibly difficult to measure the work of a software engineer with metrics which are useful AND objective. You can have many metrics which are useful, but not objective. And you can have many metrics which are objective, but not useful (and often detrimental, like "number of merge requests" per week).

Reply on news.ycombinator.com