Hacker News Reader

Comment by ryanwaggoner | Hacker News Reader

Agreed, and I think we all know this, but none of the substitute questions he proposes approach anything resembling an objective measure.

View on HN

I would say, the only valid objective measure is results.

That, interestingly, is what a couple of psychologists tried and succeeded in implementing at Best Buy, of all places. See:

http://en.wikipedia.org/wiki/ROWE

They seem to have been very happy with the results they got from that.

If you want to evaluate results, why measure hours?

What does "results" mean in knowledge work? It's impossible to know whether the results could have been achieved faster or better if you had indeed measured hours. I'm not saying measuring hours is the only thing to watch, but you can't simply track a undefined concept like "results".

If it's your company, you know what results are (you'd better!). No, you can't track that, but I'm not sure it's meanful to compare startup progress.

You mentioned further down that you're a manager.. results are whatever you've agreed with your team is the "deliverable".

So you might agree on having a set of bugs fixed, for instance, or agree on getting a piece of functionality implemented. I'm not an expert on ROWE, but as I understand it, the key is that you agree on results which you (as a manager) are satisfied with and which the team (who will produce those results) is satisfied with.

Then, you don't care about how many hours they work. At the agreed time, you expect the result that you agreed on. If it's there, no problem, move on to the next bit of work that needs to happen. If it's not there, then you can sit down with your team and figure out why (but avoid the all-too-easy accusation that "they didn't spend enough time on it").

Apparently this technique (along with a set of workplace practices that actively discouraged managers from trying to figure out whether their teams were actually working or doing something else) worked really well for Best Buy.

I guess the difficult bit is when "results" are something like "build me a new web site". Taking on clients like those on a fixed-bid basis is a recipe for disaster.

Since with engineers you rely on them estimate how long things will take, and the last thing you want to do is keep nagging them to decrease that time, it's easy for engineers in a non-startup company to lose the sense of "urgency" and get complacent with the estimates, adding buffers to make sure they are never late. How would you try to keep ever increasing estimates?

Ultimately, you have to work with people you can trust. If they trust that you won't suddenly spring some bad surprise on them, and if you trust that they won't artificially pad their estimates and slack off (which is very unlikely unless you're working with some really bad apples), there shouldn't be a problem of padded estimates.

That's one of the things that agile development should ideally give you (in practice, of course, it's never as easy as it sounds). You make short-term (i.e. per-iteration, often 1-2 weeks) commitments to get things done, and base the amount you commit to on the amount you got done in the past in that amount of time. Other than that, leave the developers alone and let them get their work done. That gives developers a reasonable target that they can be held accountable for, which tends to make everyone happier.

Longer-term commitments (we'll release in 6 months with features X, Y, and Z) on software projects tend to be impossible to make accurately or to live up to, so as an engineer it's impossible to really take them seriously or to be held accountable to them.

By making the commitments much smaller and near-term, they're much more reasonable and doable and much more likely to be taken seriously by all parties.

"If you want to evaluate results, why measure hours?"

I dunno-- saying that you shouldn't measure time spent as long as people get their deliverables done feels like saying that you don't need to track where you spend your money as long as your business is making a profit. Time is a really valuable resource.

Most people suck at time management, and should get better. Most managers have no clue how much (or how little) their team works or how workplace variables (like office layout and team composition) change how people spend their time. Most managers aren't good at guessing what an appropriate list of deliverables for a given full-time employee. If they try to dole out equal workloads to two developers, what happens when the manager screws up on their time estimates? Renegotiation mid-week? How does Dev2 feel when Dev1 gets to go home early? Who gets the bugs/tasks that have risk associated with them? Does SeniorDev get harder/riskier tasks than JuniorDev? Does he get more tasks?

The ROWE concept reminds me of fixed-price bids from my consulting days. To successfully give a fixed price bid with the least risk, you have to do MUCH more careful cost analysis. With a ROWE system, you have to have a pretty perfect handle on the "cost" of the bugs/tasks you pass around (which, with most knowledge work, is nigh impossible).

Startups kick ass because (among other thing) the founding team really cares about what they're doing-- I think it's pretty rare for a founder to say on a Wednesday, "Well, crap-- I finished all of the stuff we said I'd do this week. I'm off to the slopes!". Instead, they say, "I'm on a roll and there is an endless to-do list. Time to hit the next item!".

Don't get me wrong-- I think the punchclock mentality is ridiculous. But I think we're throwing the baby out with the bathwater-- You should measure BOTH.

I have literally gotten as much done in 4 hours as anther teammate did in 2 weeks. (Fixed the same bug.) However, usually we don't have direct comparisons.

I wonder if breaking a team down into 2 sub teams that compete over 2 week periods to fix the same bugs might be an interesting solution. I suspect most people would be more productive, but less than twice as productive. However, you would then be able to directly pick the "best" output which could be vary valuable. Add in the ability to discover and drop unproductive people and you might have a net win.

Edit: Ok, you might also create a hostile, environment that is so stressful you force the most productive people to leave.

Your experiment of creating 2 teams would be an interesting one. But I think one team should be given 2 weeks and the other team should be given 1 (or less), and they cannot know of the other team or that they're trying to solve the same problem.

If teams are in direct competition with each other, you get that sibling rivalry effect that almost tore Apple apart (Machintosh vs II).

This type of experiment was done by the Peopleware authors. They are the source of many of the productivity numbers you see quoted for software developers.

Yes. The concept of the "10x developer" is one of those examples that describe the variance in productivity among software developers

http://forums.construx.com/blogs/stevemcc/archive/2008/03/27...

I think this is partly the point of the article: when you're forced to rely on objective metrics, you will avoid some of the questions that distinguish good performance from great performance. That's one advantage of tiny companies, which don't have to rely on such metrics because everyone takes the company's overall performance as their personal responsibility.

How about milestones achieved? Anything but hours. I did the startup thing (worked at a successful startup) 10 years ago, and we did work long days -- but spent most of that time creating things from whole cloth that exist as commodities today. (Everything from fast search libraries to infrastructure tools.)

I don't believe that most founders need to work 12-hour days. Not anymore. It's not the hours -- it's your progress that counts.

Milestones alone are often abused, too.

In startup work, people are invested and there's too little time and even less reward to compare what I'm dong to what someone else isn't. If someone doesn't deliver, someone else steps up and we move on.

In mature organizations, everyone is not running to the copier, and people begin to take notice of the terrain. Politics, unpredictability, and general people challenges are perceived as a drag on the system. As a defense mechanism, especially if incentives are involved, milestones are defined not for the common good, but to route around points of potential failure.

Milestones make it really easy for the offense to blame special teams and the defense to blame the coach after the team lost, even though they all celebrated success at different times during the game.

I agree, progress and outcome is what counts. And I think iteration and feedback loops are essential: yesterday's unrefined solution isn't the best answer tomorrow.

Reply on news.ycombinator.com