Hacker News Reader

Hacker News

Comment by JJMcJ | Hacker News Reader

Parent Full thread

JJMcJ

How it often works:

Manager says I want an A that does B? How long?

Most of the time estimate is N days, actual is N +/- 20%.

Sometimes it's N days, and you get lucky it's N/3. Instead of credit, you now have a reputation of padding estimates.

Sometimes it's N days, and you go oops, and it's 10*N. Now you are a slow incompetent.

This is why estimation is such a no-win for developers.

For areas that come closer like construction, they have centuries of prior experience and often the contractors are big enough to push back.

To use the example of this article, go to a boatyard, and say I want to 150 foot yacht with a hot tub and a 20 seat theater, and my budget is $10,000. You would get laughed out of the office.

View on HN

> "you now have a reputation of padding estimates."

> "Now you are a slow incompetent."

This is also symptomatic of organizations that behave adversarially and assume your intentions to be malicious. In that dynamic, even if you perfectly estimated things, you would still get in trouble for delivering what was agreed on instead of what was "meant to be agreed"...

Usually this is more the case in orgs who view software engineering purely as a cost center, instead of as a business differentiator. When considering a position at a company, this is a good thing to pay attention to.

> orgs who view software engineering purely as a cost center

This is spot on! I've heard this phrase before and it's never made any sense to me. Any business concerned about profit should never have a "cost" center. If that unit is not returning a positive ROI it should not exist. Of course software development costs money just like all other labor, building and materials but it's assumed the value generated by the work product is worth the investment.

Cost centers are generally unavoidable, they're aspects of the business where there is a well-defined target past which there's no meaningful ROI.

Regulatory compliance is a good example of this. Both you and your competitors must be compliant, but past reaching that state there's no additional value add. As a business, you're thus interested in keeping the cost of compliance as close to zero as possible, all cost cutting that does not break compliance is good for the company.

Keeping the cost of compliance small isn't the same as doing as little work into compliance as possible. More often than not, it's the other way around.

Corner cutting doesn't work on any of the usual "cost centers", and most of them can be transformed into a differential.

But even your example can be viewed as an unavoidable cost of doing business where the alternative is to not be in business. In other words there's a distinction to be made between regulatory compliance where you have no choice and automating payroll processing where you do have a choice. I think it's pretty clear we're talking about the latter and I would not categorize an internal software development team who's job is to make the business run more efficiently as a cost center, even though there's not external sales of their work products.

All too often "cost center" is used to mean "anything except sales and marketing".

If tech is seen as a business differentiator, wouldn't the opposite be true? It's so important that they want estimates so they can deliver more value. If it's a cost center, whatever... we don't care that much.

I think my point is, the cost center vs business differentiator is not a good indicator of the root cause of this problem.

If it's a business differentiator, there's usually a better understanding from upper management that these things are hard, that there's trial-and-error, that some of this is R&D which will not have immediate value but strengthen the long-term position (or buy a patent moat)... If not, then that business is probably about to be in trouble.

> Usually this is more the case in orgs who view software engineering purely as a cost center

In my experience, this is more of a cultural issue, with it being pushed by engineering managers and tech leads who insist that everyone can be a 10x engineer if they just throw away their life.

When they’re past a deadline, it doesn’t matter why, because they were clearly just being “lazy”.

Estimates truly are a lose-lose for software engineers and they have probably caused me more grief than any other aspect of this field.

Or almost as bad, you are a nay-sayer. Why are you so negative?

Asks the person who punishes people for being wrong.

Yeah, if you don't trust the people that you hire to fight in the trenches, get the f--k out of this business.

And I think the iterated version of the game is worse. Once you give estimate N₁, they don't like the answer and ask again. Maybe they use different words, or ask different people, or make small changes. Maybe they just imply it seems high. So now we have N₂ < N₁. Repeat until the manager is happy with Nₓ.

Nₓ is of course is about 1 day past the "absolutely impossible" time. So now we have gone from 50% likely to hit the date to 5% or less. So at best we get yelled at for being "late" to a number that was about what the manager wanted, not reality. Or worse, we finish the visible work on the day of, while leaving a lot of hidden work (e.g., technical debt) for the future, increasing the size and volatility of future estimates.

And that's not even mentioning that all of the conditions of the original estimate (e.g., scope won't change) are breezily ignored even when the date is remembered.

It's a stupid game, and I've stopped playing it.

> It's a stupid game, and I've stopped playing it.

Care to share the secret of how you did that? :)

Basically instead of giving them a sense of control, it's to give them actual control. What I promise to do is release a new version of whatever they want at least once a week, and hopefully more often. If every week (or every day) they get to decide what that is, they stop caring about estimates. More here: https://news.ycombinator.com/item?id=21071374

And I also wrote something up in response to a previous HN discussion: http://williampietri.com/writing/2014/simple-planning-for-st...

An observation here is that a house or a boat is something that people can see; we've all seen a building go under construction whether it a single family home or an office building. It takes time, and we know you can't rush the process or risk the building being unstable.

With software, many parts of the business can't "see" the development happening, so they don't know why it would take so long. At the same time, unlike a boat or building, software is more likely to be easily modified. Therefore there is less of a risk to get it all right initially. I do not condone said behavior I am merely pointing out a possible scenario in which it could happen.

It is also true that a developer is more likely to be a push over than a general contractor. Source: 11 years as as software engineer and my family has been involved with land and real estate for a long time.

I’ve found it really helpful to communicate and train funders to learn how to watch software being built. It takes time to teach someone to look at builds and tests and growing code and activity and demo releases. It also takes time to teach developers not to game this.

It’s amazing how much this helps build trust. It’s hard when there truly are big giant components that take a long time between commits and builds. But I think those are much rarer than programmers think and say.

>It also takes time to teach developers not to game this.

That's a massive ask unless management stops using their old style of management immediately while learning instead of slowly transitioning. There is zero reason for employees to stop protecting themselves from their incompetent managers. This is especially true in software where the average job length is much shorter, so you get less of a benefit from investing time into helping train a boss

The reason is trust. That’s nonzero but really hard to establish.

I’ve seen projects with untrustworthy devs and untrustworthy managers. Oddly sometimes that worked. But I’m too old to work in hellish environments for pay.

Sometimes the manager just doesn't believe you at all.

Learned that the hard way on my first lead role.

Him: How long will this work take? Me: A year with this team. Him: A year? How is that possible? I was thinking six months. <starts going down the entire list line item by line item dickering on every little thing> Me: <after 15 minutes completely checked out of this conversation.>

When you don't have time to do something right you have time to do it over. It ended up taking us 18 months. To this day I firmly believe that because we tried to do it in half the time, it ended up taking us 50% longer. Accounting for our tendency to procrastinate, I believe if we'd gone in saying 12 it would have taken 14 at the outside.

> Sometimes it's N days, and you get lucky it's N/3. Instead of credit, you now have a reputation of padding estimates.

Or you just have a reputation of not having the foresight to pad the release based on your estimate.

I agree with everything except after doing this for 20+ years, I think of my estimate and then multiply by 3. I've been much better at hitting my estimates since doing that (but I still miss 75% of the time).

Yeah for a basic rule of thumb, I ask myself:

How long should it take?

Then I take that number and I ask myself:

How long will it actually take?

Then I double that number and add 30%.

If you are not embarrassed by your estimate it’s probably too low :-)

If you double a number and add 30% on top, you could just add 160% to start with, or multiply by 2.6. Since we are talking estimates, you would be similarly well served with just 3x :)

Yes, you're correct. I'm also, like many of the people on HN, familiar with the associative property of multiplication. It's just a humorous way of saying that taking your estimate and doubling it isn't gonna be enough.

I apologize for not seeing the joke: humour is a strange beast. :)

As a former developer, and as a manager of development teams, I feel that engineers tend to underestimate how long a task or a project will take them. This is due to several factors, including good developers having big egos about their capabilities, and with them focusing only on the straightforward solution, without giving credit to the inevitable (and expensive) edge cases.

My formula for getting a better estimate from engineers (call it "padding" if you wish), is to take their estimate, double it, and then round up to the next largest time unit.

For example, if they say a task will take 3 hours, I double it to 6 hours, then round up to a full day. If they estimate it at 2 days, I double it to 4 days, then round up to a full week.

This sounds like a joke, but I've found that in the real world, these types of estimates are closer to what ends up happening.

Agreed. The problem is that software really falls into 2 main categories:

1) doing something that has already been done in a very similar way. This type of software is relatively predictable and estimates are generally reliable, assuming that is being done by the same team that worked on the prior implementations.

2) doing something new where NO ONE actually knows what the specification really is or how hard it will be to implement. Most of the time, people don't even know who all the constituencies that will have opinions or requirements to contribute to the specification. Estimating this type of work is largely a guess in the dark, and there are no reliable ways of producing valid estimates, so it is all based on prior interactions with the group and experience of the senior developers with this type of project.

Most managers think they are in camp 1), but in practice if that was true they'd just buy off the shelf software. You're doing your own because something is different.

When's the last time a construction job was done on time? Or even within +/- 20%?

Happens every day all over the world. It's just such a mundane event no one writes newspaper articles about it.

Well, I've never heard about it.

Both of my parents worked their entire lives on the construction industry. It's a running joke that this never happens.

Software people tend to assume that the rest of the world is far more organized than it actually is. In particular, they seem to think that construction projects tend to be well-oiled machines with perfectly followed specs written in advance.

I work in civil engineering so I'm fully aware of what monstrous cluster fucks some projects turn into. Doesn't change the fact that the majority of projects (by number) are simple, small, unambitious projects that basically go as expected.

So, just like software projects.

It happens when people really care. I think there’s a lot of corruption and incompetence in construction so unnecessary overages are tolerated.

Atlanta rebuilt its 8 lane highway segment early [0].

[0] https://www.ajc.com/news/local/bridge-fast-construction-safe...

After Northridge earthquake in Los Angeles area, many freeway interchanges were rebuilt within months. But there was extreme urgency here.

General Leslie Groves was chosen to administer the Manhattan Project because he got the Pentagon built on time and on budget, and was viewed as someone of superior skill at these things.

Here in Germany we often jokingly say that the Berlin Wall was the last big public construction project that was completed within time and budget.

;-)

I suppose a good solution would be to evaluate the person on the accuracy of the prediction :)

Of course that's a nice dream in a management driven environment.

Reply on news.ycombinator.com