Hacker News Reader

Comment by 13hours | Hacker News Reader

Almost at the end : "Yes, all this sounds like a lot of work". Exactly. And it will most probably still turn out to be wrong, because you will forget to ask some questions, and there will be things that you simply cannot foresee that will influence the time.

Turns out it's pointless to even try to do this. So why waste valuable time to do all this, if you know it's not the truth anyway? The answer is to accept that you will learn things as you go along, and that things take as long as they take, and to rather deal with that. Marketing wants a product by Christmas. Fine, we can do that. They just cannot say what needs to be in that product by Christmas. They can have whatever the last production quality release is at that stage. Instead of taking many weeks or months to try and answer all these estimation questions, we start and tackle the high risk items first. We learn from them, readjust, and as we get closer to the deadline, we keep communicating with all the other stakeholders what we think will be delivered on that date. It's a narrowing cone of uncertainty. One sprint before the deadline we'll be pretty accurate in predicting what it will be. 2 or 3 sprints before, accurate enough to be able to do marketing. At the start : the best we can predict is simply whether we feel we can have something useful at the deadline.

View on HN

"So why waste valuable time to do all this, if you know it's not the truth anyway ? The answer is to accept that you will learn things as you go along, and that things take as long as they take, and to rather deal with that."

This maybe is applicable for a company developing new products internally, but for projects delivered to third parties, delivery dates are part of contractual obligations.

Estimates are hard, but it's not like predicting the future with a crystal ball. Good developers in general get better as they gain experience in the domain, and in estimating itself.

Good project managers in general get better at spotting murky requirements that might be causing problems down the line, and try to clarify them early with the customer.

It's not a perfect science either, but it's a necessity in many real-world scenarios.

Don't put hard commitments into the contracts then. Why slave yourself to something that you aren't sure you can deliver?

Good developers should ideally only ever do the same thing once, which means that there is very limited utility in trying to improve estimates for a task that you should never perform again.

The secret of a consultant's work is not to craft software, it is to craft contracts.

Evidence : CGI never got burned by their fiasco on delivering Healthcare.gov

>Don't put hard commitments into the contracts then. Why slave yourself to something that you aren't sure you can deliver?

Because you know full well that the contractual time limits will not, in fact, ever be relevant as you've added some neat tricks so as soon as the spec changes you get to adjust them (and the spec always changes). One assumes therefore that the commitments are simply part of the bidding process as you try to craft a somewhat realistic but very attractive looking bid

In general, this is a terrible practice.

"How much is it going to cost to build my house?"

"Er, I'd rather not commit to a price. We'll figure it out later."

Software development is very different from building physical structures.

"How much is it going to cost to build my house?"

"Give me the blueprints and tell me your finishes."

[Waits on client to provide specs...]

What's the difference?

"How much is it going to cost to build this application?"

"Give me your specifications for the app."

[Waits on client to provide specs...]

I'd guess the usage of [Waits on client to provide specs...] in the software scenario is a bit facetious.

Often clients can't produce specs sufficient to build an application. Some clients may produce 'specs' which are incomprehensible, making them impossible to estimate.

The crude part is, that such projects may still go on, and worse, the client may believe that the application is actually being built to the specs.

Often clients can't produce specs sufficient to build a house/building/pipeline. That's why there is an entire category of firms that exist that are hired by clients to produce these documents.

Guy wants an apartment building, he hires an AE firm and tells them "I want an apartment building" and they say "well what kind of apartment building?" and then goes through the exact same process of defining scope, budget, and schedule that would be done in any other project management field. Lawyers get hired when someone wants to sue someone - "well what do you want to sue them for?" Sourcing managers get hired when someone wants to make something - "well what do we need to buy?"

The fact that the software engineering industry refuses to acknowledge that their industry is not special is a constant source of confusion for anybody who has been doing this in another industry. Why is the software industry special? What makes software so nebulous that scopes cannot be defined and estimates made? The argument that "people don't understand the impact of requests" does not hold water - people don't understand that simply wanting more room in a particular part of their house can cause an entire redesign. That's why you, as the expert, have to explain the impact of their requests clearly rather than just agree to them.

The problem with estimates and scheduling in software engineering isn't a result of work, it is a result of failure by the project management.

You can ask an architect or a builder how much it is going to cost to build your house. They will ask for blueprints and finishes. If the person asking doesn't have specs then the builder or architect will often offer to help them create them... For a price. Most people that want software developed want to know how much it will cost to develop software without having to invest time or money in first developing the specs. The issue is that people who what software developed either don't know too get or don't want to pay for blueprints, then they complain when things go off the rails whether cost, delivery date, or quality... They fail at managing risks themselves (by not minimally getting specs first).

"The problem with estimates and scheduling in software engineering isn't a result of work, it is a result of failure by the project management."

The problem with estimates in software engineering is that few want to pay for them. (Which hinders the formation and demand for the category of firms to produce specs.)

That is not the only alternative.

"$300/hr + expenses; hours and expenses determined by necessity to complete work; work as described represents at least 100 hours effort and $x in expenses"

> as they gain experience in the domain

Because "gain experience" means that there are no more "unknown unknowns" for them, anymore.

But, unfortunately that is a luxury you can't afford for too long. Reality has the nasty habit of changing, shit happens and good developers need to get replaced as they burn out and run away from development into more sane activities.

Contractually obligating pi to be 3, the Earth to be flat, or software features and ship dates to be specified, doesn't make them so.

All you've done is restate the problem as an error on the part of the negotiating process. Not a problem to be dumped on developers.

When you buy something from Amazon Prime and it gets there three weeks later, I hope you are as amused by the "negotiating process."

(Yes, I know, the product you build is different than manufacturing, different than construction, different than paperwork, and every other deliverable.)

Commoditised manufacture, inventory management, and shipping are pretty much precisely the opposite of software development.

Or: your entirely counterfactual counterexample rather neatly proves my point.

Software development is commoditised. Or at least a lot is. Static web sites, simple mobiles apps, etc. Some individuals or organizations build them over and over with regularity.

But there's a difference between "production ready" and "customer ready" which Agile practitioners sometimes seem to forget.

In many businesses, software is not the core business, you're not building software for yourself, you're building it for other stakeholders. It's not acceptable to say "Well it only does 20% of what you wanted".

What happens when things start to slip, so in fact you won't have something the business thinks it can present at christmas? "Don't worry, we'll just keep working on it until it's ready" after having spent 6 months on it isn't good enough.

It's not a bad thing to do a lot of estimating up front so the business can make an informed decision about whether it really wants to commit to the effort, rather than finding out during all the sprints about the growing effort required to get it to where the customer or internal stakeholder is happy.

> It's not a bad thing to do a lot of estimating up front so the business can make an informed decision about whether it really wants to commit to the effort...

You're assuming the estimate is worth much. Which is what much of the industry is doubtful about. To underscore the point, most estimates are single values (eight months), not confidence intervals.

A better answer would be, "There is a 50% chance that we get it done between six and seven months from now, assuming requirements don't change."

...but even that assumes that all along the way, the project stakeholders are keeping in mind how many (how do you quantify?) requirements were added and discovered.

How it's said: "There is a 50% chance that we get it done between six and seven months from now."

How it's heard: "Wa wa wah, wa wa, we get it done between six and seven months from now."

You forgot the "assuming requirements don't change" part.

What's actually heard would be "Wa wa wah (up to) six months from now (guaranteed) wa wa."

Yeah, and they'll push for four months total at a meeting a month into the project, while simultaneously changing their mind on a half dozen requirements.

Ugh. I was on a project like this. I figured that it was mostly just that specific customer...

...Unfortunately, there's a lot of "specific customers" like this.

Except some of us make a living selling development of projects and solutions to customers that require pretty accurate (cost) estimates prior to even getting the contract.

Crazy world, I know...

I used to as well, until I started to refuse to do it. It truly is in the best interest of the customer to work this way, and I explain it to them. The ones that understand it and buy into this way of working are the ones I work with.

Sounds awesome. Unfortunately I don't work in my own company. Fortunately, I don't work with sales (much).

Absolutely this, clients need to know their costs upfront.

Meanwhile, I've been on the other side, working with a shop that insisted they were agile, and so refused to give an upfront cost.

That project ended up delivering a substandard project where key parts did not work well. Their answer was to tell us they would of course be happy to be paid for another 2 week sprint to fix those bugs...

As a customer it wasn't very satisfying.

I don't think working fixed price quote would have given you a different outcome though? The team produced bad quality product, fixed cost or agile won't change that.

Selecting the team to build the product on price is often a bad idea, and that's all fixed cost gives you above agile (the ability to select on price).

A fixed cost makes it easier to turn and around not pay (in whole or part) if the product is really not up to the standard specified. It's harder to do that if the cost is not fixed but overrunning and the response is "well we just need more time".

Also fwiw, the team wasn't selected on price.

In my experience it's easier and less risky doing it the agile way. You give them a 2-week sprint target. If they fail to hit that, then:

a. You don't pay them. b. They're probably going to miss every other target.

If you paid the agile shop for everything they did no matter how crap, and withheld payment from the waterfall shop because quality control, then that's got nothing to do with agile vs waterfall, it's got to do with your QC.

Not pay? Heh. For work performed? OK...

That is why my contracts require payment up front.

Sounds like they should have negotiated for a base rate with bonus pay out on meeting QC targets. For heavens sake people, align your incentives!

Everyone, on both sides of this river, they need to grow up.

To be fair, they're talking about subpar, unacceptable work.

Or it could be solid, perfect work that meets the best possible interpretation of the contract, but the client failed to communicate what they actually wanted and perceive it as subpar and unacceptable.

The problem isn't with the requirement for fixed-time, fixed-bid contracts. The problem is that agile methods and tools, and all the advantages they carry, are incompatible with those requirements: You can't start fast with minimal specs. You can't apply what's been learned along the way unless, miraculously, it costs less and takes less time. You can't make changes to suit changing business goals without a major negotiation on change orders. You can't access most of the benefits of agile inside that box.

You CAN, however, find contract developers who have mastered the art of faking agile methods and being buzzword compliant while delivering no feedback that prevents you from having bad ideas implemented in bad ways.

You will also find contractors that will lead you down the garden path. You could call them "half agile." They won't warn you that you are asking for something dumb, or unworkable. They will treat your mistakes as revenue enhancement opportunities and lay on the agile talk pretty thick.

This is why the "no true agile Scotsman" argument is so easy to make: That's not agile. You have the wrong people. You fail at agile. That's an easy case to prove but it isn't very helpful.

Real agile is hard to do and doing it with compromised ingredients, like using low-bid contractors that want to minimize their effort and/or enhance their revenue, that you have stuffed into a fixed-bid box, is agile poison.

The question is how this shop ended up getting the gig in the first place ;)

I'm curious, did your contractors host regular demos, sprint retrospectives, etc.? If so, was it clear early on that the results were going to be substandard, or was it a surprise fairly late into the process?

I'm not going to debate about whether they were doing "true Agile", but it seems like even without settling upfront costs it would still be possible to control risks by tracking value added vs. cost on a sprint-to-sprint basis. Hopefully it should be evident early on if the project isn't worth it[1].

Of course, this requires that the contractors be able to deliver value incrementally, which might not be possible in all projects. In which case, upfront cost estimates (and more thorough planning) will probably be necessary.

[1]In fact, this is exactly why Zed Shaw says he uses Scrum on risky projects(http://zedshaw.com/archive/the-c2i2-hypothesis/)

> ...customers that require pretty accurate (cost) estimates prior to even getting the contract.

If you require that accurate of an estimate, your business model cannot handle the risk of software development. Really, really wanting an accurate estimate does not make risk go away.

That's not exactly true, but it means holding the customer to exactly what they asked for, and having a formal "change request" process for any deviations. The track record of that model of development is not one of successful on-time delivery, but it does seem to make some types of customers more comfortable, even if it shouldn't.

Estimates can be fairly accurate if they're made for software that's a composition of previously iterated work.

If someone asks for a login system with X,Y,Z and you've done that 1000 times, you can be fairly accurate in stating what it will take to do it 1001 times.

The issue is when a piece of software is custom, and there's no domain expertise available for the custom bit then you cannot accurate estimate how long it will take because the developers will be learning while doing.

And some developers bring this on themselves. They could use the boring conservative technology that they know inside and out from the last project, or they could use something that's new and hot this month, which they have never done or have much less experience with (but is exciting).

Double whammy if it's an unfamiliar domain and a cutting-edge tech stack.

And it takes longer, so more billable hours.

Unfortunately, that doesn't change the fact that customers want estimates, and the more complex the project the more they need it for budgeting and scheduling purposes.

In the end, you have a few options for estimating:

1. Estimate the most optimistic scenario, knowing you'll end up using change requests and extensions to deliver (one of the worst options, but if you're responding to a proposal that's what the opposition is assuredly going to be doing);

2. Propose an initial phase to nail down requirements and deliver a schedule using traditional estimation tools (an easier sell if you're an internal team, tough if you're a vendor);

3. Use experience with similar projects to estimate, and then refine your assumptions with the client, using change orders to account for unexpected problems (roughly accurate to the degree of imported ambiguity, but now you have to risk going to war to get your change orders signed off on);

4. Use a top-down estimation model such as function points-based estimation to get in the ballpark, and then add buffer (and pray you aren't going to either lose money or blow the top off the client's budget levels -- or both);

5. Delphi the problem and get a bunch of developers and PMs to estimate the effort, then refine from there (useful for sanity checks, not useful as a real estimation process, yet one of the more common ways small shops build their numbers).

As you say, you can't make risk go away, you can only try to segregate it within your estimation. In my experience, clients really need stability in proposal numbers -- they're often willing to accept a significant embedded risk buffer in a proposal if it means they can limit unexpected costs. Conversely, you should also have a rough idea going into a project of what functionality can be cut and when so you can go/nogo those pieces when necessary.

> ...they're often willing to accept a significant embedded risk buffer in a proposal if it means they can limit unexpected costs.

So if they want 95% confidence in estimates, you need to pad the estimate a lot. It doesn't make risk go away. It just offloads risk onto the contractor.

That just means the contractor is assuming the risk of missing the 95% estimate. It also means the head of the contracting operation needs to be diligent about backing up the estimates of his employees. Or mitigating risk some other way (making sure each contract is a small fraction of the business, having enough cash to absorb risk, owning lots of other businesses, etc.).

Now, realistically, the risk is also absorbed by the developers in the form of unpaid overtime, stress, lost bonuses, and in other ways. That's just all the more reason for individual contributors to insist on honest estimates. Or find bosses that understand business models better and don't offload risk onto employees.

[] I understand that this is complicated and bosses don't typically mean to make employees miss bonuses and have terrible work/life balance. But to some degree it doesn't matter. Intentions don't make down payments on houses, make sure kids do their homework, or get us a full night of sleep.

Customers don't need to know the cost, they need to know the price. The most important factor in determining that should be the value of the software to the customer, not the time it will take to produce it.

True, but difficult to sell, especially to repeat customers who can calculate the price/hr on previous work.

Let's say they figure out you only spent 100 hours to deliver them $50,000 of value. It's your job to persuade them that that guy on the internet who says they would sell them 100 hours of dev time for just $5,000 will only give them $5,000 of value (if that). If I have a choice between getting someone who delivers value at a rate of $500/hour versus someone who delivers value at $50/hour, and I need $50,000-worth of software, I should choose the more efficient developer, not the cheapest one.

For the kinds of projects my company tends to do, when we're getting into repeat business we're generally not competing with other contractors. Our clients know that the other contractors would have a ramp-up cost that we don't, and an unknown working relationship vs our good working relationship. Instead, they look for per-hour discounts and ways to 'cut costs' while delivering the same value. They also typically have their own development team who don't get paid nearly as much per-hour as my company charges, so they look to share more of the work with their internal team. Since we do product, services, and support, including training, it's hard to argue that the internal team can't or shouldn't do the work.

It gets complicated, but in the end we mostly come out alright. Our clients are well aware that taking on additional work reduces our likely development cost but increases overhead costs and raises project/schedule risks. It's a tradeoff they accept, and we work with them to ensure the project is successful.

My point is, in the real world you can't just charge by the value you provide rather than by the cost of providing that value. That's a simplistic point of view.

I do appreciate that the reality is that development contracts work that way. I've just spent a lot of time being frustrated at how often the business conversation around software assumes a basic hourly contract cost-plus basis.

I have no problem telling a customer that something along the same lines as a previous project will be more expensive today, because it was initially underestimated and I ate the additional cost. They realize they got a bargain the first time, and I don't have to eat the same costs to keep the customer. Of course, starting from the first project, I work very hard on building a trustful relationship to support such a claim.

That gets into some interesting points:

1. Cost-plus pricing (or contracts). Here, the vendor (that's you) gets costs plus some value. Which means that the customer probably wants to know what goes into determining those costs. This in turn requires a close familiarity of both parties. This was more common say, in the 1970s and earlier, and especially in government major-projects negotiation. But it's why negotiating contracts itself is a substantial activity.

If you're creating a one-off product, there isn't a market price. There's what you and a buyer agree to. Depending on circumstances, neither of you may be particularly free to walk away and go elsewhere.

2. Moen's Law of Bicycles: bad customers make for bad products. This is almost certainly a variant of Gresham's law, itself a generalisation of the point that information sufficient to distinguish product quality is itself difficult to come by. This manifests in multiple ways: crap quality cheap products flood markets, or flashy, easily-distinguished, but not materially significant features are added to products.

True quality, measurable in the use value of products, is often quite difficult to assess. This is tightly coupled to the Dunning-Kruger effect.

3. Map-terrain relation. A plan (or contract) is a map, it isn't the terrain itself. In many planning (and legal dispute) instances, there's a confusion of these. A contract which fails to encompass materially relevant information is at risk of being voided.

The upshot, again, is that contracts don't define software development schedules. Software development does.

It also means that you probably want to desgin the contract scope for developing complex products much as you do those products: iteratively, over time, with conscious trade-offs of features and costs.

Everything else is setting you up for a mismatch of model and frame to reality. Reality wins.

I'd love to know more.

It's better than this actually. If they want a release by Christmas I can with, high certainty, tell them:

    Features will be in: A, B, C
    Features likely to make it in: D, E
    Features unlikely to make it in: F, G
    Nope: H, I, J
    Hell No: K, L, ..., ZZZ

The problem I've always found is that, no matter what, you are going to be arguing with sales and management about E, F and L instead of them just working with A, B, and C.

Ask them to give you a guaranteed schedule of sales that you will hold them to. They'll say it's absurd, you can't predict that. Turns out it's the same with software dev.

If you find yourself arguing about this with management or sales, they're not trusting you to be the expert at what you do. Tell them that.

Actually it's remarkably similar. Sales has a pipeline of deals, and someone has an estimate of how much will be booked over the next quarter. This estimate is a key part of managing the sales team and is directly tied to their compensation.

Can you imagine if developers explicitly had a code quota to work to? We usually don't, for very good reasons, but salespeople certainly are held to delivery standards on work that isn't fully predictable, so you can kinda see how they might expect the same from software teams.

And the analogy continues to work. There is a pipeline of feature requests, some at the vague-idea stage and some nearly shipped. You can provide more accurate ship-date estimates for the nearly shipped ones. And a person with good judgment who knows the stakeholders can provide a better estimate than someone uninformed or unthoughtful whether we're talking about sales or software builds.

Further: sales has a bookings target, but also has the freedom to meet that target with various deals. Similarly, product teams will have launch targets, but need some freedom on what exact features are included because per-feature costs aren't predictable.

Have you ever done sales?

Sales quotas are extremely common.

because it treat as creative not engineering.In reality A can be expand to A1, A2, A3.The most issue would be, client paid for A future but not A1, A2, A3

I do the same, except they give me an updated priority list every month, and once the priority for the month are set they cannot be changed. Misc. fixes are done eating up the time from the low priority features.

It removes me and my team from the losing proposition of agreeing to get something done before any other thing, as nothing of value can come from that discussion. And anyone who tries gets immediately redirected to the wiki page for the next month feature list, which opens one week before the iteration starts, by which time we know which feature will be complete and which will get back in queue.

"Hi. I need you to build me a house. It needs to have three bedrooms, two garages etc. How long will it take?"

"Well, we can definitely get you something by Christmas. It might be a bedroom, we may have just done the garages. Who knows? Can you write the cheque now please?"

I am not so sure there is a fundamental difference between what the article proposes and what you describe.

You start by saying this will not work, then in your description of what you do, there are many places where you estimate - or should I say guess? - what is feasible, and when.

The methods described in the article are analysis and extrapolation from experience, with an emphasis on identifying those areas that are most problematic due to some combination of uncertainty, importance and dependency, and giving them priority. You don't seem to be doing anything very different. Nowhere does the article say you have to schedule weeks or months on this.

Reply on news.ycombinator.com