All late projects are the same (2011) [pdf]
systemsguild.com
systemsguild.com
Customer: “how long will it take (and therefore cost)?”
Three vendors: “less than [the other 2 vendors].”
...iterate until “cheapest” solution is reached ... meanwhile the technical staff at the vendors are losing their minds at the sales team, telling them that what they’re specifying is impossible...
Customer, to vendor [cheapest]: “Here’s a contract. We look forward to this project being delivered on time and on budget!”
and no one is willing to say so.
On the reverse of course it makes sense to have estimates otherwise how should managers or product people decide what should be done next?
A big part of the issue is we don't have good knowledge how to estimate work. It's not a skill that is heavily invested in and is wrongly assumed as an inherent ability.
Then I make more promises and estimates, playing the emotional card of "sunk cost" to convince upper management to continue funding the team rather than torpedo the project. Then we ship, and I quickly get everyone on my team promoted "because we shipped" before it becomes clear to upper management that we shipped something essentially worthless to the company. Because we're in a huge profitable company, everyone can jump ship to the next shiny thing, and whoever is left holding the potato on its way to deprecation just gets re-org'd when it gets canned.
I know it's a bullshit game. But I play it well, and all the software developers on my team get paid and promoted because I am so good at it.
You sound wise and such in general, but perhaps your time would be better spent making a difference in how people believe work should be done at a place that actually allows brewing better practices instead of lauding bad ones.
Will you then argue that all new projects should avoid deadline dates, that no project should have anything other than an MVP and a iteration plan after that?
How do we change the expectations?
This is exactly the point of the article: Organizations routinely fail to prevent untenable projects from moving forward.
Why not work on something so valuable that even if it takes twice as long as the best estimate, with twice the headcount working on it, it would still be worth it?
As a a manager, I feel that helping to find projects that have that amount of value is an important part of my job, so that if I am fighting for a project my team's working on, I'm actually in the right from a business case point of view, not simply playing some BS political game.
If there are no projects that fit that value filter, then that's a completely different organizational problem.
On one level I have to take my hat off to you because you sound like exactly the kind of manager - one who has your team's back - that a lot of people would want to work for.
On another level you're part of the problem: you're wasting organisational resources that could be better and more profitably used elsewhere.
Still, needs must I suppose, because the real fault here - as the article itself hints - is in the determination of what the most valuable use of everyone's time is. I.e., if projects are going to be late (and they are) then you ought only to deploy teams to work on the ones with a putative 10x ROI.
But there was one occasion, where a product launch had been announced by a C*O before the product was finished, and the engineers were very clear that this launch date was impossible. So there was a lot of drama and bad feelings, and the engineering team eventually said, "okay, let's make a detailed estimate so we can be certain". The estimate came in at three weeks development time, one week after the announced launch date.
One week of the estimate was meant for testing and fixing problems that are found in testing. The business experts immediately jumped on this: "Testing and fixing unplanned errors won't be a big deal and certainly not a whole week. We'll skip that so we can get it done in two weeks".
Thankfully, this is retail banking, so releasing an obviously broken product flies even less than missing the deadline. The product was eventually launched uneventfully, one week after the detailed estimate and two weeks after the announced launch date. All in all a success.
The biggest pet peeve is when I give an estimate such as "I don't currently see a reason why we can't deliver X by the end of next week, assuming no other priorities come up." Of course something more important comes up in the mean time and I get scolded in a large meeting with a statement such as "You promised me I'd have X by Friday!" — this is especially infuriating when it is the PM's other project that is the higher priority interruption.
In short it sounds like you don't have the power to say no - but actually you do.
Of course, if you already have a toxic development culture like the one you describe, then they'd probably abuse more agile processes too.
Another thing that helped us is to answer with ranges instead of dates e.g., "this will take 1-3 weeks".
* If you just don't know how long something is going to take, and you need a date, then you're late to doing the work to know how long it will take.
* If you do know how long something is going to take, and it's longer than a externally imposed date, then you know exactly how late you are.
This is partly just playing with semantics, but it's also a way of talking around different views of the problem, in this case emphasizing that if dates are a feature, reasonable estimates are themselves a project. If the estimate work hasn't been properly completed and understood before dates are given, obviously, the larger project is by definition already late because there's work that needed to be done before beginning and wasn't.
And I think if you take this view, it's easier to see that it's a project or team management problem rather than an engineering productivity problem.
The problems start when you need to give an estimate before even being allowed to start working on something.
That means building a simple verison first that still fulfills the purpose while not being perfect or polished. Then iteratively improve it to what you and the customer ideally want.
That way, if the deadline comes before you're done, you at least have a working version that fulfills the basic need and allows the kind of workflow that's necessary, even if it still has some rough edges or even gaps in rarely used areas.
Of course this approach requires you to write your contracts in a way that doesn't tie you down too strictly on how exactly things will be solved.
So the customer can have assurances that they aint gonna be bogged down with sunk costs. The developer can have more frequent deliveries.
only if those doing the promising knows what's within bounds and what's wildly off bounds!
If the sales person makes wild promises because his commission depends on making the sale, but not the delivery, then the whole organization's interests are not aligned properly, thus fail.
If the salesman's commission comes from successful delivery, then he can bring in an engineer who can at least balance the promises. Who knows, may be different departments can cross-contaminate, leading to a better, more well rounded organization.
Please elaborate! :)
Let me first acknowledge that there my thoughts are a mixture of the types of projects that I've been working on, my own personality and my own path dependent experience. In particular, my way is intentionally short sighted and won't work for a project plan that takes five years. Also, for sure, if the sales team/ceo is wrong and the project isn't worth doing, then doing it "lazily" won't fix that. But with that out of the way, here's what I mean.
The engineering team has it's own values and priorities and they just don't have anything to do with what really matters to customers. In the end, everyone brings in emotional baggage from the last project they worked on, and that's what's important until new information comes in from customers and the sales team.
So what I shoot for is taking the requirements that we have and producing a long range design that will meet those and little more, and then producing features as quickly as possible and presenting it to the customer (or the customer's advocate).
Why does it limit scope? Because the deadline doesn't move! To meet the most important requirements in a way that actually sells the product, lower priorities are are cut or reframed.
Now he’s amending his opinion to state that estimation is not really the core issue. The core issue is that projects are doomed from the start by mismanagement. Issues such as: racing to catch up with a competitor (giving the s/w group an impossible deadline), or starting a project whose business value is so low that minor time and cost overruns reduce the actual value to below zero (at which point people start blame shifting to try and hold on to their jobs).
Instead, we systematically underestimate.
This way, we can accept that we're all terrible at estimates but we do our best not to paint ourselves into corners because of our terrible terrible estimates.
He was nearly always on time, and never acceded to the customer's desired delivery date unless he could meet it.
I'm not saying this works in all environments, but for his case, it did.
The reason that software is so hard to do in most cases is that you are making something that (by definition) has never been done before. Of course stuff is going to go bad. This is largely a corporate culture problem. In these cases, I want to have the sales people come to a weekly meeting and have them estimate when they will have 100K sales for the month from new business. I don't think they would like that very much and you'd get a lot of dithering.
Sales companies will prioritise the requirements of sales people, and tech would prefer to ignore the sales guys.
Frankly, as an engineer - I would never prefer the company where the sales people can decide how much overtime I would be doing in the end. Also - they would be getting all of the bonuses, since it's them making the decisions in the first place.
The good thing I find is that working for the engineering companies usually implies having clients with a head on their shoulders (hence them knowing what they want and what they're getting; and how much time it usually takes).
I really learned a lot from the book called Thinking in Systems and nowadays I can't help to look at projects, from within the context in which it operates. Definitely recommend the book for somebody wanting a bit of an introduction to that mental model.
If you can't do that, then you are the one to blame for tge project. If you did, though, then youre free to crash and burn happily ever after.
http://ptgmedia.pearsoncmg.com/images/013143635X/samplechapt...
There's a list of "negotiation games" that, sadly, you'll probably recognise. And a bit of more practical advice:
>> The problem I have with “estimate” is that I understand
>> the word to mean, “best assessment knowing what is known
>> at this moment and what can reasonably be predicted about
>> the future.” From experience, what I have seen happen is
>> that once an estimate is published,it becomes a rock
>> solid commitment representing the maximum amount of
>> resources needed to accomplish the project phase or the
>> entire project.
> Unfortunately, there’s an uglier aspect of the political
> negotiation when you introduce the plus-or-minus
> qualifier into your estimate: You’ll be accused of
> uncertainty, wishy-washiness, weakness, or even
> incompetence. [...] What senior management really wants
> is a firm commitment — a promise that the project will
> be finished on a certain deadline, with a budget of a
> certain number of dollars, and a staff of a certain size.
> This gives them the enormous luxury of (a) no longer
> having to worry about the problem for the duration of the
> project and (b) having a convenient scapegoat to blame if
> the promise is broken.
> Jim McCarthy, in his excellent book, Dynamics of Software
> Development, suggests that the project manager needs to
> confront this head-on and persuade the customers and/or
> senior management that they need to share some of the
> burden of uncertainty [around schedule, cost, resourcing]
> that the entire project team will be living with on a
> day-to-day basis. Thus, the project manager effectively
> says to the customer or the senior management group,
> “Look, I don’t know precisely when this project will
> finish — but since I’m the project manager, I’m far more
> likely than anyone else in the organization to figure it
> out as soon as it can be figured out. I promise you that
> once I know, I’ll tell you right away.”
> Only a manager with a lot of self-confidence, and the
> ability to walk away from the assignment, has the
> chutzpah to say something like this in the politically
> charged atmosphere of a death march project. The time to
> say it is at the beginning of the project; after all, if
> the customer and senior management do not respect your
> ability as a project manager, and if they don’t realize
> that you do have a better chance of knowing when the
> project will finish than anyone else, then why are they
> putting you in charge of the project in the first place?
> Are you being set up as a scapegoat? Are you going to
> be a “puppet manager,” with all the decisions being
> made by other political manipulators in the
> organization? If so, now is the time to get out!
> Similarly, if you’re a lowly programmer on the project
> team and you see political games like this, it may be a
> strong indication that your project manager (a) doesn’t
> have the confidence to believe in any estimate that he
> puts forth, (b) doesn’t have the backbone to stand up
> for himself and for the project team, and/or (c) has
> gotten himself into a political situation where all the
> key decisions will be made by people who are not
> directly involved in the project. Again, this is a
> strong indication that the project is doomed; and
> before you get too deeply involved, it might be a
> better idea to seek greener pastures.
You'll know this is the case when you say "at least a month", and the PM looks at you like you just shot his dog.
"The answer I need is two weeks."