Driving engineers to an arbitrary date is a value destroying mistake
iism.org
iism.org
You wanna move beyond anecdotes and nice sounding business arguments? Read FIRO by Dr. Schutz prob out of print or "Human element" by same. His son Ethan has updated these books.
Look, in normal high functioning groups conflict is normal. What confers distinction there is two things:
- no individual on team is ridgid and unchanging
- conflict is resolved with 100pct buy-in ie not cause the boss yelled, or you were out gamed, politics, favourites. IOW there is trust in the team (together with self awareness, competence etc which the book hits on)
When human factors burn down projects down the issues are quite a bit more about trust, individual self awareness, defensive behaviors, and not resolving the problem in a group way. Trying to eliminate conflict with rules of thumbs is a fool's errand. Conflict itself is not a sign of badness.
References to organizational modalities (the strong boss, the clear vision, and similar sounding approaches) help at the margins only and then only randomly.
The signs of a good compromise is everybody leaves unhappy about something. Not that products should be built entirely on compromise... that also leads to garbage.
It is difficult to pick the pieces apart, but frequent strong-armed conflict resolution is definitely a sign of a bad environment.
See https://en.wikipedia.org/wiki/Disagree_and_commit for a list of successful organizations that have adopted a principle meaning that and why they did it.
Buy in resolves people with wrong ideas trying to steer into the iceberg, and loses the benefit of disagreement overcoming resistance and inertia towards bad goals.
The Amazon principle is "Have Backbone; Disagree and Commit". Most leadership principles in Amazon are in conflict with each other, creating a sort of checks-and-balances situation. But "Disagree and Commit" is so powerful that it needs a check in itself. It doesn't work without "Have Backbone" - but most senior leaders I see who love it never mention that part. They always spell it out as a "follow the orders once they're made" sort of thing.
1. Put together a ~medium detailed listing of what work needs to be done, broken down into sub-tasks.
2. Include estimates for how long each sub-task will take.
3. Roll that up into a gantt chart with dependencies reflected.
4. Identify opportunities for parallelization so you can answer the question of "will adding more folks make things go faster?"
5. Identify and be prepared to speak to which portions of the schedule represent padding, and explain why it was padded in that way. Share your thinking in the trade-off you made and be open to adjusting -- and de-risking the schedule in other ways.
6. Realize this is a bit of a negotiation, not necessarily you vs. them but more you and them vs. what you can deliver. In a negotiation it's important to not appear intractable. Have things you can give away.
7. At the end of the day the engineers are writing the software. If you're not actively dragging your feet, what you say does go, so it's a question of where you're spending your time, and on what. With that in mind, your boss has a job to do too, so help give them the tools they need to do so.
8. Don't burn bridges, y'all have to work together.
I'm sure it's happened but I've seen far more delays due to gold plating, over-architecting , or bad estimates.
I think this type of wording is part of what causes animosity in the industry about this topic. Estimates are not called "promises" for a reason, and any attempt at treating them that way will result in failure.
Software estimates are notoriously hard to get right and it's not uncommon for them to be wrong by several orders of magnitude, and this effect is magnified for bigger estimates/work. Any decent project plan needs to assume this is the case, and it needs to be continually refined as work progresses and the accuracy of "estimated remaining work" gets higher as the problem is better understood.
The worst project plans are the ones that take the initial estimates and use those as delivery dates, but I'd classify that as a "bad plan", not "bad estimate".
> Estimates are not called "promises" for a reason, and any attempt at treating them that way will result in failure.
So management 95% of the time treats estimates as commitments. What they are going to do with the numbers you give them is promise upper management or clients "feature x will be done by y". As engineers we have two options, scream until we're blue in the face "that was an estimate not a commitment" or just give them a commitment and call it an estimate.
You're right software estimation is hard, but the really bad estimates I've seen (> 100% over) were due to a combination of dysfunctional organizations, lack of estimating skill, and an organization that didn't put effort into making sure it's estimates were right.
OK now hold on there just a sec... I've seen estimates off by 20%, 50%, even 100%. I'm sure there are estimates that have been off by 500% (I'd hate to dig into why). But "several orders of magnitude" is something like 10,000x -100,000x. Hopefully that is just extreme hyperbole, but honestly in the context of discussing estimates, it seems particularly out of place.
True, but there needs to be common understanding that they are estimates, and if the projects take consistently longer, it means the dev is bad at estimating, not bad at writing software[0]. Also, if you get a median time, you'll have, by definition, half the projects taking longer than that; a mean, mode, or 90th estimate will still have plenty of high points; and if you want 100th percentile, you don't even need to ask the developer: "sometime in the next century, probably".
0: They can be that, too, but that's a independent flaw from being bad at estimating.
You promised, and it didn't happen, so you are a bad person.
I think you're missing the point. Nobody has that knowledge prior to the project being complete.
The only exceptions are very small projects or projects that are just rolling out pre-established boiler-plate solutions. Any reasonably complex and reasonably sized software system contains far more uncertainty than humans are capable of thinking through.
This means any given "good" estimate is either unlikely since it's in the tail of the distribution or overly optimistic if in the bulk.
Why? They are always wrong. Asking for a plan? Sure, because planning makes you think through things. Planning good, plans bad, to paraphrase. But estimates are always wrong and always useless, dev with 30+ years experience and I've never met anyone that can actually estimate software except in the trivial sense.
The error bar on software estimates, like solving a math problem (because they are the same thing! If you don't think so, you don't understand software), is infinite.
Although I like your quote that "conflict itself is not a sign of badness".
Almost no software development task is like proving or disproving P=NP: often, they are like homework problems. And, like uninspired homework problems, it is often a matter of grinding through a process with all due attention to the details.
Without prioritization it's very easy to get stuck in a loop of endlessly adding small improvements/testing new ideas, but never releasing the final sellable product. There are a few notorious examples from the game industry where no time pressure lead to either no product, or a hopelessly late-to-market product (e.g. Half-Life 3, Duke Nukem Forever). There are more smaller examples where entire teams endlessly juggle the code between web frameworks, as they come in and out of fashion, and neglect more user-impacting features.
That said, most of time pressure should only be applied at the decision-making level. Once you have decided to ship feature X with framework Y, it's useless to nag on your devs to do it in half the time by dropping unit tests, or doing 60-hour work weeks. Instead, you should have picked feature Z with 50% the complexity and 80% user impact.
I'm not sure they even worked on it, have they?
After the release of Half Life Alyx a few months ago, they have heavily implied a recommitment to Half Life 3 in interviews.
I have a simple rule of thumb: a deadline is a "real deadline" if I would reduce scope to hit the deadline.
A deadline is just a goal if I would not be able (or willing) to reduce scope to hit it.
Treating a date as an absolute deadline without being able to adjust scope is where productive work gets totally derailed.
It's all about weighing everything and making the right trade-offs. Sometimes short term solutions are the right approach, maybe you need a POC more than a polished product to get that next round of funding or something.
I've struggled with that in the past, and next time I'm presented with this kind of project, I'll clearly explain the trade-offs, for example translating lower quality to: this may increase the number of bug reports significantly, increase the maintenance cost, prevent the addition of other features with significant overhauls, this and this engineer may start looking for another job, etc.
It's also what most "agile" environments end up turning into: a bunch of inflexible arbitrary deadlines designed to make developers work more, amongst many other problems.
I forget who I saw do this (it may have been Nat Pryce at a talk somewhere). He took a list of the items that needed to get done. He rendered the first item with full opacity. With each subsequent item, he reduced the opacity. At the end of the list, the items were invisible. Then he said, "This is roughly the likelihood of me getting these things done."
It's a mistake to draw a line on your list where you think the deadline will fall, because then you are fixing your scope as well as your time. But if each day you adjust the opacity of your list, hopefully people will get the point.
I'm not even sure you disagree with the article, given you are in a radically different position; how often do you think, as an implementer, that your business ideas are complete bullshit, sometimes even just because you don't know all the details you are thinking about on the business side?
Project management is about communication. Having that problem reduced to communicating mainly with yourself changes a little what is effective and what is not.
> That said, most of time pressure should only be applied at the decision-making level. Once you have decided to ship feature X with framework Y, it's useless to nag on your devs to do it in half the time by dropping unit tests, or doing 60-hour work weeks. Instead, you should have picked feature Z with 50% the complexity and 80% user impact.
So yeah, we could continue to interpret that as you mainly agreeing. Implicit cultural incentives are a thing and it is illusory to think that "devs" won't be aware of schedules and won't drop half of unit tests if there is a risk of looking "bad" at short term if they stuck with state-of-the-art practices.
Note that the solution is not in the extreme opposite, of course. It never is. There is a system of values to be tuned, and I'm not sure a lot of projects exist where the date of completion does not matter at all. Your example of Duke Nukem Forever was on point :) But being date driven would mean the date is the most important thing: this also is rarely the case.
Before my current project is designing an API for logging and viewing image segmentation masks, A mask that segments an image into n classes. We started with a large design that allowed users to configure to masks in different layers. Toggle them on and off and do composting. Instead of building this exactly I shipped a small fast very that renders images with masks and adds some toggles for each class.
Turns out one of our main assumptions was wrong we thought the number of classes(n) would be small. It's 1-2 order of magnitude larger on average for all of our users. And sorting through large amounts of classes is the biggest issue our users are clamoring for. Now instead of spending a bunch of time building a UI for editing the masks. We first focused on handing large counts of classes usefully.
As well, all of the different settings for mask composition and layout we dreamed up really don't matter. While there is 100's of different ways you can layout and compost the masks. Turns out there is 3 default layouts that everyone wants, we don't need a UI for tweaking layouts we just need a toggle between the three common states.
Instead of spending extra time to get it right. We aggressively cut scope and shipped a not particularly useful feature. We got two big benefits out of this. We ended up building something much smaller in scope than our initial design and we ended up building something better and more useful.
I think it's also potentially reasonable to say "I really need these 20 features, and I'm ok if you cut corners on quality to get me those 20 features by this date". But in my experience, I've never heard anyone say that and mean it; they still get pissed off when the end result is buggy as hell. And I just don't think it's a good business decision anyway; burning customer goodwill is only a winning strategy when you're a monopolist, and even then only until you get unseated, as everyone eventually does, given time.
This can be true, but only if the team is not otherwise good at prioritizing things. And although deadlines do create pressure once they get near, they often create a feeling of slack when they're still far out. They also create a feeling of slack after they pass.
1) is anybody aware the team sucks?
2) if yes are people open about it? If no, why not?
3) now what?
Reliable estimation requires a fair bit of information. The technologies have to be well understood. The team has to have worked together sufficiently. The business needs user impact has to be well researched. Estimates can't be more precise than any of those things.
Further, estimation requires stability in team, technology, user needs, business environment, and competitor behavior. To the extent that any of those things are expected to change, estimates become less and less worthwhile.
Lastly, estimation assumes that nobody will learn anything significant over the course of the project. That's surely true for some domains. But in quite a lot of areas, especially the startups we focus on here, the whole point is to learn new things. We see how technologies perform, and then make new choices. We test user reaction, and then change product plans. When that happens, estimates quickly become stale.
So when we're in situations where estimates aren't particularly useful, we have to use different methods. Personally my favorite is using the Lean Startup framework and updated versions the Extreme Programming practices, especially Continuous Deployment.
I have an older talk on some of this here: https://vimeo.com/24843552
And I've written some about it here: http://williampietri.com/writing/2015/the-big-board/
I hope that helps! I may not check this far back in the HN comments again, but feel free to email if you'd like to discuss it further.
The real losers of "schedule chicken" are the customers, who receive buggy and bad products. The company will suffer long product update cycles because of the effort needed to stay ahead of the quality debt. Explaining this to a manager who's never even read even one of his copies of The Mythical Man-Month can be tough.
Things aren't necessarily better when you let a project freewheel. "You'll ship someday, just do a good job and get it right" doesn't necessarily translate into the urgency to ship a product, and I've seen efforts on the scale of tens of millions of dollars fail.
Other arbitrary dates include things like shows (well, what is the real cost of missing that show versus shipping something buggy to bad reviews?) or holidays (hitting Christmas used to be a good excuse for schedule pressure in the game industry, much less of an excuse now).
[Anecdata: Years ago there was a manager at Microsoft who told his team to ship by some arbitrary date, which turned out to be when he was planning to go on a long vacation. I'm happy to say that Microsoft fired him.]
I often suspect that the true goal of most time-estimation exercises is to guilt you into working unsustainable amounts of unpaid overtime.
If you don't think you have the money to meet the estimated project time you need to find ways to reduce the time. Sadly that leads to problems as described in the article.
The is an impedance miss match between software development and real world business reality.
A. aren't testing enough / at all or haven't given any consideration to what is the smallest change needed to deliver value
B. have serious organizational problems around coordination
C. are running a company completely unsustainably
I don't see the point in working all-nighters for a company unless you have a stake in the company (shares or owner). For my own company, sure I'll pull an all-nighter before an important meeting, for a company where I am one of hundreds? No thanks.
Labour laws tend to be more protective here in the EU though so no fear of being fired by refusing rediculous requests.
Haha, isn't this the truth. I work at one of FAANG as well and was curious how one of my colleagues was accomplishing so much. His output is astonishing. I checked his work profile and noticed he had commits, notes, and research actions starting at 7:00 AM and going until 1:00 AM the next day, every day, including weekends. I don't think the founder of this company ever even worked that much. I guess there's just some people who never burn out. Meanwhile, my wife gets annoyed about dinner when I work past 6 PM...
I once read there is only concrete that has either already cracked or is about to crack.
We all do this. Once. When we grow up, at ~23.
Every ten years or so I get stupid, I guess.
I guess I finally wised-up when I turned 51, we'll see if I stay that smart in a couple of years :-)
That tends to happen once every 10 years.
It doesn't tend to happen in day-to-day grunt work at an AdTech shop, pushing JIRA tickets.
Turned out he was outsourcing his work to a job-shop in India... Couldn't actually programme worth a damn.
For instance, when I was in college, I had a job that paid by the hour. I didn't need a lot of money, so I worked one day a week.
In our field, there's no option where you can opt to work four days a week, or work part time.
You would think that at some point, and employer would come up with the idea of opening up some part time positions. There's got to be thousands of talented techies who've reached a point in their life where they don't want to work 50 hours a week. But such a role simply does not exist.
It's particularly odd since a lot of us make good money. IE, my Dentist probably works 20 hours a week, and I imagine the reason he works so little is because he isn't living paycheck to paycheck.
In the US and other high wage countries, most developers are hired because of the need for maximum velocity at all costs or having them available to discuss things directly with non-technical stakeholders. Neither of these roles fit well into spending 20 hours a week knocking out bugs and well specced features.
I lead a team four days a week and it's working quite well. The other team members work five days a week. They didn't request shorter time.
It would hugely open up the field to women with caring duties if more companies would do this. Perhaps this is why they do not - I don't know.
Sorry HN! Of course I meant that it's actually just a complete coincidence that so many women leave tech after having children. We all just choose this completely on our own and childcare is not a factor in the least. There's literally nothing we could do about this completely natural and voluntary situation. It's not even worth asking the question.
Hammer it, lads. Happy to go to zero!
The difference between "full time" commitment, where the business might say "we don't care how much or how little you work, just give us results" vs where you are saying "I work for you 32 hours per week - no more no less, that's it, thank you very much" is huge. Mindblowing.
Suddenly you are not under pressure to deliver deadlines, you are only giving them 4 days a week of your time, doing your best, and that's it. You are off the hook.
I really wish they wouldn't, because I have to come along and clean up the messes they make in their fatigue induced haze. Meanwhile I just finished a project that we originally plotted as taking 18 months. It took about two once we sat down and thought about what really needed to be done, and those two were mostly slowly rolling out configuration changes.
Most of the problems I deal with are young engineers coding like the wind, adding new services, and locking the system more and more into its current state until all wiggle room is gone and it becomes a multiyear, multiteam project to modify the system semantics in the slightest way.
I like to think that most big-ish products can be divided into three equal parts, each consisting of 90% of the work. It's that third 90% that separates the decent products from the great ones.
[Yes, I can do math. I mean that you get to your original 90% goal and realize you've still got a LONG way to go.
"Problems worthy of attack / Prove their worth by hitting back" -- Piet Hein]
Anecdotally, the place I work for in the US doesn't expect or even want people to work all-nighters, if a project is that under-resourced, it needs to be halted or more resources need to be allocated. Under-resourced projects always fail and waste a lot of time and money. This is only the culture I've observed at my engineering dept though, FWIW.
Perhaps he should read both simultaneously and finish in half the time...
I think a lot of developers deep in the bowels of large companies are completely disconnected from their work and their paycheck. Granted, a lot of management deep in the bowels of large companies are equally disconnected.
It would be good for a developer out of college to spend a couple years in a small consulting shop. You get to see the whole business of software development end-to-end. It gives you perspective you wouldn't get in a large company or in like a highly specialized technical team.
I was better than most devs when I was in a big organization (due to reading Steve McConnell's the black art of software estimation), but starting my own consultancy has made a huge difference. Nothing like eating a part of an invoice to keep a client happy because you mis-estimated to take your estimation skills to the next level.
Taking away the date is definitely not the right move. Now you've given every PM and sales guy carte blanche to throw whatever flavor-of-the-moment idea comes in their head. The team will be churning constantly to keep up with the requirements and then someone will suggest, "I know, let's do scrum!" so now we can all enjoy the stress of unreasonable deadlines on a continuous rather than waterfall basis.
To the contrary: a properly-managed date is an incredibly valuable constraint which the entire team can use to make tradeoffs against. To make it work management needs to trust the team, and engineers need to understand the larger business picture because they are the ones with the knowledge to find creative solutions that actually work and decrease time to value without compromising technical quality or long-term maintainability.
I've left that company after having been unable to fix the situation, which was ultimately a political problem; as you said, we had an "executive whose philosophy comes largely from unskilled, fungible-labor economies". Is this fixable, or is "leave" the only right answer?
But to actually make that happen, there has to be someone with authority do actually do the firing, and that person needs to understand the problem...
One anti-pattern in engineering management is shielding engineers entirely from timeline estimates. A CEO with limited runway, reporting to the Board, doesn't have such a luxury of infinitely elastic time.
Edit:
Note: dates should never be prioritized over value. If you're hitting dates and not delivering value then it won't do good for anyone. Product value should always be the function to optimize for, but rarely is it entirely independent of time.
YES! My current project has this issue. There is no deadline for my project beyond "urgent" (but so is everything else) and as a result, any request from a client gets approved. Client wants a button shifted slightly to the right? Approved. Comms wants the semi-colons gone? Approved.
For a supposedly urgent project, we are making no real progress week after week on functionality simply because urgent has no tangible meaning.
Took me some years to fully internalize this, but I endorse it.
I.e. Notes are useless, but note-taking is invaluable.
I used pen/paper. I also tried, separately, computer-based stuff and that ruined my ability to concentrate.
I think the key is to still allow your mind to wander and build the connections. I found I always had the best luck if I was a little distracted, but mostly paying attention.
Perhaps because the variations in attention helped to differentiate which information felt important vs what was just background noise.
so having a date matters, just "dont do something stupid to hit a date" what i built: https://moca.computingarchitectures.com/en/~hello-world/
I use this exact strawman to bounce ideas off of people and play devil's advocate.
He's such an integral part of my routine that I even gave him a name and a function: Richard Johnson, CEO of a pate factory.
Natural pressure is the best: someone you care about suffers if you don't solve the problem, heating is broken and winter is coming, that family needs a place to stay etc.
The worst kind of deadline is completely arbitrarily set by someone who has no idea, because they thought promising early deadlines would make them look good and get them the contract.
This is a structural problem IMO, because freelancers usually don't make that mistake very often. If you manage someone elses time you can afford to become unrealistic.
There's nothing wrong with deadlines, even arbitrary deadlines. Without them it's easy to get lost down a side path of "oh, it would be nice if we could do X". They serve to focus attention. The thing is not deadlines, but how you work within them. If it's arbitrary, Work backwards from the deadline to what can reasonably done, and present a detailed account of those estimates. If you have a hard-charging "just get it done" boss, part of your responsibilities is to manage upwards, educating your leadership on what your job entails so they can have reasonable expectations. Most managers are not actually pointy-haired bosses, but you have to work towards that relationship: if you treat them like a PHB, you'll give them no way to evaluate your work except their own faulty metrics.
Will this work in every situation? No. But that isn't representative of some universal problem with deadlines, it's representative of poor management.
here's nothing wrong with setting dates. There's nothing wrong with setting arbitrary dates
What you want to avoid:
1) Pressuring devs into crunch/burnout/quitting/morale loss/checking out. Turnover will eventually cause brain drain when your best talent leaves for greener pastures, and makes hiring more expensive when your reputation sours.
2) Pressuring devs into sacrificing long term productivity for meaningless short term milestones. It's one thing to prioritize a feature or two for an important presentation to investors who will be making decisions about your financial future at the temporary expense of, say, test coverage. It's another to do so routinely that codebase stability suffers, developer velocity gets wasted on disturbingly routine debugging sessions, and build engineers disable code coverage metrics for being "just too gosh darn depressing to even look at."
Basically, the project plan has to allow for some scope cutting along the way. There will always be a few false assumptions that are discovered only after the project is well underway, and the only way to manage that is minimize the surface area of any given dependency.
The responsibility of delivering a project on time belongs to the manager (which for the purposes of this comment can belong to an IC if she or he decides to think like a manager). For every milestone on the schedule, the manager should know all the risks for reaching the milestone and should have a plan for handling each eventuality.
This type of planning goes hand in hand with protecting the team from distractions, preventing scope creep at all costs, and setting the goals of the project before it has even started.
Software created with time constraints is higher quality and is more fun to make.
After about 20 years working in tech, at multiple companies of all sizes, and in dozens of teams, working through all levels of engineering, management, and other jobs, I can say without any hesitation that without due dates (reasonably negotiated of course), engineers will spin indefinitely and nothing will ever ship.
A receipe for disaster? Only produces bullshit results? Far from it. Ofc there is a percentage of people who just cannot deal with the freedom, but in terms of results (I studied in the Film department) this school creates the best films with the lowest budgets in the country. Most people that work from you get nothing for the job.
Again fittingly I now work in the IT department of said university, where I often create software that we need. There is never a deadline, only a "Somebody needs this and wants to use it and is doing a horrible manual workaround in the meantime" that is usually a far better motivator than an arbitray deadline.
Rhink about it this way: Imagine you are a carpenter and you have to build a cabin in the woods. What motivates you more:
a) An abstract deadline set up by your manager (you know the cabin will be sold to some rich guy who doesn't need it anyways)
b) Someone you know will live there. They are couch-surfing with their family in the meantime. There is no deadline.
Both create pressure, but one is by far a more effective motivator to do things fast yet good than the other.
Of course that would mean your company has to solve meaningful problems with a natural deadline.
Natural deadlines by their nature are usually fuzzy. The trick is: make your engineers care. If you can't, you either hired the wrong person with the right skillset or what your company does is not important enough to most people.
Many programmers and engineers instinctively know that their work acts as a multiplier for good or for evil. However if you multiply anything times zero it remains zero..
After all the feedback I got from this...
https://news.ycombinator.com/item?id=17154355
Someone pointed me to Reinertsen's Principles of Product Development Flow book, which put into numbers and models everything I'd been trying to say (and then some).
It worked great with my team, but I was failing hard at communicating and involving everyone on the business side so I started researching for something that provided the business side, wrapped around his methods. Which drew me to Scaled Agile Framework.
There's a lot in it, but the core of it is simply on a pace of 8-12 week increments, you spend 2 days letting your developers provide you with a plan, estimated based on 2/3 of their actual available time. They make the plan...not management.
The plan is presented to management, management discusses tradeoffs and options, then ultimately management will accept a final plan and the developers provide a confidence level in their ability to deliver the plan that they have set out.
When new requests come in, they are discussed for the next 8-12 week increment so the developers can focus on executing the plan that everyone agreed on.
There's still room left for small things that come up, there are progress check-ins and communications that happen along the way. Risks to the plan are identified and discussed so everyone is aware of them up front. Variability is assumed (up and down) throughout the plan. The only assumed commitment is to the 8-12 week deliverable. Nothing in between is expected to have a guaranteed delivery date.
It leaves little room for surprises and plenty of room for people to work in a reasonable manner. I can't speak highly enough of how effective it is to addressing this problem.
There’s no “we said it will take X” and management saying “do it in X/3”. That’s no longer a plan made by developers that developers are comfortable standing behind.
Over 10 weeks, you would plan based on 2/3 of time for 8 weeks, with nothing planned in the final 2. If things are running behind, this becomes buffer to finish it out. If things are finished on schedule by the 8 week mark then the final 2 weeks becomes time for developers to pursue other things. Training, prototyping things they’ve wanted to build, whatever.
It’s the old Google 20% time concept built into each increment.
But it works wonders for keeping out midstream new work from side tracking your team so that you can actually deliver on that commitment.
But if there is no external reason to push for a date then setting an arbitrary internal date when it provides no additional value is just silly.
but you have to admit that without a date developers will just noodle around forever.
I think the only recourse is to constantly be evaluating the current slate and the date and keep sliding it out as necessary or throw stuff off the train.
really, you should never miss a date. you should have realized some time ago that it wasn't going to happen and slid it out already.
This seems more a statement about extrinsic motivation than dates in particular. Dates are the most obvious blunt instrument for applying extrinsic motivation, but there are others.
Intrinsic motivation is in some ways even better. You can't force someone to be intrinsically motivated to achieve something... but pushy dates and expectations can decimate any motivation that was already there.
Speaking for myself as a software engineer I have only seldom fixed dates and deliver constantly. My main goal is to deliver value to the party I work for and my end users.
True artists ship.
On the other hand, I do stall occasionally. It's because I don't understand something, and rather than just ship some random junk I try to make my junk do the right thing.
To an outside observer this carefull design - engineering if you will - is quite indistinguishable from noodling.
If you put an arbitrary date for me then you will just get bad code and that is a bad deal for everyone.
There are times when the date is fixed. Then you haul ass and raze mountains to ship, no matter how ugly the result.
If the date is constantly set to some random date I have no idea should I dial quality down and increase velocity or vice versa. Therefore a constant fixed date will just get worse quality, even if there was no good reason to hurry.
I think you meant to say how to have enough transparency so that everyone is honest and accountable.
You don't need imaginary dates for that. Just have people report at fixed cadence what they've done. Good people remember they need to explain what they've done and dont wander off into the wilderness for no good reason. This is sufficient to keep everyone on the right track in almost any environment.
Now you say - "Aha! But what about the occasional person who is incapable of doing their job! How do we point him out!" - to which I answer: If you need to design your development process to safeguard against botched hiring since they seem to do it constantly you really need to fix your hiring. Good people will detect a rotten egg eventually and there should be ways to deal with this outside of regular project cadence. If you try to micromanage great engineers and programmers you are just wasting talent and eventually everyone will just move on and all you have left are the rotten eggs.
Why do software developers have arbitrary dates at all? We certainly dont. If a customer says they want their work done by the end of the day, we'll take that into account, but the best we can do is estimate how many hours it will take to work on, update you on our progress and inform you of obstacles we encounter. Work proceeds at a safe and productive rate.
For instance, if my wife asked me to make a pizza, I know very specifically what steps are necessary.
When a client asks my employer to create a piece of software, it's a lot like being told to write a book. Sometimes you don't know where to begin and a lot of the time you get the thing 40% finished and you feel like throwing it in the trash and starting over from step one.
Management doesn't really set dates for us right now.
I saw this written during the 2017 cryptocurrency craze. Has stuck with me as one of the most insightful things I've ever read. Also a fan of 37 Signals' budgets. [2]
[1] https://medium.com/graphprotocol/introducing-the-graph-4a281...
[2] https://signalvnoise.com/posts/3746-drive-development-with-b...
If you come up with a realistic offer, which should be pretty close to reality in both time and budget, you're out of the race. Other contractors have put in offers with totally unrealistic numbers, making you look both slow AND expensive.
And like clockwork, the undercutting (bid) winners will deliver a 60% finished product come project due date, and then spend years on fixing bugs and "upgrades".
Their in-house devs, and outside consultants for that mater, are worked to the bone. Bugs happen, and the tickets pile up.
And worst of all, these companies get awarded new projects, again and again.
In reality, in software development all deadlines are pretty much arbitrary. Sure, there is profit to be made if you move quicker, but there's considerably more profit to be made if you move in the right direction instead of just moving quickly. The latter tends to often be overlooked in the management style outlined in the article.
I think the healthy choice is to estimate, but not stress over the situation. Your estimate is not a promise, and make that very clear when you do give one. If your estimates are taken to be promises, it isn't probably an environment you want to work in for long.
It matters plenty of times. If Turbo Tax doesn't ship their 20xx edition by Jan 1 20xx, they're losing sales. I also work on (non-tax-related) software that is tied to specific dates by regulations coming into effect on certain dates. I don't think this is rare.
So if your "natural" release date was a few weeks after that birthday, it was definitely on that birthday. You only had a chance to escape that if your "natural" release date was months apart from the birthday.
But then again, this company never ever hired anyone without a positive graphological expert's report.
So, your company, generally, has to book all of that stuff long before the game is complete. So they go to the team and ask. When will this be done? The team says (in 24 months) and so marketing and sales get to work making all the deals for the things above and now the team has to met their deadline or all that stuff above is not only wasted, those people will never work with you again because you messed up their scheduling. This is especially true for retail space because it's not like they can fill the shelves with random stuff on a moments notice. THey've basically got a hole in their inventory and shelf space they would have filled with another product so you've made them lose money.
Much of this is also true for consumer electronics. Deals are made to feature the next smartphone at all outlets, and run commercials etc, months before the phone is ready to ship. All the teams making software for that device have to be ready to ship by that promised date.
But now you can turn on a national campaign in a few days. (Disclosure: I work with Blip Billboards who are one of the major drivers of this change.)
But we have ~1500 signs and each of them is playing a new ad every ~8 seconds. In other words, there's plenty of space. Those 8 seconds can range from a penny to a buck depending on the time and place and on the current competition. If you want to show up, you can.
I had precisely one meeting with the person who issued the project, during which he made various Agile noises. This taught me something I would later come to expect about those sounds: I met with the stakeholders precisely zero times before delivery, despite my repeated asks. I worked evenings and I worked weekends to deliver something on time that was perhaps a little better than expected.
And crickets. For five long months, nothing. I asked tentatively and was told I got paid whether or not it was in use.
So this strangled two values for me: Deadlines, certainly. If I hear a deadline, I evaluate from where it came. If it is a nonsense deadline, I lose respect. False urgency is just a way of life with some people. Additionally, the "getting paid whether or not something was used" is an excellent way to develop apathy in an employee, it destroys the value of engagement.
Sometimes real dates exist, and I'm all for that. Floods can be expected. But too often dates are picked out of a hat.
So projects that worked reasonably well and developed very fast, or even just somehow fast, are presented. Everybody agrees that's good and a success and better than if they were done slower; but so what?
Some are great major engineering projects, but in the context of extreme speed there is a world between a few weeks or even months, and more than 5 years, even if you compare the 5 years in question to project that are a parody in the other direction (yeah, a train project that is planned for 37 years looks ridiculous, but again so what? is it really because diva engineers want to spend all their careers doing "perfect" stuff and reject any deadline they are presented? doubtful)
Also, some are successful, but controversial (JS...); some I don't even know what to think about (is growing the population of a city by 22% in a year good?); some are just the work of geniuses.
And this is a mix of major physical infrastructure/engineering projects, projects for mass produced physical goods, and random software projects. Then suddenly the discussion focuses on physical infrastructure, ignoring all the other kind without even stating why.
This is also hard to contrast with the original article, because we don't know how many of them were mainly date driven. Probably not so much, in the sense that it would be retarded to insist on delivering something that cost a shitload of money to build at a completely arbitrary date without enough concern about the quality.
Too finish: some insane major infrastructure projects have been successful in the modern age: what comes to mind is for example the LHC, yes it took quite some time but it is probably somewhat more difficult to design than a random consumer electronic gadget? (is the story of the iPod that much spectacular? I found it quite common...)
Sure, we set goals which have dates attached. We usually plan out a whole quarter of what we want to accomplish. Our timeline is based on best guess estimates. We aim to deliver 80% of what we committed to. Sometimes we do it, sometimes we don't. The business rarely complains because they see regular value being delivered. When things aren't going well we communicate openly and regularly so that no one is caught by surprise.
> The testers keep running into situations where things don't seem to be right, luckily your product manager is happy to defer most issues raised unless it is a real "show stopper," as Jeff loves to exclaim.
That might have been the critical mistake. Jeff should not have done that. He should have insisted on lots of feedback loops during the project. He should have put more effort into convincing himself that the product is of sufficient quality.
As for the developers, they should have made it clear to Jeff: if he's not allocating resources to convince himself that the product is of sufficient quality, he shouldn't be mad at them when in the end the product doesn't work.
I think the author vastly overestimates the value and importance of project that most developers work on. I don’t think I’ve ever worked on anything that I could consider “high value”. I’ve worked in big (and small) companies, working on Big Important Projects, but if I’m being realistic I think the overall value and actual importance is actually quite low.
Other than that, I think the essay has a fatal misunderstanding - it’s not about setting deadlines, but it’s about estimating what it costs to do build software. This is useful because it helps us guess when it would be done, how much can be delivered, etc.
It’s the job of engineering management to convey to upper management that these dates are projections only with certain levels of confidence.
Every new hire at Toyota has to work on the assembly line for 2 weeks before they can begin their role. This applies from secretaries to vice presidents. How many software companies can say the same thing?
No one who manages my job is capable of basic web development tasks, e.g. centering a paragraph. The result is they are continuously confused and frustrated and our group is unproductive.
I was burnt out for a good 6 months after that. Even simple tasks took all of my will power to do. It was horrible. 3 months of the hardest work I've done and considered a failure for it.
1. We have this high level idea, we want to ship it by XXXX
2. Here are the features we want it to have that we think can get done by XXXX
3. Here are the features prioritized from mandatory to nice-to-have
4. Move the bottom 10% of features to v2.5
5. Move the next 10%-20% of features to v2.0 to make room for bug fixes, acts of god and management led boondoggles
6. Start development.
e.g. in the case you described, the engineers need to be able to communicate without fear that X1 and X2 are basically the same engineering work as X, and still can't be accommodated, and the other side needs to trust that they are not just being lazy.
"We can do X... I suspect we'll need at least N weeks after we have a proper scope/spec/X/Y/Z from you, though, so if you want it done by $(MILESTONE) we need it by $(MILESTONE-N) at the absolute latest, or more preferrably by $(PREVIOUS_PLANNING_MEETING) so we have some wiggle room. No, that's not a spec. Yes, you need to participate in defining the scope of this thing - otherwise it'll take even longer when we're forced to rework completed things to work how you actually wanted them, and longer still when we work on things we thought you'd find important but don't care about. When can you meet so we can get this properly defined and fleshed out? We need a meeting anyways to discuss what other things you want that we're going to have to cut or defer to fit this into the schedule as well."
If they work with you to cut/defer other things they've asked for and actually get you a scope/spec/X/Y/Z on-time, then good! You've successfully responded to a change in their priorities.
If they suddenly lose interest when they might have to do some work... you've probably deflected some high work low value ask. Also good!
If they put in the work but can't get you scope/spec/X/Y/Z in time... well, hey, that's not on you. You need enough time to get these things done.
They might still want 3 women to make a baby in 1 month... but they might believe you when you say that's not possible if you at least look like you're trying to work with them on fitting X into the schedule.
You can't just say "no, can't do it", because in all honesty, you don't know. What you can say is "based on our current estimates and priorities, it will not be possible." If they insist it is needed then they need to increase it's priority until it fits within the amount of estated work that can be done before X dates.
I think the underlying problem you are facing is viewing a set of tasks as a schedule to be set rather than a prioritized list of tasks with estimated completion dates.
I like this a lot more than the sticky style that most things use.
I know marketing lines to pretend that they can't do their jobs without hard engineering deadlines but marketing is overrated anyway. Ask yourself if you really need a deadline or if you're just slowing down engineering with your incompetent management.
1) > Giving estimates for software dev is 10x easier
No, it's not. If I'm writing software similar to another project, then I can estimate based on the previous project. But that's rare.
Instead it's something entirely different, otherwise I'd be able to juse reuse the previous project. In addition, estimating eng. projects like bridges is easy, because bridges obey physical laws, like gravity. That doesn't apply to software.
2) > the same management that's always responsible for everything that is wrong
Ultimately, mgmt. is responsible. However, mgmt. being at the apex can and do bury their mistakes, no matter how large. As we found out in 2008, there is no criminal penalty for mgmt. stupidity, regardless of how extreme.
A case in point is Boeing. They kill 350 passengers and damage America's trade balance, yet their new CEO is from the same board that was responsible. Mgmt. has no accountability, in the US anyway.