Deadlines are pointless – what to do instead
lucasfcosta.com
lucasfcosta.com
That might be moderately true for, say, someone developing a new app feature that gets shipped to users who aren’t expecting it. However, it’s not true at all when the software development output is a gating feature for real-world contractual obligations and other such business drivers. In that case, target dates must be carefully forecasted and estimates must be performed with great care and detain. This doesn’t guarantee that a target date will be hit, of course, but it’s not impossible to produce estimates that are reasonable accurate a large percentage of the time.
Estimation, planning, and progress tracking are difficult skills that must be learned, but they’re not impossible. I suspect many of the developers who believe that target dates are always arbitrary, pointless, and impossible to estimate have mostly worked in big cash-rich environments where it didn’t really matter when a feature shipped. If you go to the other end of the spectrum and talk to engineers who work in industries where target dates are critical you’ll discover a lot of people who are very good at breaking down projects and accurately estimating dates without relying on crunch time to get things done. It may sound impossible if you’ve never worked in an industry that requires such precision, but it makes sense after you’ve worked in such an environment for a while.
But in most cases when I asked managers about the reason behind the deadline, the answer was something like "oh,I promised this team/director we'll do it by then". When I dug deeper to see what happens if we miss the deadline, it appeared the said team would be just fine, there were no real life consequences, just that they wanted to finish their part of work by certain time, and they depended on us.
I'm not saying deadlines are useless, but they're very often driven by some arbitrary promises made by managers before even consulting developers about the complexity of work to be done.
Proper safety checks wouldn't let the plane leave the ground. Delays in that critical state are inconvenient, but you accept them because the alternative is not acceptable.
We should treat software in exactly the same way, It's ready when it's ready.
The issue is the illusion of control. A bad CEO will measure the performance of his CTO by productivity alone. A good CEO will measure the CTO's output by consistently increasing quality.
But that is how it is done??
The delivery date is an estimate and the date can shift until the plane is safe to fly. The deadline for delivery is not a true deadline and the factory will only work to its safe production limit to deliver the plane.
A bad practice would be to just put the plane out on the tarmac, regardless of state.
It works to the safe productivity limit of the factory, not the arbitrary demand of the customer.
And what makes you think planes don't have to safe for flight, with controls and sign offs, before take of?
Both Boeing and Airbus have missed a bunch of delivery dates this year and will continue to do so. For some of them they're blaming regulatory approval. Guess what, it happens, life goes on, people in that industry can at least recognise that it's better than flying a plane that hasn't been approved.
> And what makes you think planes don't have to safe for flight, with controls and sign offs, before take of?
They do, and again the point is: what happens if the plane isn't ready? It doesn't fly until it is. They don't ignore the checks because they have a deadline.
You want to do creative work without deadlines and real world consequences for missing those? Become an artist, a rich one preferably because sponsors and exhibitions are a thing in the art world, too.
For example, if the customer’s plane isn’t delivered on the agreed upon date, you now have pilots, flight attendants, mechanics, etc. that need to be paid for no work, or let go and new hires made / trained at a later date, which is expensive. It’s also lost revenue and market opportunity.
There’s also the potential that the customer cancels their order and buys a different plane from a different manufacturer.
Agreed you can’t put out a plane that’s not safe, but you can pay overtime, or shift resources between projects, etc., to produce a safe plane and try to deliver on time and limit the above negative effects on the customer.
In general, if you ask someone when they want something done by, they usually say “as soon as possible”, not because it’s arbitrary but because there’s typically real world value in completing a task / gaining access to an asset sooner rather than later.
Boeing and Airbus tend to hit their deadlines for supplying planes all the time. Where is this hypothetical universe where they refuse to give a deadline, or refuse to meet it?
Even for the most objective deadlines (contractual obligations, holiday promotions), there are alternatives to releasing a low quality product, and the pros and cons must be measured. Does paying damages for failing to live up to the contract beat the reputational (or even legal) damage of releasing a low quality product that technically meets contractual obligations? Then a responsible company will pay those damages. Does releasing the product on time for Christmas, even with the level of bugs known and/or potential unknown ones, gain you more good will than delaying it by a few weeks? Then uphold the deadline and promise a patch soon after.
The author I think could phrase the argument better by saying a delivery date (something that is promised, based on quality) is more useful than a deadline (something demanded, regardless of quality).
Delivery dates do not preclude working hard and bring prudent, but they are not hard stops. Rather they are good estimations that can and often do shift.
Sales team needs that thing by December 1st? Don't ask for it on November 29th. Put leeway into the model by giving time for the delivery date to be missed. It's called contingency.
But, depends on your philosophy - move fast and break things,or deliver consistent quality code.
The alternative the article proposes is entirely different - it proposes a work queue system, where the team is always working on the most important current task(s), and where every task that is taking longer than some pre-determined number of days is re-evaluated to make sure it is still as important as believed.
> Sales team needs that thing by December 1st? Don't ask for it on November 29th. Put leeway into the model by giving time for the delivery date to be missed. It's called contingency.
That's not how the article thinks. Instead, it views it as "sales team needs some task done by some date? move it to the top of the work queue as soon as you know about it and the team will finish it as fast as possible anyway". Will that be before or after December 1st? No one knows, no one cares: it will be done as soon as possible.
For inherently fixed scoped tasks (say, "our product must ask users for consent before collecting data"), this can work fairly well. But I don't think it is realistic for the vast majority of work, which is far more vague in terms of scope.
You've booked a flight on dec 1 at 3pm. You check in, sit at the gate. At 2:40pm you are informed that, sorry, the plane isn't actually there yet. It's still on route. We'll fly at 7pm instead. You sit and wait. At 6:45pm you are informed that the plane is there, but the airport is out of fuel. More fuel will be there tomorrow. Rescheduled for next morning at 8am. You wait. At 7:40am next morning you are informed that the pilots are not awake yet. They had a long day yesterday, then went for some drinks, they just had to sleep in today. Rescheduled to 11am. At 10:30am somebody tells you that, well, things are in place but ATC went to a team building event, all flights grounded, but will be back at 4pm. You finally lift off, 25 hours late.
Entirely preventable had a deadline be set and all necessary components be scheduled correctly to meet that deadline.
Deadlines serve a purpose beyond coersion. There is misuse, but if you widen your horizon then you may realize that not all of it is.
The pilot turns up drunk, she turns up on time, but drunk. Do we stick to the flight deadline?
A schedule is a set of events happening at certain time, right? They have to happen at that time since there is a contractual agreement on them happening at that time, and if they don't happen at that previously-agreed-upon time, then there will be negative consequences. That's exactly what a deadline is.
My schedule for today says that I am to meet a certain customer today at 3pm. We have a contract with them that stipulates this. In order to be there at 3pm I need to get the train at 2:10pm and for that I need to be done showering latest at 2pm. I can't make it otherwise. 2pm is a deadline for me. Bad things will happen if I don't make it. I can't justsay, oh well, quality just wasn't enough. I had enough time to plan ahead and be in the shower on time.
If you had to reexplore and reinvent flight routes every single time, yes deadline in that scenario would also become meaningless as you'd have no idea what lies ahead each time. It would be the same as setting a deadline on how long Columbus should take to reach India.
That's true. But the article is not titled "Why deadlines are pointless for creative processes." It's arguing that deadlines are pointless. Always. That's oversimplifying at best, click-baiting at worst.
I'd argue that even in a creative process, if you add enough of a safety margin on top of a conservative estimate, then a deadline can be expected to be met with reasonable confidence. This is absolutely necessary for doing business. "We release when it's ready" is a workable strategy when there are no obligations, but claiming that this always works in all other contexts is just not honestly considering other contexts.
“We release when it's ready" is focusing on quality only, and deadlines are just fictional dates, and “we release whatever is ready in time” is the real world process, where the product is bargained to whatever can meet the deadline (in the airplane example, that would be redefining your destination as wherever the plane is at the designated landing time, and see you in court if you ever make it back to civilization)
I think few people understand that setting deadlines equals to choosing the second strategy though. A mix of both just puts it in the first camp (“deadlines are flexible”)
Think about teaching your kids peogramming. Do you set deadlines on when they need to finish learning to read, when they'll touch their first computer, the date they need to have a working hello world ?
For any of these you'll probably be buying toys, books, discuss with your partner and arrange time etc. But none of those will be "deadlines", even if you intend to be very optimal in the progress
For instance infrastructure projects: a delivery date is set, but developpers blowing out that date is just a fact of life. What will you do anyway when your new dam or highway is 6 months late ?
Even at smaller scales, you'd schedule moving to a newly built house at a set date, but painfully know the developper can easily blow past that and you can't just take the date for granted, or you accept moving to an unfinished house.
Software projects are of course the same, at big and smaller scales. Just look at game delivery for the most piblic instances of setting "deadlines".
The same way setting unreasonable deadline will only bring unusable results, which won’t help anyone interconnected to you.
It all comes down to proper management, there’s no magic way out of it.
Yep! It sure is.
FWIW, I've spent most of my career producing custom software for paying customers. I've never missed a deadline because the the contacts I've worked on were negotiated appropriately. I've worked a whole total of 1 hour unpaid overtime in my entire career (and that hour was completely my choice).
If the people negotiating these contacts didn't know what they were doing, the companies I've worked for over the years wouldn't be in business anymore. I'm glad I've only worked at functional organizations.
So, yeah, the first few sentences of the article were completely foreign to me, yet the author claims, with confidence, to speak for everyone. Didn't bother reading the rest.
Deadline & No Fixed Scope. Fixed Scope & No Deadline. Deadline & Fixed scope and padding appropriate to uncertainty to hedge risks.
All 3 are admissions you can't estimate accurately.
Of course, the further you go out, the more your error margins (and the less others naturally rely on your estimate, and the more likely you say something generic like coming soon in 2025)
But this notion that it’s all creativity born of the ether, as if you’re sitting around waiting for eureka to suddenly strike is complete bullshit.
There is the issue that external resources will latch onto your vague estimates like it’s the word of god, and try to hold you accountable to unreasonable degree, but that has nothing to do with your planning. Usual solution there is to figure out what feels reasonable, and then double it again for your boss — and double it again for the client (which you’ll probably lose a bit to negotiation). There’s also the issue that a single heisenbug can take stupid amounts of time to resolve… but that’s why you doubled up — you keep buffer for that kind of thing.
The vast majority of any project is just plumbing — creative or not.
The point I’d still raise is that very few organization are OK with actually going the full length to reduce the error margin to what they want to tolerate (they still end up eating the costs associated with the actual execution time, of course)
For instance: you integrate a new API from a new vendor, and a request is made on a time estimate. It’s probably a very bland task with no crazy thinks to do, except you have no idea what surprises you might get actually running the API. What if the vendor din’t account for one of your use case for instance ? (let’s say they round money amounts to the nearest cent while your business requires round ups everywhere instead)
The safeguard against that risk is to either: fully run the API beforehand and cover all your use cases, or document every single requirement and run them through your vendor. If you’re not a bank nor the NASA, you probably won’t be allowed to do that before giving an estimate that will set a deadline.
It’s to me one of the reason why outside I put “creative” in quotes, as the problems we’re trying to solve don’t need to be complex, unexpected is sufficient. And ‘deadlines’ are more probably more ‘guidelines’ as most orgs don’t really care to secure them.
I'm working on something truly novel to me every day. The fact that it's novel to me is enough to make any wishful planning arbitrary.
The amount of brain waste caused by agile is immense. So many minds toiling away, trying to pick apart some long lost engineer's cleverisms and build something mundane on top of them.
This is pretty much what regularly happens.
Plane was supposed to leave at 2:00 PM. We all got here at 10:00 AM for the long check-in. Got word the plane would be late, so 3:00 PM. Then 4, then 5, then 7, then 9:00 PM, finally took off at about 9:30 PM, over 7 hours late.
It turned out that this was because the incoming flight on this airplane was a 1st class-only flight, and the airline had failed to allocate time in the schedule to swap out the seats so there would be enough for the regular-class passengers. On arriving in Santiago, we had of course missed our connection. So, had to be put up overnight for a flight out the next day. Basically arriving a day late.
Had lots of other examples, but indeed, deadlines make no difference whatsoever when the plan fails to match the reality.
It is the plan that is important, not the deadline.
I can also say from developing software for several major events that occurred at specific times — there is no slipping of the date, ready or not — that the process is much more oriented towards managing the balance of development and scope than all-nighters. Sure, there's a few of those, but the overall key to the making the deadline is having a manager who can keep pruning the tree and keeping distractions at bay.
An airplane better arrive before running out of fuel. Somewhere, at least. That's a real deadline. You can't negotiate yourself out of that one. Empty tank == you will land NOW. One way or the other. So this is all full of safety margins, extra fuel, alternates, etc etc. But there is a time span which every flight will not exceed. Guaranteed. The pilots better be prepared for this, otherwise they die.
A flight is usually structured with many "lesser" deadlines. Latest time to abort takeoff. Latest time to turn to an alternate. Latest time for this-or-that to make for a safe landing. Glide slope. Localizer. The whole procedure to land a plane is full of deadlines. If you miss one, you execute a go around. Until you are out of fuel, then you better nail it.
So, no. Deadlines are not pointless. They can make the difference between living and not living.
Erm, except air travel DOES run like that.
Deadlines are at the very core of air travel.
The crew agree on a flight plan and file it.
If the aircraft goes AWOL and does not arrive at its plan waypoints by time + buffer, then all hell breaks loose.
There are also deadlines for landing slots, deadlines for flight closing, deadlines for this, that and everything else....
Aviation without adherence to deadlines would not work.
As for maintanance, airlines keep a ready stock of parts and have relationships with maintenance organisations at all major airports. There are also regular scheduled inspections. An aircraft on the ground is a loss making asset, it is in the airline's best interest to maintain the to flightworthy standards at all times.
But needing a new working iOS by ship time for the next phone? That is a pretty hard deadline, and why apple doesn't use agile. They say agile doesn't support synchronous hardware/software releases.
The problem is when I bsupport multiple efforts, some with deadlines and aome without, the ones with deadlines always get "done in a minute" descoped so that I don't starve an ongoing que as is described here.
Imagine more if the airline promised a flight would be ready using a brand new plane that was just now being designed out or at some stage in development someone unknowledgable felt comfortable to start making promises about timelines and even extrapolating out production. New plane designs and manufacturering take years and years sometimes. Have all sorts of delays, have safety issues that require them to redesign things later and so on. You don't see a lot of new plane designs you see the same known things used over and over.
It's one thing to make forecasts around fairly well known, established, and mature systems and processes, it's an entirely different beast doing so around less known systems and processes. The vast majority of software engineering and development is done around something new or different that has little contextual reference. If you do have reference then you're rebuilding the same things using the same pattern and it won't be long until libraries, frameworks, systems, processes, etc. emerges to automate that at which point your time shifts to the new things you need to automate which haven't been automated or done yet (again, otherwise you'd just leverage the prior work).
Planning to make a site using something like say Django vs building a new framework with different features like Django are very very different and a lot of development is trying to do something unique and new which means high uncertainty.
You build process around things to make them happen. Nobody is waiting for some artisanal process to magically produce a plane. There’s maintenance and cleaning schedules, crew schedules, etc. The preflight check is a final verification.
A good CEO measures the performance of the CIO by availability and quality of services. The CIO looks downstream to ensure that the operations teams have appropriate processes in place.
The problem is external artificial pressure almost never helps.
Say you book a wedding with your fiance and on the week prior ask her if there's any reason behind the wedding date being when it is. And if you dig deeper you'll see no "real life consequences" will happen to either one if you miss it, either, but I'm guessing she walks over missing this "arbitrary date".
The fact that nobody dies or a regulatory body doesn't fine you straight away, or as you put it, if it's "arbitrary" doesn't mean much. The industry your whole company is in, at some point was also an arbitrary choice by whoever started it. We're humans, pretty much everything we decide to do is arbitrary.
The problem is when you treat deadlines as something very important and absolutely essential to meet, especially when the dates have just been made up, without even having the full picture of what must be developed, way back.
That's how you get crunching and people having to sleep in the office. That's how you get working until 1 AM just to ship something so the system can go into prod on this made up arbitrary date. People who do this sacrifice their own wellbeing and those who don't no longer seem like team players, leading to a culture that's unhealthy or toxic.
This is especially noticeable in the video game industry, where it's become a widespread problem - even by outsourcing to lower paid developers and getting a bigger workforce as the result, the deadlines for game releases are still unreasonable and you ship buggy games (e.g. Cyberpunk 2077) on the backs of delusional young people who work too much and burn themselves out.
If you put those conditions in place, are you still against deadlines? Because we've tried not having them, and collectively agreed it's worse. A team that has no deadlines performs worse than a team that sets deadlines for itself. You might've noticed this in your personal life with your own motivation.
For anyone that doesn't want to ignore the fact that humans are way more productive with deadlines than without, the question is what do you do when it looks like you're going to miss the deadline, so that this balance is a healthy one between motivation and doesn't go into overwork. In this mode of operation "meeting the deadline" means cutting scope instead of working more hours. It means pulling another team into the project, it means leadership being OK with cutting scope, etc.
But pretending like we're all intrinsically motivated and deadlines are harmful is papering over the fact that what's harmful is how you deal with failed deadlines, not the fact that you're aiming your work to be completed at a certain date.
In a business, a deadline should have valid a business case, one that’s not arbitrary - does it help improve the bottom line, or not? One senior manager thinking he will look good to the C-suite by having it done by a certain time probably doesn’t. Agreeing on a rough date with an external client, who have their own considerations to take into account, probably does.
The people setting the wedding dates are the fiances, who are the ones organizing it and are also the main recipients. In that case they are both the vendor and customer, it is pretty much like a DIY situation, even if there is some outsourcing involved.
Now I won't say deadlines are bad, but they shouldn't be set and promised without first getting a careful estimation, which is what the parent comment is criticizing.
- deadline set randomly or by gut feeling of a manager = bad (more often than not)
- deadline set after careful estimation of resources involved, their own estimations and possibly the number of side projects that might have to be rescoped/paused for that purpose = good
However once you agreed to it it's no longer arbitrary. It's the agreement. That other part of the business that changes their accounting software or replaces the machine on the factory floor has agreed with you to ship by that date.
Arbitrary but committed to in a sense that would be detrimental to some other part of the business if you miss it. That's not quite completely arbitrary.
That sounds like evidence for deadlines not being pointless.
Someone, somewhere, depended on a certain ETA. That's a deadline, regardless whether you think that there are no consequences for missing it.
Imagine, for instance, writing software for an interplanetary probe with a short launch window.
Pretty different from writing the 'Yo' app.
[1] https://lucasfcosta.com/2022/08/03/talk-to-your-customers.ht...
> If your team can’t ship without deadlines, the lack of deadlines is not the problem. Your team’s careless attitude about customer feedback is.
> When teams obsess about customer feedback, they ship as early as possible so that they can listen to their customers.
> For customer-centric teams, artificial deadlines are useless because the only “deadline” that matters already exists: “as soon as possible, as long as it’s valuable.”
You're choosing to conflate listening to customer feedback with Agile.
HE draws that link... >In this post, I’ll present a robust and disruptive Agile framework. It’s called “talk to your customers” — or, in short, TTYC.
You can set out to have a set of features, and you can have a deadline, but thinking all the features will be completed (and completed well), on that day has never worked (in my experience). You just wind up with rush hacked garbage towards the end.
This is what I’ve done successfully for years now (when I am not forced by a PM to do agile): https://concentric.works/
You can have a deadline but no fixed scope, this what CI/CD is for. So when deadline day happens what you have is already released.
Or you can have fixed scope but no deadline.
Or you can have fixed scope & deadline, then pad things to hedge risks that might pop out.
Agile prefers the first.
And almost never does because it's designed in a way that's easy to abuse. Instead you often see dynamic ever-changing scope, dynamic ever tightening deadlines, and no margins for risk. Everything is on fire always and you just chase the fires around with your dev teams to put out fires, crack whips, and ask why features are behind schedule. It's madness in much of the industry.
Maybe, but I am not sure, and that’s not the way I’ve ever seen it done.
The only way I’ve ever seen agile done is “waterfall with two week sprints”. Large wireframes of an entire application which is “unreleasable” until the end.
Sure it compiles during each sprint, but it’s not a releasable product at any stage (until the mad rush at the end when all the negotiations and trade offs begin)
I am yet to see this done well, though hearing that it's apparently possibly has me cautiously optimistic.
In my experience so far, however, I've mostly seen incomplete requirements (one could argue that a perfectly accurate spec would be no different from executable code, the end product itself), changing requirements, difficulties in estimating the impact of needing to migrate old data or accommodate/refactor old logic, little proper estimation of testing overhead, the impossibility to predict how long working with external services and integrating with them would take, especially when cross coordination with another team is necessary, the impossibility to predict what might break in regression testing or additional edge cases that might be discovered, as well as the greater impact of needing to utilize particular tech stacks (e.g. Kubernetes with a particular service mesh) due to external requirements.
It's gotten to the point where I feel like adding a new form to any given application might take anywhere from 2 hours to 2 weeks, depending on what's going on in it (e.g. a well architected and simple monolith with onboarding guides and commented code vs microservices with little to no documentation, to serve as examples of different orders of magnitude of complexity).
Estimations feel like just making stuff up on the go at best and like something actively harmful (being urged to make promises that are going to be false a lot of the time) at worst.
The only way to do them would be not to do them - instead have some software look at the issue request and say: "Okay, you have X requirements of the following type: ..., after analyzing this project's issue history and time tracking across 5432 issues, it would appear that this issue would take between 4.2 and 7.3 days to resolve."
To me it seems that if you need to be able to give accurate estimations, you'd also need a very predictable and boring tech stack across many if not all of your projects, which doesn't seem feasible in web dev, given how fast everything seems to progress and given how much CV padding one must do to remain hireable (instead of, say, using jQuery in 2022).
Judging by the constant stream of overdue and overbudget projects in more traditional businesses like construction, which are significantly stricter in terms of regulation, planning and have been around forever compared to software engineering, I'm not convinced accurate deadlines are possible.
We’re there some serious overtime crunch periods on a couple of them? Yes, but for a multimillion dollar project with expiring Federal funds, it was achieved as set forth.
I think we tend to hear more about the failures than the successes - the Omni hotels in Houston and Dallas built with bond funds were repaid years earlier than mandated because they were successful, for instance.
I do believe that estimates, even for complex projects, are useful and can be fairly accurate, and in a lot of circumstances they can be used as the basis for deadlines, with certain considerations.
Let me give you an example of a real life consequences deadline where the project is state financed with money provided by EU, the end of the current political party term is nearing and the money is available on the premise that the project is finished successfully. "Is done" is actually approved by a third party inspector who not necceseraly works in your or the project's best interest. The project done by several separate subcontractors and if fails the political party looses faces before elections and if it fails to deliver the money provided by EU won't ever arrive and you won't get a dime. But you will pay the consequences of a failed contract.
I was an IT pm/ba/architect for one of the subcontractors. This project gave third of our annual income. Should we miss it it has the chance to bankrupt the company and as a bonus you have an angry political party against you.
Go, plan it, estimate it, deliver it on time.
I agree that every deadline is artifical and negotiable but EU financial support deadline is a fuckin hard deadline because you cannot possibly have the necessary network and position to initiate any kind of change.
So deadlines can be a lazy, dumb or crude way of pressure and may not have the intended effect. Surfacing the real reasons and dealing with them directly might be more effective.
Cool, I hope that whoever agreed to these obligations knew full well that they were taking a gamble and are prepared to deal with the consequences of the gamble not playing out.
A complex software project is already a gamble. It may drag on for years even after the most pessimistic time estimate. it may totally fail (the outcome which is rarely taken into account). Drawing an arbitrary deadline on top of it just makes it a more risky gamble.
> You can’t simply start training today and run a half marathon tomorrow, for example. You can, however, create a system in which you consistently increase the number of miles you run.
The marathon will be on a specific day, ie a deadline. If you haven't finished your training before the day of the marathon, you fail.
How long you have until the marathon would determine what kind of training schedule (system) you create.
Doesn't the same logic apply to the training system that we created based on time until the marathon? Does the existence of a deadline guarantee success of a training system?
And, doesn't any training system increase the number of miles consistently? Because, if I am to run 26 miles, any training regime should make me able to run 5 before 10, 10 before 26.
I feel one of your misconceptions is that running a marathon (or a half as in the article) is something binary. A lot of people could just go out and run a half marathon tomorrow if they were forced to. The injury risk would be high and they would feel miserable, but they would make it. A full marathon requires much more training but all marathon training programs I have seen are overkill if the goal is just to complete the race. They are instead aimed at either making people complete it at a certain pace or complete the race while being able to enjoy it. If you have a lack of time to train you can sacrifice pace/enjoyment. Rushing the training on the other hand is stupid.
So for marathon training the deadline is in the form of a hard cutoff where you have to decide if you are satisfied with that little training or if you want to wait until a later race. It is more like a release train which only happens a couple of times per year.
Edit: Sorry for the rant, my real point is actually that you are encouraged to change your training plan on the fly to adapt to real life events and that while the race is a deadline it is very unclear what will be delivered at it until you get pretty close to it. Will it be a 3:30 marathon or a 3:15 one? Unless you have sponsors nobody will get angry at you for not delivering what you had planned to.
The marathon happens on a specific day. You can train however you want, but if you want to run the marathon then you need to be ready on the day of the marathon. Deadlines exist.
So, yes, races are deadlines but not deadlines you can rush towards because then your body will break and then you will fail the race for real. You still need to slowly and consistently build up your body just like if there was no deadline at all. It is much better to not have finished your training than to be injured.
> You can, however, create a system in which you consistently increase the number of miles you run.
This is indeed how you become a fast runner and how most training plans work, except they often peak at a mileage most people do not want to sustain to top the form right before the race.
There's an old saw that goes something like, "fast, cheap, good. Pick two."
I've found that to be true over and over again.
All three options can be useful depending on whatever it is that's being worked on.
Many of the complaints in this discussion about setting deadlines seems to be that not enough resources are being allocated to meet the "fast and good" selection. Which, of course leads to an end result of "fast and cheap," which isn't the goal.
The main argument seems to be that "cheap and good" is better than "fast and cheap" because there aren't enough resources allocated to achieve "fast and good."
The solution, of course, isn't to eschew deadlines, but rather to determine the work required and make a decision about how the work should be done (with the above being a simple-minded way of describing that 'how').
It seems like deadlines become an issue when those setting such deadlines are focused on the "fast and cheap" pair, while the folks actually doing the work want "good" to be part of that pairing.
As someone who has worked both in environments which focus on minimizing time and cost when bidding for work, and in those where estimates realistically reflect the time and cost of completing the work, I find the latter to be much more effective.
The former is generally a shit show where no one is happy. The client either doesn't get what they need, or it's going to cost a whole bunch more than they were told, and those actually doing the work are pressed to work harder and faster because the client is pissed off.
And the business generally loses in the medium to long term as well, as they're unlikely to get repeat business and certainly won't get referrals from that client.
When good estimates are provided and the appropriate resources applied, projects often go much more smoothly and everyone wins -- over the short, medium and long term.
Obviously this requires good planning and a strong understanding of the goals and requirements and translating those into reasonably accurate estimates. Those (as others on this thread pointed out) are skills that can be learned.
Sadly, those skills seem to be in short supply. And more's the pity.
Edit: Clarified my prose.
I'd go for an old school MS project wielding project manager any day of the week.
Working on the edge of bankruptcy for years, never seen my “push hard this week (or: this sunday) it’s important to us” taken as important or used right away. I even learned to detect with high probability when the rush is based on cocaine and can be safely ignored.
This is not a rebuttal (I agree that deadlines are useful), it’s just the back side of this shiny coin of “effective management”.
The one thing I hate more than that, is articles such as this one that waffle on for eternity using pseudo-science.
For example, what do all the pretty graphs and paragraphs of waffle about coin tosses have to do with the price of eggs ? Its basically space padding that doesn't add anything to the argument they are trying to make.
So I would like to know how your development environment deals with this.
That is exactly what many software engineers accuse of quite some project managers not to do.
If you want to plan for real hard deadline with a fixed scope, you need to be giving it A LOT of time to account for unknowns and feedback cycles.
I've worked in environments like you say, all they do is massively pad estimates. That's not accurate estimates, that's just hedging risk. Estimates should be taken with a pinch salt, then risk should be hedged. But they're separate things. You can't just massively hedge risk and say estimates are accurate. In fact that's more of an admission they're inaccurate.
From my POV, I’ve been in the tech biz for a long time and never had an issue with managing deadlines. Sometimes they need to slip, but overall, we seem to middle through.
Planning how long something is going to take when you're completely in control of your own schedule is difficult but not impossible. Planning how long something is going to take when you are also expected to drop everything for every meeting or production problem that comes up is impossible.
One can only estimate things done before. By the same person. One the exact same machine. In the exact same circumstances. And even then, the % savings of only replication after every known unknown has been eliminated, is mostly guesswork.
The truth is also, that most planers constantly overrun there own project planning deadlines and the promises of their projeccts. But hey, the pope forgives himself, thats the easy part.
I wish the likes of you would for just one year walk in those shoes on whose toes you usually step to apply pressure. And no, repeatetly asking for a better timeline on nailing a pudding to a wall, does not get you closer to reality. It only gives you a metric on how fast somebody insufferable gets to hear whatever he wants to hear to dissappear. And your planning and all the deadlines, are not even part of the after project renegotiations usually, were whats actually was demanded in the final form and all between forms - and what was missing is broken down into kharma and money.
For constant change is even in physical projects, when things do not work out. Usually the planners are then "taken off project cause they are needed elsewhere", aka the grownup need to find a solution and they need to find it fast and efficient, without additional barking waking the real mean dogs.
Deadlines are far better. Tell me something needs to be done by next month it will be done. I can negotiate inside these bounds to prevent scope creep because a deadline is far more flexible than points. If you find yourself done with deadline work, you can refine and add touch ups. If you find yourself done with pointed work the PM will just assign you more work to "keep your points up" instead of allowing you to do the same.
The tyranny of the agile PM hinges on convoluted point systems. Even the one the writer prescribes. Force the management to actually agree on a deadline that is reasonable (not ship me facebook next week) and watch how productive and accurate developers can actually be. Read the comments here. Agile has been butchered to the point it's unrecognizable. Give me deadlines.
I left 2 jobs because of rigid scrum, couldn't handle the insane discussions.
"how many complexity points do you estimate?"
"euh, what was our velocity?"
"no, you can't think in hours, give me complexity POINTS!"
"I dunno, probably 2 to 3 weeks of work, to many unknowns"
"unknowns? we refined this story, there are no unknowns!"
"ok, 3 weeks = x points"
"no, you can't say 3 weeks, we have 2 week sprints. You should split the story in 2, where each story is a well defined functional component which we can show in next demo"....
"but, it is 1 thing, you can't split and have it working for next demo..."
mind torture... so I left
Nowadays I have a very experienced team. We do daily standups (without a scrumboard, just real conversation), we define deadlines, can be a week, can be 2 months. Demo's only when needed, kind of the way I used to work before rise of scrum :)
Every single sprint planning meeting and every single retro around half the time was spent talking about points, discussing estimates and commenting on our velocity had changed.
But we never had even one conversation where we put any of the 'data' we had about how many points we could do in a week to use to decide what we should focus on or when we should promise something by. The whole system was upstream of nothing.
Of course, this was a tacit recognition that it was worthless, and if we had used it, it would have been worse than nothing. But why didn't this acknowledged uselessness allow us to drop it?
“OK I think we agree on the scope of this task. How many points do we think it is?” “Well, that depends if we include X” (it turns out, we didn’t agree on the scope of the task!)
You can’t anticipate everything, but I’ve had countless valuable discussions off the pack of pointing that resulted in things like: - agreeing on whether to include an anticipated set of clean-up as part of a task, that the team might have missed or disagreed on later - realizing a task is bigger than anyone thought or maybe could be parallelized, allowing the team to approach it more effectively or reprioritize to deliver more value
Basically pointing results in a ton of timely & valuable insight that helps the team learn faster and row together better. There may be better ways to drive these conversations but I find pointing to be a pretty good driver.
That said, points and valocity are not upstream of nothing - predictable velocity lets you know when to stop sprint planning. It lets you have these discussions on a reasonable cadence and not waste time fleshing out the tasks you won’t have time to touch until next sprint.
Which is it?
For the latter, my experiences and expectations seem so different from yours that it's hard to know where to start. I haven't experienced much full on Scrum to disagree.
The nice thing about a deadline is the tech illiterate understand them just as well as the people building it do. You can check in and have stand ups about the deadline all you want. You can even track work in tickets. Just no pointing. Ever.
I like the way we've set it up at my company. In product/engineering, we use our estimates and plan our projects in quarters and project more or less based on how many points teams have completed in the past.
Outside of product/engineering, we report milestones on our projects with expected delivery dates. We don't mention the points because they mean nothing to the rest of the business without context.
The only big problem I have with pointing is when those numbers are used as performance metrics or judged by outside stakeholders without context on the work. I think we've solved for that and get to reap the benefits
A deadline is a clear moment in time when the feature is needed. There's a reason behind it. "We need to be ready for event X", "Next billing period is in a week".
Fixed deadlines allow you to negotiate the content of the feature, and make meaningful choices when it comes to the amount of details and edge cases to handle.
Otherwise how are you supposed to estimate something? I can make you the same feature in a week or in a month. It's going to be more polished in a month, of course, but I can squeeze it in a week if needed, with limitations.
Embrace deadlines, you will love them.
My current role had flexible deadlines. It makes me crazy. All that allows for as unchecked scope creep. So I can't estimate, because the teams requesting the work refuse to define a project's final state (or are incapable?) and it's often impossible to make arbitrary deadlines because they're not real either. I get almost nothing shipped. And when I do, it's literally $7k over or $6k under!
I'd say 75% of my dev work is hard deadlines.
Well, in reality, estimates are deadlines that they want you to feel like you came up with in the first place. You don't have to play that game very many times before you realize that they ask for an estimate, then try to argue the estimate down to whatever they had actually budgeted in the first place - but make it seem like it was your commitment in the first place.
At my previous gig we always had deadlines and we never missed them (technically, stuff gets de-scoped). Before a project is even started, time is spent doing a "feasibility study". The hardest problems are usually solved then (in isolation), tech stacks are evaluated and chosen, any hard algorithms are written and bench-marked to see if they scale, etc.
The deadline is padded to hell to allow room for stuff to slip and scope is clearly defined. If/when development starts to slip, stuff gets de/re-scoped pretty quickly to make sure things do hit the deadline. The deadline was pretty much never arbitrary, it would depend on when a different project(not software) would start/finish so that the next one could use it. If all hell broke loose and the deadline was somehow too optimistic, then the deadline would move to the next time it was a good time to launch the software, not just a few weeks (since no one cares at that point).
A specific example, we made a tool to analyze production test data for semiconductor devices. tl;dr a few thousand devices per wafer, _many_ production wafers, _many_ tests per device, basically "big data" (trillions of rows). We spent a few months evaluating different databases/schemas until we found one that worked. We knew pretty much all the queries that people would want to run against the data and made sure they would be fast.
The tool was meant to replace a vendor tool we paid for. The deadline was for when the contract was meant to renew. If we missed the deadline, the deadline would have moved to the next year (next contract renewal). This wasn't a software company, a lot of processes were pretty messed up, but the stuff the author complains about was pretty good. Probably because it was a semiconductor engineering company that always hit deadlines to get stuff to the fabs on time.
The downside is of course is that you are a "cost-center", not great for career development.
I call those “deadlines” time boxes. I believe that the industry would greatly benefit from distinguishing the two.
Interesting, I've not heard time box used in that context. I've seen it used to limit the amount of time you spent on something. So if there is some tricky bug, you would timebox it to 3 days one week. Regardless of if you fix the bug or not, you stop working on it after 3 days even if you don't deliver a fix.
In what I was talking about, you still delivered _something_, it might just be less than what was (initially) agreed upon. Normally it's polish/ux niceties that don't make the cut and then come in a later version.
Seriously. I work in proposals and deadlines are not negotiable - well, unless the solicitor changes the deadline, but it’s up to them. Whining about the pressure that comes with deadlines is pathetic to me, because I make my living whipping people into shape to deliver on time. I have all the accountability and none of the authority in my role - sound familiar to software engineers?
The difference is that my deadline work is people management and software is working with a machine that does what you tell it to do. People don’t do what you tell them to do, at least not consistently. They try, but unless it’s a priority, it’s hit or miss.
The reality is you have deadlines because your customers have deadlines and problems they need solved. If your sales team and executive leadership consistently puts you in positions where it’s difficult to succeed at what you are capable of doing, start your exit plan. Just because shit managers push their terrible deals downward doesn’t mean the system is at fault. Deadlines are healthy to keep progress and be able to move on to another new challenge.
Besides, we humans are on a deadline to get all our living done. It’s as ingrained as yelling at clouds or tilting at windmills. Get busy livin’ or get busy dyin’
I really think that, unless we're doing very repetitive stuff, the best we can do is give rough estimates on small deliverables, i.e. stories in a sprint.
If what you write is true - deadlines are important - and what I write is true - it's almost impossible to estimate how long it takes to develop complex software - then there's a mismatch that needs to be addressed somehow. I thought that this was what Scrum was designed to solve (by continuously updating estimates), but you seem to think that we need harder deadlines. If so, I'm not convinced that this is actually feasible in most cases.
And this is actually part of the issue (and in part what the article mention). Most project, including the one of the clients don't need deadlines.
I never saw a project where the deadline was not arbitrary or govern by another project deadline who was also arbitrary. I also worked in many different context, some with hard deadline, some with 0 deadline.
Almost all the time, the project with hard deadline stem from the fact that the management didn't trust their engineer. They taught that without this "pressure" we would just sit on our asses and not do anything.
While I certainly encounter co-worker who would gladly not do a thing and be paid, almost every engineer I worked with was happy to work and took some pride in his job.
The no deadline / soft deadline project usually faired much better with a much more happy team and client. First off there was a mutual trust between management and the team "on the ground". They trusted us to do our job and we trusted them to steer the project direction and give priority. This made the job just that much more satisfying. Secondly, there was almost never a period of rush / crunch. It could happen that we had to rush a feature, but it was infrequent, short lived, and never to the extent of putting an all nighter. This means that the team had a very consistent and predictable productivity, we didn't have any burnout and people where not likely to leave in the middle of the project, so we were never short staffed out of nowhere.
Deadlines are useful to me, however I have always advocated the use of periodic check-ins to give us a chance to adjust our course.
It’s weird when people make claims that are so easily shown to be wrong, as this author does, except some of that might be him using “deadline” to mean something less heuristic and more draconian than I do.
I published a regular feature in IEEE Computer magazine for 12 months. I never missed a deadline. But the way I did it was by using a series of fake deadlines (up to three). I asked the editor not to tell me what the real deadlines were. I missed at most two of the fake deadlines during that year. The system worked well for me because I don’t like to stop daydreaming until the last minute. Daydreaming and playing are divergent activities that generate value; but convergent activities are needed to actually deliver.
If you cannot schedule and execute properly, you are out of your league. As a professional engineer, I stand and stake my reputation on meeting agreed deadlines. Otherwise, it’s a sign of amatuer work or even worse, unprofessionalism.
I’m cursed/blessed with an Engineering mindset, and expect the same from my peers.
Deadlines aren't inherently evil. They're necessary. Abusing deadlines by setting them unrealistically, or by using them to try to goose production speed, is evil.
Or from another perspective, the probability a given team can successfully finish a key project deliverable diminishes after a projected time. If the hard parts of a problem still remain, the progress has in fact stopped when resources start being diverted to non-critical tasks.
The statistics indicate if a person says they are 95% certain, they are in fact 60% certain, and unaware of their bias. Good luck convincing the Dev-ops crowd though... being busy means progress to some folks. ;)
In my opinion the biggest problem is not understanding that estimates are constantly changing based on continuous change of information as the project is worked on. Also, I've rarely been in a team that takes the time to define a unit of work, which is relative to the stream of work that is done. Breaking things down into work units allows consistent measurement relative to who works on it, and then allows for pretty accurate relative estimates, which is good enough to predict completion dates. I suspect this is harder to manage, but if done right, predicting the completion date accurately (despite any deadline) is a necessary step to then speed up that process.
That's the problem. Of course you want that. Everybody would want that. Our lives would be easier if we always knew exactly when something would be finished and how much it would cost.
What you don't seem to recognize is that if predictions could be made with that kind of accuracy... you could be replaced with a spreadsheet. The only reason you're still around is because our work is inherently unpredictable.
For me it starts with predicting accurately units of work, regardless of desired completion date. Then work with that to speed things up if needed, and use that information for estimation.
> Instead, we must ensure the system is always processing the shortest, most valuable items.
Typically the most valuable items are always the longest leaving this criteria impossible to evaluate. Do you do the most valuable? Or the shortest?
> Imagine you have two sales teams. One produces unreproducible and unpredictable revenue spikes, while the other generates consistent and continuously improving results. [...] Which team would you rather have in your organization? I’d definitely prefer the first
I think he means the second.
The labels of the X and Y axes on the graphs for the coin tosses are transposed. It's not possible to have a negative number of coin tosses.
At the start, he also mentions training for a half marathon but then running a marathon later (not a half).
https://www.joelonsoftware.com/2006/08/10/the-identity-manag...
He was sort of leaning towards April 2007; I was sort of leaning towards December 2006. [snip] What I explained to Brett, though, is that we want to hire six new people in the spring, and the chances that we’ll be able to afford them without FogBugz 6.0 are much smaller.
There are deadlines and deadlines.
Take some sporting event for example. Let's say F1 competition. You must have racing car ready by the start of the season. Having it ready two weeks latter is not an option.
Particularly bad example - most organised half-marathons occur on a specific date, which means that there is a deadline for the training.
What the author is actually saying is 'unachievable deadlines are pointless', which I largely agree with.
For one thing: "In a system that awards bonuses to those who reach a $50,000 deadline..."
Not a deadline.
Sure, deadlines add "pressure". But we can't wait forever for something to exist. A feature needs to be shipped at some point.
https://www.thisamericanlife.org/776/i-work-better-on-deadli...
This is just fundamentally not true for human beings. Certainly not for all of them; I know personally I need a bit of time pressure to perform my best.
- career growth
- compensation, bonuses
And that is by design.
Keeping people always in a rush is a way to distract them from their growth aspirations.
I’d definitely prefer the first"
The first team that was introduced was the erratic one, did you mean the second?
> When teams miss a deadline, it’s too late to take any productive actions.
> Think about the actions you can take once your team misses a deadline:
> 1. Set a new deadline > 2. Fire someone > 3. Cut scope
Obviously, if you only discover that you missed the deadline on the day it passes, you can't do anything productive about it. But no one (competent) works like that: you adjust your work and your scope based on the deadline for the entire duration of the project. Starting from basic design ("we need to ship this feature in 2 weeks" will lead to a different design than "we need to ship this feature in 3 months") to design/scope adjustments ("we've been working for one week and we can see this is not going to finish in one more week, do you want us to adjust the scope/design, or is it preferable to push the date further out?").
I suspect a root cause of this problem is the following idea, from the "Queueing disciplines section":
> We can model every software development process as a queueing system. In such a system, tasks come in on one end, and software comes out on the other.
This is a fundamental misunderstanding of how software development works, at least for the vast majority of cases. Tasks are not an input of the software development process, feature requests are. Tasks are an intermediate output - you have to put in work to refine vague feature requests into concrete tasks. The same feature request can be translated into many different tasks, depending on various factors (competition, expected timeline, team experience, external dependencies etc.).
For a concrete example, say the team is working on building an alternative to Wireshark (a packet capture and analysis tool). A PM may ask the team to add support for QUIC to the product. In the ideal case, the tool would be able to understand every kind of QUIC packet, partially recognize some of the more common malformed packets, support decryption of the contents via one or more popular key logging formats etc. But, depending on product architecture, this may take half a year to fully implement.
If the PM desires some minimal support for QUIC in, say, 2 months (perhaps to coincide with some fiscal quarter), then they need to work with the team today to find a meaningful subset of features that can be implemented and delivered in ~2 months. Of course, during those two months, the team may discover that they are working faster or slower than anticipated, and may need to adjust the scope even more. Even so, the deadline may still be missed of course (say, one of the lead developers falls ill two weeks before the deadline, or a fundamental misunderstanding of the protocol that was causing sporadic bugs is understood only 1 week before).
However, if the PM only added a "support QUIC" "task" to the team's queue, with high priority, the team will almost certainly have nothing even close to releasable 2 months after the request was added, even if they are working at the exact same pace - since they focused on a more advanced version of the feature. Even if they are internally working in a way where they avoid having non-releasable features for such a long time, they are very likely to have prioritized other aspects of the feature than what the PM needed, and have only finished and polished those by the (unknown to them) 2 month "deadline".
>> Preemption points are regular time intervals at which the team will review a task’s progress and decide whether they should cut scope, change their strategy, or drop the task and cut losses altogether.
In seriousness, the author is right. Deadlines don't work as a tool to coerce people to work faster/better. Unless you have a detailed plan to get there; in which case the deadline is less important. It's just the last step of the plan. You'll know long before the deadline whether you are on track. Scrambling a few days before the deadline when you realize you are definitely not going to make it is probably doing more damage than good. And there's only so much that you can pull off in a few days. Doing that means you've already failed and you are just trying to cover it up.
A useful tool to get out of a deadline driven / firefighting style of project management is to switch to simply releasing often and frequently managing the pace at which a team releases new things and improves their software and prioritizing the next things to do correctly. We do a weekly all hands meeting where we take a few minutes to call out new/improved things. It's a very positive meeting where we celebrate progress. The flip side is that it's very visible if you have nothing to show. So, it creates a pressure to show something meaningful.
The rest is just normal planning. I set the goals for the team on a month by month basis and drive things forward. This in turn is done based on a roadmap that we revise every quarter. We identify themes where we want to do better. If I set the bar too high, I simply end up with a pile of unfinished things that spill over into the next month. You actually need to keep the queue of changes small and create space for people to do the job properly. Piling on more stuff actually reduces the likelihood of anything getting done or done properly.
We're a startup and we release often so there's no question about there being something to release because there always is a usable version of the software. Our sales people have been instructed to sell what we have, not what they'd like us to have. That just creates pressure to deliver random crap that they thought the customer wanted to hear. Usually exactly the wrong thing to begin with.
We talk only about things we already finished and know we can deliver externally. If you run a startup, stop talking about what you are going to have and start talking about what you already have does wonders for your credibility and sales. Unless what you have is nothing of course. In that case you fake it until you make it (or fail).
And there's also very little doubt about what it is we're releasing. It's literally what we showed last week + some small changes that we can do this week.
People get motivated by noticeable progress. Makes them want to do more of it. It's great to see this working. Our velocity is crazy good.
Personally I’ve found that an engaged boss who I respect and occasionally points out I’ve sure been working on X for a while, is everything okay? To be at least as helpful as schedules. But if I could figure out how to get that consistently, I’d be on a book tour instead of keeping an eye on some people doing a deployment while I should be in bed.
Without deadlines I have a LOT of trouble keeping on task.
My suspicion is that it has something to do with learning the wrong lessons from Boolean logic. The principle of the excluded middle applies to bits, not process.