Why long-term plans don't work and how to fix them
lucasfcosta.com
lucasfcosta.com
Vision is why you do what you do. It is your North Star, the reason your company or your team exists. It should rarely, if ever, vary. Yet it should be revisited every 5 years, at a minimum.
Mission is what you intend to achieve. It outlines what is in your scope, and out of your scope. Revisit it at least every 2 years.
Strategy is how you will achieve your mission, and how what you do will move you toward your vision. It brings market data and analysis to the party. Revisit it at least once a year.
Roadmaps show when milestones of the strategy should be achieved. They are guides for expectations. Stakeholders typically begin consuming information at this level. Roadmaps are not plans, they are measuring devices for strategic progress. They must be adjusted quarterly, and will often change.
Plans (sprint plans, projects, task lists) show who is assigned to deliver small decomposed pieces of the overall work. They line up to roadmaps and strategy, but they are what is real. Adjust them at least each sprint.
There are other structures, feedback loops, inflection points, and ways to measure and adjust these components, but they are all commonly spoken of together as a "long term plan" without laying hyper-technical baggage on the word plan.
I thought this was product management 101.
That said, regardless of labels, this leveled model is reasonable and better than a model that isn't cascaded.
https://smile.amazon.com/Technology-Strategy-Patterns-Archit...
As of this writing, you won't find anything better.
Given an agreed strategy, you can then develop a plan, which is the specific sequence of steps to achieve the ends.
Long term plans do not work very well for sure. But they're not _meant_ to. Long term planning is meant to give people an idea of what's coming after.
This is important because what's coming after may change the implementation of what you're doing now. Some people in the team may say: "hey if we did 3 before 1, then we'd have 2 for free".
Estimating the time it will take to do things is also important cause you may think that you know what's coming after, but if you don't know how much time it may take (apply a 0.5-2 multiplier range), you may actually never have time to do it.
Keep making plans. Keep selling them to stakeholders, and keep telling them that it's just a rough plan and that details will be very different.
In any case, it'll cost you more pain to fight your CEO and tell them that you don't want to make a 1-year plan, than to actually making a 1-year plan...
Of course long term plans/road maps have their place when everybody agrees on that it is an "idea of what's coming"
However most long term plans I have encountered at my last two companies have absolutely been treated as commitments to detailed plans two years ahead of time, before we have no idea what we really want.
This is causing project members constantly stressing/hurting about not meeting plan still years after everyone knows that the plan does reflect real life anymore.
On the other hand having only short term "agile" plans can on the extreme end turn us reactive and without any focus. If we are dealing with a "big" problem this is a death sentence for the project.
If the project is big complex and hard to predict we have to be able to be agile on both the short term and the long term, and sometimes on the really really long term, all at the same time. Oh, and also for the group and the whole organisation at the same time.
It just works very well for us.
Long term plans "work", but by sticking to them you might be giving up opportunities to do something more profitable. They can also "work" in the sense that the culture encourages abandoning them if something more profitable comes up, but then you've spent time planning for something that doesn't happen.
How often do you decide to abandon your long-term plan for something more profitable? And how often do you decide to pass up on something more profitable to stick to the plan?
That is an issue with your company, not plans themselves. I'd agree with avoiding long term plans in a disfunctional organisation.
Yes and: Strategy & Tactics
Beyond the inability to reason about stuff, not even asking Who What Why When Where and How, the inability to divide & conquer, the hostility to verifying assumptions (eg will any one buy our stuff?), etc. etc.
I've met very few people, any where, in business or politics, that can war game scenarios. Much less reason about tactical decisions. Or even know the difference between strategy and tactics.
My hunch, totally unvalidated, is a strategic thinker isn't especially smart or talented, so much as everyone else is fantastically, tragically inept.
I always marveled about the rise of Microsoft. Sure, they did some good stuff, made some good moves. (And then later was simply criminal.) But OMG their competition was terrible. Ashton-Tate, WordPerfect, Novell, Lotus, and probably a zillion others. So many defeated themselves. Microsoft was like the Bolsheviks; they found power lying on the street, after every one else imploded.
This is precisely the problem though. Your statement is contradictory because if (as you say) long term plans don't in fact work, that "having an idea of what's coming" is complete fiction.
And trust me, confidently believing in fiction is much much worse for effective planning than proactively preparing your product for uncertain outcomes.
If I am 100% sure our product will be doing X and not Y in 18 months, I'm going to build that product with a strong bias toward X and zero consideration for Y. If I don't know what's coming, I'm going to build it with weak contracts toward both and a heavy emphasis on being refactor-/pivot-friendly. Which of those approaches will serve my company better when we unexpectedly end up switching to Y after a year?
I am pretty sure there are teams / projects for which one methodology is better than the other.
Stating that there are no situations in which it is beneficial for some team to make long term plans is simply too strong a statement.
Still, I do think there's quite a large portion of detail-planning that can be radically different dependent on the high-level expectations, even where the range of high-level outcomes is small.
i.e. (random off-the-top-of-my-head example): I'm building a piece of software offering some kind of service to end-users. We're considering staff-curated -vs- user-generated & curated content & we opt for the user-gen approach as we don't think the first option will scale. As we get into the market, we quickly realise the demand for content is high, we're in a good position to adjust our pricing upward, but engagement in content-provision & curation activities is low. So we decide to pivot to staff-curation.
The above pivot doesn't involve a radical product change (the offering remains similar overall), but does involve radical technical changes under the hood. A long-term-planning oriented company may have invested heavily in building robust (expensive & complex) community features that may be thrown out later. A short-term-planning oriented company would more likely hedge their technical decisions - leverage some 3rd-parties temporarily during user-testing, maybe test earlier, be more sensitive to priority changes.
Even within the same organization, it’s such a gamble when:
- X might never come if the project takes too long or gets deprioritized. 18 months is a long time, perhaps half of an engineer’s average time in the company. X’s sponsor going away could mean the project stays in limbo for the rest of its life
- X might become Z when you realize midway that X wasn’t a good idea in the first place, or the assumptions behind X changed enough for the project to not make sense anymore. The org might not care for the cost to readjust now that X is gone, but you’re still worse off than if you weren’t waiting for X.
That’s something long term planning can give you a reasonable idea about. If your working on Madden 24 then the business model is about minor changes every year, while most games are one and done.
Absolutely TRUE - and this becomes a political problem in some organizations, depending on whether you're around people who can handle hearing the truth (whether they just don't want to hear bad news, or whether they don't understand complex webs of interdependencies that can cause time estimates to be wrong).
A long term plan should be not a linear chain of events. But a scenario-tree. You are always at root - the now. Now you gain a new a capabilitz or the world changes around you, and the tree adapts. Its branches thicken were they become more likely. Unlikely scenario branches wither away. The whole structure is recursive, one capability reached makes other capabilities more reachable, but might introduce risks, that shrink the whole tree down.
This scenario tree is not focused on a single product, as a product is just one of many expression of a capability. But when the capability is reached, the whole thing has changed, including your view on risks and rewards while moving on. So pivoting might be in order. Meanwhile others move to similar trees, and change the world around you for the better and the worse.
To compress this structure down into a "historylike" sequence of events, is the main fallacy. The past may be a linear graph, but none the less a ever changing root-recursive-tree was traversed by those moving within it.
I think the problem with the long term plan is the destination. If your plan is based on being in a specific state (“I want us to have made this product in one year.”), it’s doomed because it’s very hard to target something that accurately that far away with everything that comes along as the posted article elaborates.
We should be shooting for an outcome. “I want us to have a way for our users to do X in a year.” That way, you’re less trying to wrestle a shoe onto a goose, and more coaxing something into shape based on where you want it to go. It also prescribes less so the path toward completion can always be optimal. For example, I know that for our users to be able to do X, we need to house the data in a certain way and expose these other services in a certain way. After those steps are achieved (or maybe they fail), we now know where we are in this moment and can take the next optimal path.
All the problems the article describes are real, and I found it thought provoking and useful, don't get me wrong.
It's the phenomenon where the actual work increasingly gets pushed aside to make room for recording information about the work.
Jobs of higher and higher complexity get invented to measure and analyse this meta-work, ostensibly in the name of productivity. But I suspect a large portion of it is just the paper-pushing meta-workers making jobs up in order to fill their time.
How about putting this in the context of a company & product like Coca-Cola though? It's not bad for them per se, to hand over the reins to finance/commerce/bureaucracy. Can we call it a "business cycle" perhaps?
Basically after a certain size the question is not how to make the product better, because ... it's what it is. It's a mature product. Sure, there are sometimes quirky versions, seasonal versions, promotional versions, but the high-level focus is on keeping the market share, and to grow it if possible.
For example at that level it just makes sense to pay for lobbyists. To know where health regulation is going (soft drink tax), and of course CocaCola probably tried and tires to contest these proposals on many levels.
Bureaucracy is the process of adhering to processes, management is the process of coming up with new processes. Then of course the bigger the whole enterprise the more these seem to be ... the same.
But then the question is, how does management determine whether to stay an enterprising voyage (eg exploring space endlessly) or become a settlement (eg McDonalds franchise)? Maybe the analogy ends here.
and society at large (culture!) decides what is a worthwhile endeavor, for example by pre-ordering space telescopes, or by complaining loudly and shaming anyone who does not behave like good McDonalds patronizing people.
This is true, and the usual proposed timeline is usually wildly inaccurate.
Sometimes we technicians adopt such condescending tone, like this article's, out of ignorance. There are decisions whose underlying logic we won't fully comprehend while we don't consider factors outside tech and product design. This article seems to share the misbelief that plans are supposed to predict future. Granted, several managers also think like that, but many people who produce and consume long-term plans are relatively well connected to reality and understand their limits.
Long-term planning approaches the problem from the ass end of things. It's going to the engineers and saying "design feature X on paper for us" and once they've spent a month doing that, management says, in the best case*, "well, this work cannot be done for real in the remaining five months, so never mind".
The original question would have been answerable in a day without going into too much detail.
----
* Best case. The more common case would be "well, now that we've spent a month planning this we better get the actual work done in the five remaining months, no matter what it takes!!"
We engineers can be really fast at refusing unrealistic estimates imposed by other departments, especially when we take them for loosers. But we are not very good at estimating as well, and we are often overoptmistic. I don't track my times too fanatically, but I do it +/- regularly and, still, I err way more than I think it's reasonable considering the data I have.
I don't think you are going to get much reliability on simply asking 1 question to an engineer, especially if you approach the matter with such overconfidence.
You might argue that an engineer would be more conservative and would avoid many mistakes by replying "no, it isn't feasible" more often than not, but then they would be wasting a lot of opportunities by discarding feasible projects, ruining the company as well, only in a different way
> asking 1 question to your engineers
If they can't do their own feasibility assessments, they will be producing bad work, because there are many feasibility questions that arise during design and development, and never even make it to the parents^Wmanagement.
Then I feel like that's the bigger problem that needs to be handled with training, and perpetually working around it is still a bad idea.
You seem to be describing an issue of trust and risk assessment.
For instance most companies won’t micro-manage projects that probably last for a few days, as they trust their team and/or feel comfortable pulling the plug after a few days if it didn’t pan out: the risk is low enough.
Same way buying a new mouse probably requires less oversight than contracting thousand dollars of GCP for 3 years.
In that respect, how long a project can be before needing a fleshed out plan, internal milestones with verifiable deliverables and a set timeline comes down to how much the team is trusted.
I think there is no single reason for why long term plans don't work. It is more like you need to do this dozen things right for the plan to work. The longer the plan the more things you need to be doing right to be able to see through the plan.
Accomplishing short term plans can be done relatively easily. There is less chance something unexpected will happen that will throw your plan into disarray. You have a chance to "just focus and get it done". You can take on debt in order to accomplish the task.
The longer the plan, the better you have to be at dealing with various things that require you to have much wider view of whatever you are trying to accomplish.
For example, if your plan is to loose 3kg in one month, you can just do it pretty much on your own with little interference/inconvenience to your wife and kids.
Try loosing 30kg over course of a year and you start having many more problems that you might not appreciate before you took on the project. Maybe your family is unhappy with the change of the diet and your schedule? Maybe your wife is unhappy with your lower libido/irritability? If you want to keep your gains you need to learn to make your diet stick and change your eating habits, which is super hard.
In a company setting you might find that long term project might need you to rethink many more things like how do you keep people dedicated/motivated when it is hard to see the distant goal? How do you keep yourself motivated? How to deal with people leaving the project? How to deal with more urgent/short term project preempting the long term goal?
Also, among any other analogy you could have taken, choosing diet is pretty ironic (or perhaps completely apt actually?)
Even with losing weight, it is much easier to understand what you are getting into when you want to drop 3kg vs 30kg.
In my case feeling cold all the time for about a year wasn't something I have predicted when I started it. Or how difficult it is going to be to manage clothing department. Or to learn new recipes.
Now, none of the above would cause the project to fail, but one quarter into that year I figured out I need to find ways to keep me focused on the goal or I am going to fail. Just eating less wasn't doing it for me and I felt constantly hungry and miserable and I would soon drop out.
So I had to pivot and be innovative to be able to finish the project.
With that you would have set yourself with activities that stretched the span of your endeavors, say for half a year each week I lose 1 kg. I do that by eating this, exercising like that, etc. Then, things start falling apart with the hurdles you mentioned, and you adjust your plan by changing recipes, trying to find more fitting clothing, etc. So, you adjusted your long term plan to still reach your goal.
The point this article is trying to make is that you shouldn't stick to your plan but rather keep in mind your goal and change the plan along the way.
Project I normally work on span tens to thousands of people but they are usually very complex to give as examples without spending some more time thinking how to best explain it.
> "Don't be afraid to take a big step if one is indicated; you can't cross a chasm in two small jumps"
I like to sum it up as the tension between two contradictory yet true axioms:
(1) Failure to plan is planning to fail.
(2) No plan survives contact with the enemy.
You can’t just abandon long term thinking because it’s hard. Grow up. You’re an adult. Life is hard. You still need to have a destination in mind. The path to getting there probably won’t look like the one you planned, but your odds of success are much better when you both have a plan and are ready to adapt and evolve that plan as you go.
> Grow up. You’re an adult.
Being an adult is also about taking paths that you don’t know where they go, but have enough resources, experience and confidence you’ll make it work somehow. I think your point has merit, but it’s not that solid to match such confidence in it.
In virtually all initiatives, we don’t know where we’ll end up. Not really.
But the best way forward, the best way to set yourself up for success, is to develop a plan based on what you do know. It’s incumbent upon anyone responsible for keeping others employed that we don’t dive into things blindly. We make informed bets, and hopefully we’re right more often than we’re wrong.
I’d like to add that you were far more respectful of me than I was of the OP. I appreciate your respectfulness, and I’ll strive to do better.
With such a solution:
-You can start dogfooding early on.
-There's less to go wrong.
-There's less sunk cost in those of the features that don't make it to the final product.
Integrate one of all those payment processors that were required, one external API and move on. Also don't do fancy, handcrafted components on the frontend. Or component libraries for that matter. Have separate tickets for validation.
Once you're finished with that you'll find that in the meantime the Nth payment processor went bankrupt, Nth external API was deprecated and UX finally managed to convince the stakeholders that a color picker the shape of a peacock following the mouse with its gaze is not the best solution in terms of user experience.
hahahahaha
“The product didn’t work because the developers didn’t follow the plan”. Doesn’t matter that the plan was half baked and had no connection to reality.
I think that things like GTM strategies are super important but long term plans are useless. Build the next thing, ship it, learn, repeat.
The Marshall Plan worked.
The James Webb Space Telescope took two and a half decades of effort, but the plan worked.
I could list a thousand plans that worked.
Now, why do we, as software developers, not think plans work? Because they're less well suited to our domain and the reason for that is that our domain is incredibly complicated, changes frequently, AND it's incredibly maneuverable.
What this means is that its more efficient to rely on this inherent maneuverability and build fast feedback loops to code our way towards a vague goal than it is to start planning for something 20 years away.
Is someone really going to game out whether webhooks or HTTP long polling are going to fit 2042's software landscape better? No. It doesn't matter. There will be transports for data. There will be instruction sets. The abstractions will change and morph over time, but it's not fruitful to ponder the explosion of possibilities. It's enough to pick a direction and step on the gas and figure out the rest as you go.
This most often comes up in rewrites (which are usually a mistake) and the problem is that time doesn't stop once you start you rewrite (or, more generally, your long term plan). Features get added to the original. Bug fixes are made. Software entropy happens. These need to somehow be folded into your new thing. Or worse, you halt new development on the old while you write the new.
I've seen pointless rewrites with a 6 month estimate take 3.5 years to deliver, all the while the once-leading old version languished.
More generally a lot of this comes down to risk management. Your long term plan needs:
1. Shorter term milestones. Due to changing circumstances you may end up terminating your long term plan early. You should aim to have something to show for it if that happens. Ironically, having something to show for it makes this outcome less likely;
2. Reversibility: your migration (data, software, etc) needs to be reversible; and
3. Incrementality: you need to be able to gradually roll out your long term plan. It can't be all or nothing.
This is a common pattern in data migrations:
1. Double-write to the old and new systems. Start doing this with a small percentage of users (<=1%). Identify problems, fix them, rinse and repeat. Do nothing with the new data;
2. Increase the rollout to 100% gradually;
3. Have the system be able to read from the old or new system and work fine;
4. Roll out a small number of users to read from the new system. If this goes badly, you just read from the old;
5. Identify and eliminate all the old call sites reading from the old;
6. Once old data traffic gets to zero you can switch it off.
I've seen too many systems where the project plan is basically to shut down development for 18 months while you write the new (often leaving non-critical bugs in place), do a 3 day data migration where the system is down and hope everything works.
The best way to figure out feasibility and timing is to just start doing whatever. Presumably, we are talking about software and computers so mistakes are cheaper.
An estimate that comes out of 15 minutes of trying to do that actual thing is infinitely more informed than one that a committee deliberates for months.
[0] https://chrisgagne.com/1255/mary-poppendiecks-the-tyranny-of...
1. Vision: setting long-term goals and speculating on the path that will best meet those goals.
2. Incentives and accountability: trying to make sure a team delivers X in Y time.
Setting a long-term vision is of course extremely valuable - find me a great founder or leader who doesn't have a long-term vision! It's the trying to force X to happen in Y time that increasingly breaks down over longer timelines.
I’d like to propose a slightly contrarian, if not, more nuanced view of what is the purpose of a plan.
If one considers a plan as a map that must be followed in minute details, to travel the proverbial territory then, of course, you’re much better with small, unplanned iteration that allow you to learn from each story you deliver.
The image that the author proposes is a very powerful one: yes it is much better to play the lottery by buying one digit at a time, because overall, you’ll cut your losses short in most cases.
However, this only focuses the value of the plan in it’s execution phase, absent from the planner, and the feedback he will get from the divergences from the plan, and the updates he will be able to bring to the mental model / explanation linked to the plan.
Let me be clear: *plans are meant to fail*. Why? The value of the plan is in *HOW* the execution diverges from it, and in so help us to update how we think and reason about the world.
Each plan, has as origin an explanation, theory, of how the world/market/customer works. Changing your explanation of how the world works is the goal of learning. To learn, you must make predictions. That is what the plan is, a simple prediction. Of course we should not expect those to line up with reality 100% of the time. It is quite the opposite: *expect your plan to fail* but most importantly *improve your understanding of the world / your market / your customer based on what you did not predict.*
A basic framework like "SMART" can help. But it does require some careful management; people can get really, really conservative if you're not careful, i.e., "undercommit and overdeliver" becomes a mantra. Some engineers and their direct management will throw up a lot of roadblocks to avoid looking bad.
This tension between trying to deliver consistently vs trying to take on bigger problems absolutely wrecks plans. If you want to deliver consistently, just focus on small problems. But not all problems are small, and some are even hard to break down because there's a lot you simply won't know. Periodically redefining and re-evaluating your goals is probably "step 1" in this process, and from there, the plans should flow.
> Instead of rewarding teams for delivering useful, profitable features, companies which worship conformance to plan reward developers for features on time and within budget
"The plan does not look at the usefulness of work done. The plan decides in advance on a time frame and budget for each feature."
> Consequently, to make up for these delays, teams are pressured to deliver the next feature in less time than they originally estimated.
"The plan must be rigid with respect to unforeseen bugs. Scope or time frame cannot be adjusted."
> Consequently, long-term plans either force you to stick to useless features or pressure the team to accelerate the following features to compensate for the delay created by listening to feedback.
"The plan does not allow adjustments based on feedback."
In companies where I've worked, a few Fortune 500 ones included, there is this thing called "budgeting". More formally called AOP, Annual Operating Plan.
Your team, IT department, whichever...gets a budget. For the year. And you're going to tell the owners/accountants what they get for that. And it isn't just going to be "we'll add dark mode", it's going to be tied to bottom-line business metrics like revenue, cost of non-quality, compliance, etc.
Further, this budgeting exercise also reveals cross department dependencies, critical path issues, etc.
Of course, this still doesn't mean that the plan will fully work out, but this idea "let's just not do it or wing it" doesn't really work. At least not in the companies described above, but things are likely much different in well-funded startups.
OR you handle it the way casinos do, using the law of large numbers[1] to make extremely accurate predictions over many consistently-produced values. There's a good reason why every single formal study on estimation, and every single agile framework has come up with abstracted estimation units and the same velocity calculations. A mature estimation process, run for a month or two, will give you medium- and long-term estimates that are at least within 5% of reality - that's perfectly useful for long-range planning. So the first half of the article is simply showing off that the author has never learned about professional estimation processes. Fair enough, most developers haven't.
Others have already pointed out in this thread that the purpose of long range planning is to indicate what we expect comes next. TFA acknowledges this as a good use of long-range planning.
There is one good remark in there, however:
> Instead of rewarding teams for delivering useful, profitable features, companies which worship conformance to plan reward developers for features on time and within budget; it doesn’t matter how useless or broken those features are.
As an organizational principle this is exactly correct. By all means use long-range plans, and adapt them as the situation changes. But don't build a rewards scheme around them!
Could you people on HN share some numbers? If I get a lot of good responses I might present a summary later.
1. How long do you usually spend making your long-term plans?
2. How much time does one of these long-term plans cover?
3. How often do you make these long-term plans?
4. How often do you end up abandoning a long-term plan to do something more profitable as it comes up?
5. How often do you pass on doing something more profitable because you choose to stick to your long-term plan instead?
1. A plan took roughly 40 person-hours to make.
2. The plans were meant to cover 6--18 months, although a bit unclear at times.
3. These plans were generally made 5--6 times per year.
4. 100 % of the plans were abandoned for something more profitable.
5. About 80 % of the time when something more profitable came up, we chose to pass it up to stick to the plan.
Halfway through my time there, we had a change of process which moved emphasis much more toward short-term planning. The long-term plans became much more flexible. I would summarise the outcome as
1. The long-term plan is built in 4 hours.
2. It still covers roughly 12--18 months.
3. The plans are adjusted to agree with reality every 5 weeks or so.
4. The long-term plans are still abandoned around 100 % of the time.
5. We now pass up on around 50 % of the more profitable things.
1. There's a few weeks every summer where planning is a major focus, but then evaluating and adapting that plan might take up to half a day a week on average throughout the year. Basically: What assumptions underlie the plan, and have those assumptions changed? Maybe we assumed wrong, or maybe facts changed. So I try to keep an eye out for things that require an adjustment in our direction.
2. I have 10 year goals, 3-5 year milestones, and 1 year plans. I'd like to push that to 18 month plans, but at the moment I only go into any detail for 12 months.
3. Once per year in terms of formally planning the subsequent year.
4. I can't think of a time where I completely abandoned a plan (as opposed to adapting it to changing data).
5. Interesting question. I would argue "never", because if there's a truly more profitable option on the table, then the plan needs to adapt and take that into account. But are we talking about short-term or long-term profitable? I very rarely change my plans in response to short-term opportunities, because that's a great way of killing long-term profitability.
"I thought all late projects were the same in that they were really estimation failures, not performance failures. I still believe that all late projects are the same, but for an entirely different reason. All projects that finish late have this one thing in common: they started late." -- TD
My company used to have ten-year major product cycles. They have since, shortened it, but it was still at least five years ahead.
They would have a "long range" plan, that looked many years ahead. It would be strategic, and very "fuzzy," with tea-leaf reading and crystal ball-gazing. They would deliberately eschew details.
It was like looking at a map, and saying "We should go West."
They would have a "middle range" plan, usually, about a year ahead. Sometimes, eighteen months.
It was like saying "We should follow one of these highways, West."
Then, they would have detailed "project plans," that would be (in my opinion), way too detailed and mapped out. These were usually handled in six-to-nine-month milestones.
It was like saying "We should follow this highway, and take this side road," etc.
I follow a similar pattern, in my own work, these days, except that I keep my project plan flexible, and pivot as necessary; sometimes, on fairly short notice.
If your plan is built with these assumptions you are not very experienced at long term planning.
The real reasons long term plans don't work well in real life is are:
1. People don't spend enough thought on strategy before putting the plan together [1]
2. The people assigned to make the long term plan often don't have buy in for the results
3. Most management teams have, at max, a 1-2 month attention span
The most important result of a long term plan is that it produces a collection of important, documented, well understood projects and their relevant priorities and approximate costs.
[1] See comments on "Why don't we have a strategy?" https://news.ycombinator.com/item?id=32094469
You’re suggesting that plans fail due to poor execution, he is saying long-term plans are pointless when you don’t know where you’re going in terms of market fit, design, feature set, which is usually the case (and arguably should be the case if you want to build a competitive product).
Proper long term planning EXPLICITLY includes:
Risk Planning for "1. Nothing will go wrong"
Provisions for Product marketing/Spikes/UX Design to address that fact that "2. Product developers [DON'T] know precisely what they must build"
Padding and provisions in estimation errors to address "3. Product developers know exactly how long each task is going to take"
It's like Long Term Planning 101 to address these things.
The point is that even “proper” long term planning doesn’t work for the former. You never have enough information for the plans to actually materialize, it’s not simply the planners fault.
Long term planning "works" everywhere ... you just have different plans: sometimes less detail on the 3-12 month items, sometimes you need a detailed plan out for a full year or more.
The idea that something we "need" but maybe not in the top 5 and it seems weird to deal with it with "bring it back to each meeting until either you don't want it anymore or it becomes top 5", why not instead put it into the roadmap?
Someone else has alluded to external investment driven by strategy and long-term plans. It is uncomfortable if there is no long-term plan even though we are told that investors invest in the people, not the product.
Some tasks are long-term and might involve communication with customers, deadlines to deprecate something, other work needed to support something. If all of this will take 1 year in elapsed time, you can't simply forget the later bits since the earlier bits are only being done to support the later bits.
Because then there will be 100 things on the roadmap, 20 of which have a sponsor that have long since left the company.
In an ideal world, yes, everything would go on the roadmap which would automatically take into account the market appetite and internal economy and sort things so the most important thing is first.
In reality, trying to intelligently prioritise among hundreds of things takes a huge amount of work -- time we could instead spend on doing the real work of development. In other words, it's a trade-off: do you want to spend time shuffling Jira tickets around, or actually implementing them?
Having people bring up their top priorities repeatedly ensures that the only tasks under consideration are ones that people think are important right now, not ones that were important at some point some time ago. And we get that filtering for free!
Let me give you an example. You go on touristic vacation. Most people would plan the core sights to be seen and places to be visited, probably center all that around pre-booked places to crash for the night. This article sort of advocates shortening the plans to half a day plans, because you might to go a tad bit different route.
Of course there are uncertainties and the longer the term, the more uncertainties. Knowing exactly how long a task takes is worthless if you do not know what and why you are building. Where does the itemized list with well-defined tasks for sprints come from? The longer term plan. Both are very relevant, but are tools used at different levels.
Without long term plan Facebook could have ended up a dating app or blogging platform.
My best travels have been those where I only ever planned one or two days ahead. Backpacking is great!
At least in my circles people do vacation like that - general macro plan with exact micro decision taken at the moment. Macro plan is there to give general shape and maybe put certain anchor points (hotels, flights in case of vacation). Just because you planned dinner in your macro plan does not make macro plans a bad tool in general like this article tries to portray.
At the start of each year we have a plan, but it's a rough plan. We know we have certain external constraints that need to be met, for example system A is moving from sending XML files via SFTP to a REST API in June. If we already know the scope, we'll have some rough dates for when we should be ready for testing and so on, if not we have some (much) earlier dates for studying documentation to determine scope and requirements.
We also have some internal goals we would like to meet, for example making a new module for doing X. They also got some rough dates attached.
This yearly plan is then used to ground our short-term planning.
However, I didn't get the feeling that it was this sort of long term plan the article had in mind.
Or in plain english: ~70% of detailed JIRA tickets written more than 6 months ago will need to be updated significantly or get closed as 'wont do'.
Real "Plans" are trees. What is being presented as "the plan" is just the most desired or likely path from root to leaf. The problem emerges when people stop considering the rest of the tree exists. The primary thing to focus on in planning is "decision points" and the branches they point off to.
Then deal with all your workers/coworkers/bosses getting frustrated with your "too complicated" decision tree based plan and ask you to throw away 99% of it because they would rather ignore undesired/nonideal futures.
This means that we can either reduce the feature count or we can reduce the code quality. If we stick to the plan (which inevitably talks about features) then it’s quality that will suffer, but reducing quality has a significant cascading effect on the entire project.
So the only real option is to change the plan - change the deliverables timing. But if the value of the plan was predictability, this option suggests that long term planning isn’t particularly valuable anyway.
The idea that these are somehow constant is one of those things that looks superficially reasonable but really is not.
I often hear people wanting to reduce quality to increase velocity. But that's not how any of this works.
Code that is hard to read, issues with tooling and papercut-type bugs that aren't showstoppers are the velocity tarpit. And these are the first things people skimp on in order to move faster. It's not just a long- versus short term thinking either.
Part of it is that nobody really knows what "quality" is in a software development context. Except for the people who specialize in software quality, but they mean something different than the rest of us.
To put it another way, the dimensions of software projects are mostly invariant; absent an unlimited budget and unlimited time, one of the few effective knobs to fiddle with is the number of features shipped by a given time.
The problem is that long term plans and detailed road maps attempt to make feature delivery invariant as well, which - IMO - is why at least 30% of IT projects still fail [0] - and of those that are said to have succeeded, many of them seem to have redefined success post-hoc.
When we start to believe that delivering features to some inflexible plan decided 12 months ago is why we’re here, we’re well on the path to building crap. What’s important to me is to ship good software as quickly as possible, learn from the users, and be flexible and respectful about the future.
[0] I’m sorry but it’s late and I’m too tired to pull out the PMI stats URL, but a cursory search should find the latest stats.
So, the issue is, don't separate long vs short, but:
- Have a long term plan
- Decompose into short term plans
- Fix short term plan instead of changing the long term plan.
- Enjoy the ride.
In reality none of the long term plans have worked apart from long term habits and principles. Plans are there to fail.
"War will not go according to the plan but no war was won without a plan " :)
Yet, here we are. In Agile Land. And _everyone_ treats assumptions as some proven datapoint. We now call them 'estimates'. But if it looks like a duck....
That's how Agile has gone wrong in practice. Responding to change is a true need, however, lack of due diligence to understand the current requirements is not. However, that's the way I see this as being used -- Start working now, we'll figure later.
There are very few software shops with the management discipline to not change a plan for a full year, because the plan is working.
Adjusting the long term plans constantly is an entirely different animal.
Deadlines are useful. Yes it may need to be pushed back but saying it'll be ready when it's ready with no time frame is not, in most cases useful.
In a perfect world (for me, from my point of view), there are no deadlines, but rather short plans and enough understanding of the plan to decide whether doing something specific is useful or a hindrance (and to what degree) for executing that plan.
But alas, deadlines are easier to decide on.
Shorter, soft deadlines and milestones are also good, since they provide a measurement between long term planning and current progress. Without them larger projects will likely failed than success.
However management loves short, hard deadlines with artificial reasons, which makes them stressful.
It’s fine to say “we need to ship a product by 2023”. But the kind of long term plan the article is talking about would not only set a date but list the very specific features that will be delivered each month leading up to the date. This kind of planning happens really often. It’s nuts, for the reasons they list.
I once gave a version of this to a manager and he replied by telling me that nothing gets done without a deadline.
He kinda made a point. Curious to see what people think about this?
I think the reasoning KK gave was the deadline will force you to drop all the unnecessary cruft to focus on delivering the essential as fast as possible.
To me, a good plan is a map of dependencies in time.
That prioritization does not come out of nowhere, there need to be some business goals and a strategy. Those are longer term things that change less often. Often that is summarized in some kind of roadmap. A roadmap is also not a plan but more of a tool that you use to align short term planning with long term goals. Ours is adjusted on a quarterly level and we tend to try to figure out the two or three most important things that we need to make some progress on. And we set some targets based on that.
But a list of items that you want without a clear idea of how you are going to do these things is not a plan. Simple reality: unless you do waterfall, you won't have such a plan. And if you do, it's likely a bad plan. Adjusting plans regularly and figuring out business goals is something that happens a lot in software engineering. A classic misunderstanding is that that is a problem that needs to be fixed. It's a feature, not a bug.
Building software is simply not like manufacturing, architecture or other engineering disciplines where you work off blueprints on predictable schedules. Until those blueprints are done, planning an engineering project is not doable either.
You are not going to start building say a nuclear power plant without a very detailed plan and some blueprints and lots of analysis having taken place already. Of course, creating such a plan typically takes years and represents a massive investment in itself. And of course a common complaint with nuclear plants is that the planning stage is expensive, runs too long, and seems hard to plan. Agile nuclear plant design seems to be not a thing but if you think about it, the process of coming up with detailed plans for a nuclear plant has some similarities to how software projects work. Lots of uncertainty, constantly changing rules, requirements, legislation, and other factors. Maybe they should be more agile?
Anyway, in software, the blueprints are the main thing that you build. That's why it's called software. It's an executable specification of some sort. Building that specification is the whole work. Having a blueprint for the blueprint is not really a thing (though of course many have attempted this). You might sketch out bits of it on a whiteboard and do some mvps, prototypes, and what not. But that's not the same as a detailed plan either.
What's interesting is that some engineering companies are actually applying agile practices. E.g. the way SpaceX rapidly iterates on their prototypes seems a lot like the chaos you see in software companies. It seems to work for them and is very different from the way some of their old competitors work. What has changed there is that a lot of the work has moved inside computers. They are iterating on software, cad designs, using simulations, etc. Modern engineering inherently involves a lot of software. And you need to plan accordingly.
Some moments later i remembered, i had made a comic about exactly that. Searched for, and 5 (um no! way more years ^^) years ago we both sat in the kitchen, when i was writing a note after she confronted me with: 'you seem to forget too much!', after i had forgotten something. So i was writing when she ask me: 'what do you wrote?', and i told her 'A to-do-list'.
So she sayed: 'Show me...' i handed it to her, written on the sheet of paper was: 'What you can do in under two-minutes, do instantaneously !' (-;
edited:typo