On the other hand, for a complex SaaS back end and front end doing something unique, a complex system to which you are being asked to add something groundbreaking, where you need to pay off various tech debt as you go in order to get the job done, where you are negotiating the requirements as you go along, and finding dead ends and impossible things that you can't do that you thought were simple and vice versa - I think it is hard to estimate. I think in that case estimation is a waste of time and it should be thought of like a "value train". If there is a genuine deadline (rocket launch) then best to manage it well with appropriate large buffers and dependency management. I don't see this being done anywhere, it is usually the business trying to haggle down the engineers on their estimates (even though the engineers are paid a salary so ???).
It's always maddening to get into debates about this stuff with project managers who believe this nonsense. I know the Fibonacci point scale is supposed to address this by giving PMs something, but I've only ever seen it turn into time-estimates-by-proxy.
These days, I only ever give estimates in terms of time scale. A task will take "hours", "days", "weeks", or "months". As long as it's within my power, my team will only ever estimate on that scale.
The only question I ask around prioritization is "Is it worth doing?" If the answer is yes, I ask the stakeholder if they care how long it takes. If they do, I ask for a range and give a best estimate if it's achievable. If the answer is no, I ask why they care how long it takes if it's the most important thing to do.
My engineers know that they're expected to do that one project to completion, and we loop in other stakeholders (marketing, sales, QA, etc) progressively as we approach completion. Admittedly I work at a company of 20, but it works remarkably well, despite the absence of a "schedule"
Well... would you rather get $150,000 two years from now, or $90,000 next month?
You generally don't want to be making any decisions off ordinal comparisons.
Ask them to prioritize the tickets. Maybe they want some rough estimate to decide between some of them, maybe not (which is, suprisingly, in most case). During the same meeting you can groom the tickets.
The developers then start working on those tasks, when they are ready you release them according to your company policy and calendar.
By then everyone have forgotten about Scrum, management is amazed you release so much stuff, you can probably put all that in a one hour max weekly team meeting and everyone is happy.
I'm pretty sure that's applicable in a lot of pure player tech teams and quite scalable (I managed up to a 15 devs team this way but PR validation etc overtook the management part a bit much).
We also work with a Kanban-like methodology. Basically a pull system with a hard limit on WIP.
"Why can't we better estimate these things?" Because when I give honest estimates I get threatened by my boss. Your process is not reality driven.
"Can we break this work down into smaller stories?" No, I can't break down the final integration of all the parts into a smaller story. Or more commonly, they told me we have "too many stories" and can't keep track of them all when we break them apart.
The best is when a Product Owner or other business type says "I'm not a technical person." Then why the hell are you working in a such technical business and calling the shots on stuff you don't even understand?
Same people: you must complete assignments in the estimated time, or else, be prepared to work overtime including weekends.
It drives me bonkers.
I ask first will the task take 20 years to finish and they usually laugh and say of course not. Then I ask what about 20 seconds and they laugh again. We keep going on both sides of the boundary until they stop laughing and it starts sounding more reasonable.
Then I have something to work with like 3 to 6 months which gives me more options. I can accept the risk or I can take other steps to reduce it or break it into more manageable bites. It’s not perfect but it’s worked pretty well.
Another approach I had some success with was estimating the 80% likelihood case, I.e. I’m 80% sure I can finish this in the time I’m estimating. Less accurate, but much more predictably on schedule, which is more useful for many stakeholders.
A last one is giving rough estimates and being clear about how rough they are and how much time it will take to get more confidence. I was handed a 50 page PDF of API docs for a service and asked what the estimate was for integration. After 15 minutes I said “1-4 weeks”, if you want more accuracy then I’ll need a few days. The answer was “no problem, we wouldn’t consider it unless it was < 1 week”.
These examples are all on the weeks to low number of months scale, but the same applies further up. Having a good understanding of the codebase and domain are very useful.
Estimating isn’t easy, but treat it like any skill, it can be improved. It requires some flexibility from product managers, but open communication goes a really long way and results in better decisions overall, less wasted effort.
The problem is that on a non-trivial project, these breakdowns are guesses which carry considerable uncertainty. And the less you know, the more the unknown-unknowns get you. If you write a detailed breakdown of a major piece of software before writing a line of code, at the end of the project you will look back and laugh and laugh at your own innocence.
As just one random top-of-mind example, consider Carmack writing Doom's rendering engine[2]. He tried several approaches that didn't work before striking the right balance with pre-computed BSPs. Some of the things he tried were fundamentally good ideas and would later be used for different games - but didn't work on the hardware of the times. How do you estimate something like that? "I'm going to spend a month on my first approach. That could be one month or two. If that doesn't work, (50% chance), I'll read research papers and textbooks (1 week - 3 months) until I find a promising technique. Then I try the most promising of these; since this is an approach I haven't even read about yet, it could be one week to implement, or six months. And there's some chance that will fail to and I'll have to do it again." The final estimate is anywhere from one month to a year. You just can't know.
[1]: https://www.amazon.com/Software-Estimation-Demystifying-Deve...
The only thing I can do is move them to the front of the project as much as possible, so we have the risky bits as early as possible. Once we get them to work the rest should be smooth sailing, but I'd rather fail when that time wasn't spent yet. PM agrees.
E.g. I worked on a warehouse app before. None of us ever had before.
If you have no experience with any of this, how do you break down picking: SKU design (ours weren't just random numbers), barcode printers, and barcode scanners to be used in your software project? That part alone took us about 7 weeks. Including a complete SKU redesign because our initial test hardware worked better than some of the stuff we got later.
Nevermind all the hardware integrations, which was also new to us: multiple printers (barcode label, shipping label, pick list, packing slip), barcode scanner, a scale, all seamlessly integrated into the warehouse application.
This also reminds me, one of the services we decided to use had a serious (for us) bug that we didn't find out about until late in the game. They didn't fix it for 3 months and it blocked us for a while. So yeah, you could be very well end up integrating more than once. You can't really break that down into trivial parts.
The trick is to treat everything that you can't break down to a triviality as an area of research until you've solved that problem.
So, say, you plan three days in a sprint to research a certain requirement, the results being a set of small east-to-estimate features and maybe more tricky features.
If you're working in an area where your team have absolutely 0 experience then there's no way you can estimate anything accurately. In these cases I hope that you're working with a small team (2-3 developers and a BA/PO) who are highly experienced and work well together. Then you should run flexibly with features. Work Kanban style and implement the 80/20 effort/win features - and don't be scared to drop a feature if it's looking like it's not part of the 20% effort group of features.
If everyone could do that trivially, then there would never be any problems.
I don't have a statistic, but I'd be willing to bet the majority of code in existence is not algorithmic in nature.
Every iteration there'd be one or two small tasks that blew up to >1000% their original estimate.
But the way you go into a market is by figuring out what doesn't work. What is an unaddressed pain point. This is a hard thing to do by definition (that's why the payout of being successful at it is so big), but if you manage to do that, you don't need 10-15 persons for an MVP... in fact I'd argue you actually can't truly do an MVP if you have a 15-persons team. You need the large team to expand the MVP, but to identify it/ have a customer-worthy demo, the 15 people will just get in your way.
The more likely explanation is that your "strong differentiator feature" is not really as strong as you'd wish it was.
No it doesn't imply this. One granular piece of Waymo's software is "develop a component that understands the external environment as well or better than a human". It's easy to say that, and it's even relatively easy to break that down into risk scenarios etc.
But developing it is.. non trivial. And there are parts in it which it was unclear if they were possible.
But that is because large parts of Waymo were a research project, not software engineering.
Regardig the "research" vs "engineering", I don't think you can cleanly separate the two; many otherwise straightforward projects include a research component, even if it is just about using a new browser feature in a novel way, or some such.
Obviously, you have implicit constraints in mind. "Is something possible given $constraints?". And the set of those constraints is where the line between research and engineering starts to blur. If you set $constraints to "in time and under budget for the project", which seems like a very reasonable value, then it turns out that a lot of programming tasks become research work, unless you're only churning out cookie-cutter websites.
And no, "in time and budget" doesn't suddenly make something into a research project.
Speeding up some technical process might be research (eg, the old 1TB sorting benchmarks drove research).
But project constraints, and the fact that the team themself has never used a specific technology or whatever doesn't turn it into research. Yes, I know that's a term people use, but it's a different thing to scientific research.
Most software projects are more like R&D than construction. For the adventurous analogy, software is like hiking an unknown trail. You might generally know where you're going, but you can't foresee the river, detours, and feral hogs you'll have to battle to reach the end.
You simply can't estimate software the way you can more tangible products. You can do Scrum planning poker and pretend your're not playing Numberwang, if that makes you feel more in control.
Then you look at big civil engineering projects that seem to always be over-time and over-budget.
The treatment of estimates in project management for software is way more like the assumption we have an assembly line where even if there are issues these are routine enough that they can be padded in. The problem is that software is an exploratory process and not an assembly line.
It's even worse the more creative the project is. I've worked in games for fifteen years and I don't think it's possible to estimate when a game will be 'good'. Making one is a process that requires a lot of feedback, iteration and scope management. It's part of why we see so much reflexive crunch as the reality of making a game hits poor project management strategy based on this assembly line thinking.
The words Gantt chart make me shudder.
I think those projects might be best compared to the software projects we think of here...
I'm not off by more than 20%.
Also, I think by definition, a non-trivial project obviously hasn't been done by you many times before. Doing something you've done a dozen times is trivial in this sense.
Consider that many software projects led by experienced software engineers go so far over time that they are never finished.
Category 1 is
>I've written a script to get the data from tables 1 to 24 of this database, now I need to write one for table 25. It should take me an hour.
>Oh look, it took me nearly 2 hours because I forgot that table is owned by a different user to all the other ones and I didn't have the right permissions.
But this (a) is still quite optimistic, and (b) ignores variance. The potential time a new research project could take is unbounded.
Whereas padding in a more assembly line like situation can actually be measured and forecast from historical data.
The problem is that most people I've met dont really understand what they are actually doing and/or have the leadership to grow people to achieve.
It is not trivial in the least and requires a great deal of lead time to even understand the problem(s). This field is at its infancy, and I dont yet know if my insights can be replicated yet.
I'm thinking of writing a book where I illustrate thought exercises and career challenges that help people grow from junior to senior principal, but it is proving difficult.
(If I seem bitter, I'm not. Just sharing my lived experience.)
There are many other challenges as you point out.
I am currently on year two of a five year project. The key (and part of the essential difficulty) is finding quarterly or six month valuable deliverables that help ease anxiety of everyone involved.
It is not easy, but it is possible.
While all this is going on, I'm also working on a design document for a potential 10 year project. I doubt that I will pull this one off since I'm having difficulty finding those quarterly deliverables, but I'll keep grinding away.
How long was the discovery and design document phase on the 2 year project?
Some sketch of the end-design is always important, thinking about major components and future evolution, but a detailed plan has never been worth it in my line of work.
The most generous way to view what you described is that you’ve had to cancel your project half way through and start a new one. There’s nothing you can do to make business decisions like that work smoothly. In my experience though, a more likely cause of what you’ve described, is that the project was just planned poorly to begin with.
The kind of problem we have when launching a new product is that we can have a well-defined list of everything that is required of the new product to be assured of successf - it's going to take 5 years to build that, with at least 20 people.
What can we cut and still have a successful product, that we can launch in say 1 year (or 6 months) with say 10 people? That is much harder to say, and different product leaders have different opinions, based on discussions with different clients. And the consensus opinion may simply change.
So sure, the architecture is generally driven towards that 5 year product idea, but the short term estimates are always going to be geared towards a specific subset of that. And that subset is easily subject to change, from one quarter to another. We won't throw away the design if that happens, but we will definitely throw away any longer-term estimates.
And charting out a 5-year plan that we would only 're-jig' as priorities change would be a recipe for certain failure - teams change, people leave, etc. You can't hope to have an accurate estimate on that time horizon.
> a well-defined list of everything that is required of the new product to be assured of success
Only bad software tries to be everything to everybody (not to shit on bad software, it’s a multi-billion dollar industry). Your customers have a problem that other products weren’t solving, your solution to this problem is the core of your value proposition. It’s the only thing you need to implement to launch your product, everything else can be added iteratively over time. If you don’t know what that problem is, then you’ve failed before you’ve even started. You’re not driving your company forward by writing code as soon as possible, you’re just doing skids in the parking lot.
I’ve seen so many people get sucked into the fiction that their product is by necessity too complex to plan and design properly. In reality I’ve never seen it to be anything other than a cover for not actually knowing how to run a product well. You’re not saving any time by making it up as you go. All of the design decisions you need to make still need to be made, you can choose to make them in a controlled manner, or just do it randomly and hope you pull through. But good design is not an emergent outcome of the resources you devote to implementation.
Most work can fit within quarters and get executed quickly, but it takes something else to have long term vision.
The discovery process was paying attention and looking at what is actually the goal, and it generally requires the discipline to slow down and focus on the future without giving into the short term time sinks.
I had the benefit of 70-100 minutes per day of being stuck on a boat commuting for years to focus on discovery. The particular project that had a two year vision had about six months of shit to eat in office while thinking and planning on the commuter boat. It is amazing what one can accomplish by simply writing and the time to focus.
Discipline and time estimates are important, but not more important than a product setup for success. One that is easy to sell and market because it has value, is polished and a good experience.
Game development notoriously is late, and as long as the developers have time to finish it right people forget. Everyone knows when a game company and the business/marketing/financial side pushes out an incomplete or buggy product, it can permanently harm perception.
Valve Time is something that is really common game development where engineers/developers/designers/product people are still in charge and making good solid product. [1] Technology, innovation and especially game development needs time to make it fun, a good experience and a solid product.
Creativity can't always be rushed, there must be "open" and "closed" modes of development, not crunched in "closed" only modes. John Cleese has an excellent talk on managing creativity and I recommend all people I work with watch it, very valuable insight. [2]
Whenever I’m asked to estimate I tell them that I’ll take a week to estimate a 4 week piece of work they almost always say don’t bother - then when the made up estimates are wrong we can discuss the fact that they didn’t bother estimating.
It is a funny thing but works every time (in corps).
I’d guess that freelance developers who are paid per project, and thus have to own their P&L, are very good at estimating.
I see this over and over again with people I know bootstrapping projects - the over optimism I think is just part of being a builder. Sometimes in startup circles there seems to be this meme of how you should 'throw together a prototype of the tech in a week' being the standard way to get going with an idea. In practice though, seems so impossible that you could get anything even useful shaped built in that amount of time. For me when I get down to work on something, even just plumbing together the vague structure of an app takes longer than a week full time.
now the hard part is selling the idea to the management, like "here are our ideal estimates, lets multiply it by 4". when doing freelance work i usually bill for the ideal estimated time multiplied by 2, the rest being my risk, on the other hand any changes in the requirements, or miscommunication is a risk of the client.
If you can't break it down you don't understand it and if you don't understand it you aren't ready to estimate it.
I think the real reason estimations are hard is that basically every developer is working on projects too hard for them. Like, if you are so good at your current job that you can reliably churn out good solutions with ease then instead of just producing you would move on to do harder things for better pay.
I was talking with my project manager today, and he was asking me if we could finish on time if we got twice as much time as I estimated. My reply was that we’ve tried that 4 times now, and we’ve never reached the point where we actually finish within the time estimated.
So increasing the time would just mean we fail later, not that we suddenly succeed. I’m firmly starting to believe that work expands to fill the time allocated to it.
The rule of thumb i use - anything which takes more than 3 days needs to be broken down further into tasks. If any task takes more than 3 days during execution then it needs to be broken down to more tasks.