Why are developers expected to estimate tasks at all? | Hacker News Reader

Why are developers expected to estimate tasks at all?

pm.stackexchange.com

277 pointszikohh377 comments

Manager here. I ask everyone for estimates. Then I track how long shit actually takes. I find that:

- Some folks are scary precise in their estimates. I’d say this is like 10% of engineers. You ask them for an estimate, they tell you, and then it takes exactly that long every single time. In other words, I have confidence that if I ask one of those folks for an estimate, I can take that shit to the bank.

- Some folks always overestimate. This is rare. Maybe less than 10% of folks. When I find that someone overestimates, I know that I can take that shit to the bank as an upper bound, which is still useful.

- Some folks say they don’t know. That’s fine. Those folks are even rarer, so I don’t have to do anything smart for them.

- Most folks underestimate, sometimes hilariously so - they are always one day, or one week, or some other too-short amount of time, away from finishing their multi month effort. That’s fine. Once I know you underestimate, I know that I can take your estimates to the bank as a lower bound. For more than half of the underestimaters, I find that there’s some formula that works: like if Steve says he needs just one more day, he always means he needs five more days. So if suddenly Steve says he needs another week, then I know he probably needs over a month. That’s still useful to me and I’m totally fine if Steve then tells his friends (or HN) how dumb it is that I ask him for estimates. They may be dumb to him but I’ve got my napkin math that turns his BS estimate into something that I can take to the bank. (I don’t actually manage anyone named Steve but I was a Steve as an IC.)

So yeah. When I ask you for an estimate, I expect it to be wrong and then I look for patterns in your wrongness. For most people, there’s a pattern that allows me to turn something that looks like a nonsense estimate at first into something I can then plan around. That’s why developers I work with are “expected” to estimate.

>Some folks are scary precise in their estimates. I’d say this is like 10% of engineers. You ask them for an estimate, they tell you, and then it takes exactly that long every single time.

It's possible they are hiding their true skill and sitting inside their comfort zone, so they are really over-estimating based on what they can actually do, but are keeping that truth to themselves. A good reason to do this is to avoid being assigned more work for no real gain and pushing the work-life balance more towards life. This situation annoys some managers when they find out they weren't getting the maximum effort possible from someone for their compensation.

This was roughly my first thought, too. It may not be nefarious, though. In many software projects, if you know what you're doing, there's a lot of work that's useful-but-optional that can be delayed to the end stages of the project. So then you front-load all your hard requirements so that you can minimize the risk that you'll miss any of them. Once that's sorted, you can just knock your way down the rest of the to-do list until you run out of time.

This is actually how Scrum is supposed to work, and most the Scrum rituals are just a formalized process for doing that in a collective ownership setting. But at some point we forgot that and started confusing the means for the ends, and it all kind of fell apart.

I came here to say the same thing. It's either that or Parkinson's Law [0], usually expressed as: work expands so as to fill the time available for its completion. I find that for myself it is true, when I have more time, I take my foot off the pedal. When I am running late, I work harder to meet the deadline.

[0] https://en.wikipedia.org/wiki/Parkinson%27s_law

If the manager's happy with the employee's output, and the employee is happy with their task and compensation... Then how is that a problem?

Our field is based on trust. When the backend is having massive performance problems, and Steve tells you it will take a week to fix, that might make sense to you because you’ve seen hairy backend problems in the past, and he’s the one that looked into the technical details on this one. But you’d be pissed to later learn that it was a one line fix that he knew about all along, and he just took a week at home chilling out while others were waiting on him.

That kind of Steve wouldn’t fool me because I’m all about redistribution of XP. Even if Steve is the best at that task, I would assign it to not-Steve with some probability. If not-Steve was able to fix similar issues in less then a week then that I’d end up assigning fewer tasks to Steve in the future, which would affect his output and that would eventually affect his livelihood.

or in XP fashion you could assign it to Steve and non-Steve to work on it as a pair?

Every field is based on trust, so I don't see why you emphasize that.

Once again, if as a manager, you are happy with your employee's output and feel that their compensation is fair; why would the knowledge that they could have done more change that? It was fair before. Why should the worker do more to earn their salary?

> Every field is based on trust, so I don't see why you emphasize that.

Most workers around the world have no "trust" from their manager on the number of hours they spend in a week.

> Once again, if as a manager, you are happy with your employee's output and feel that their compensation is fair; why would the knowledge that they could have done more change that? It was fair before. Why should the worker do more to earn their salary?

Because there is a tacit understanding between the two parties (if most cases, apparently you disagree) that the engineer is working the "standard" number of hours in a week (35+, more or less). If you tell your manager "I work one hour a day, and this bug took me three days to fix" and your manager doesn't mind, then it's all good.

But if you tell your manager "this bug took me three days to fix" I can guarantee he'll think it was a challenging problem that took 15+ hours of your time, and he would be pissed to know that everyone else was forced to wait for what was actually a quick fix.

As an engineer, I mean it kind of comes down to competition doesn’t it? If entity A is able to get something done 5x faster than entity B and just as well, wouldn’t you choose entity B in the future? Sure entity A might have been acceptable but at a certain point, who’s more likely to get a contract, a promotion, a raise, a pay cut, or laid off? This stuff matters, it’s how the world works, it’s no secret. Generally, pay cuts and layoffs are reserved for when the company is suffering but really what is happening the tolerance for slack is dropping, and productivity will absolutely play into that.

Nah. The people I’m thinking of are fucking magicians.

If they are hiding their true skill then OMFG.

Back when I was on product teams I could do this. The trick is knowing the codebase and problem space well enough that you're doing all the planning upfront in your head*, so the estimate given is basically just implementation of an already-existing plan, with few unknowns - just some padding for testing and bugfixing near the end.

Nowadays I'm on a maintenance team with products I'm not very familiar with, so when something needs fixing I have to spend time investigating it before being able to come up with that plan. I can usually still give vague estimates based on guesses of where the problem is, but nothing like I used to be able to.

* Okay maybe not all all, but enough that you have an outline of the entire solution where you can fill in the details as you go. Some details will increase the overall estimate as you get to them, others will be simpler than you thought and reduce the overall estimate - for me it tended to balance out.

This sums up my experience as well, giving mostly accurate estimates maybe comes naturally for some of us. I’ve been doing this for a long time though and I think that plays into it. It takes a lot of intuition that really just comes from experience but it does require a knack to some extent.

I didn’t think I was an outlier here but the more I read these comments the more it seems like we are.

I am 100% one of these people. Unless something really unexpected comes up, then I choose whether I'm working that morning (which is less than 50%) and will usually finish the work after a few hours, then commit it either at EOD or next morning. I am still considered an over-achiever by a vast margin.

Good for you. I don’t mind working with such people. I have zero interest in knowing how much of your 40 hours you actually spent working, only whether you did the thing you said you were going to do. An engineer that is more productive than everyone else but only works 5 hours a week still deserves to be rewarded the same way as an engineer who worked 40 hours but achieved the same result in the same calendar duration.

If you really can overachieve while working very little then all it means is that you could have achieved more in your life if you spent more time working. And that’s none of my business.

> If you really can overachieve while working very little then all it means is that you could have achieved more in your life if you spent more time working.

I mean, not really right? That's the whole thing with the totem pole systems we create. You are only rewarded for over-achieving so much. Anything above that causes problems, because we are jealous creatures. There are daggers in mens' smiles.

I have other interests in life and don't care that much about work, which is why I understand how it works in the first place. If I didn't like philosophy and speak four languages and lived around the world, then I wouldn't know that. Most engineers don't.

If my company were willing to make me a principal and comp me with a big chunk of equity, then hell yeah. But almost no one is, and until I have kids, I don't want to start my own company or put too much effort into office politics. C'est la vie, c'est la mort.

Manager here. Managers know this too. As long as the estimate falls somewhere between over-estimator and under-estimator, most managers don't care.

Good managers understand slack capacity is needed.

Dude. Stop giving away our secrets. Some of us want to have a reasonable WLB!

I just don’t accept that accurate estimates are actually possible unless:

- the requirements are well written and complete

- they never change

- the duration of the project is very short

- the number of interruptions is minimal

- there are no other priorities

- it’s a simple change to an existing, well defined system

If all of the above is true then sure, it’s possible to come up with accurate estimates.

But most of the time accuracy problems are not (only) due to the inability of the engineer to estimate, but also the overall uncertainty of the requirements and productive time available.

The narrative around estimates is that the engineer is responsible for things being late, but in several places I’ve worked, it’s largely the externalities that cause estimates to be wrong.

When I was able to control these factors, my team and I were able to make good estimates most of the time. But it’s really rare to be able to control them.

There is no narrative around estimates that engineers are responsible for things being late.

Strictly speaking I don’t think any of your conditions have to be true for estimates to be accurate, though obviously all of those things help.

I’m reporting empirical results here based on my experience. Your theory doesn’t change the facts of my experience.

> There is no narrative around estimates that engineers are responsible for things being late.

This is a strange thing to say since I read the article as being exactly about how the engineer becomes responsible for things being late. Otherwise why is the engineer "pressured to work unpaid overtime to meet that estimate"

> I’m reporting empirical results here based on my experience. Your theory doesn’t change the facts of my experience.

Well, your experience is totally different from mine. I've certainly had my feet held to the fire when a project has been delivered late due to changing management requirements and inappropriate prioritisation of management meetings over deliverables. I've seen others do it, and maybe I've even been guilty of doing it myself.

But we all have different experiences. I was just sharing mine.

> I don’t think any of your conditions have to be true for estimates to be accurate

What would be most interesting to me would be your view on how estimates can remain accurate when requirements change. How can that even happen?

Of course estimates shouldn’t remain the same when requirements change. If you told me the same estimate after I changed requirements on you then this would tell me that I should probably ascribe a high error bar to any estimate you make.

But you said that "I don’t think any of your conditions have to be true for estimates to be accurate" and one of my conditions was that the requirements should be correct and shouldn't change.

And the point I was trying to make was that - in my experience - is that often requirements changes that are a huge driver for estimates going wrong.

If you're lucky, you work in an environment where a changed requirement automatically results in re-estimation at the customer's cost. But in the places I work, the requirements are often not well stated in the first place, so they are subject to interpretation, and it is very difficult to estimate reliably in such an environment.

And for whatever it's worth, that's been my experience for the whole of my long career. As engineers we're forced to quote against crappy requirements, and as management we're forced to accept crappy requirements in order to win the gig.

But all too often engineering is blamed for late delivery of something that sales shouldn't have sold in the first place.

> like if Steve says he needs just one more day, he always means he needs five more days. So if suddenly Steve says he needs another week, then I know he probably needs over a month.

That is a beyond frustrating even if you have gotten used to adjusting the time in your head. It also probably means it affects other work that Steve is doing and his sanity. (Steve thinks he'll finish Task A in a week, and has already allocated time for Task B in a week's time, but he's not even close to finishing Task A a week later, and now he has to work on both at the same time. Which is fine, we're all expected to multi-task, but if he constantly isn't prepared for this then it's not sustainable.)

In these scenarios do you talk to Steve about what is up with his estimation skills and time management?

It’s usually not person-specific but domain-specific.

Like, I’m really good at memory management but whenever I go to write a new malloc or GC I somehow convince myself that I’ll Carmack it in a weekend. And then it takes five months.

But if I write a compiler then I’m usually one of those precise estimators. If I tell you a month then it’ll be a month.

So, I don’t try to talk to the Steve as if he’s the problem. He’s not the problem. The problem he’s solving is the problem.

(Or maybe I just make that excuse for Steve because as I said, I’m a Steve.)

Right. Requirements staying constant is not a requirement for estimates to be accurate because when requirements change it’s customary to ask for a new estimate.

If your mechanic gave you a 1 month estimate to fix your flat tire, they'd deliver on time every time

The precise estimaters are doing more than fixing a tire and they are doing it in less than a month. Empirically, these people are often more than average productive.

It helps that I also know how to program. So I know that the task in question is hard and I know that the patch they submitted is the real deal.

Get your bathroom remodeled in your home and have the worker tell you it'll be done whenever and you'll understand why developers (or any hands on worker) is lacking in their skill and practice if they have no clue how long it takes them to perform their work.

"Management" or a PMs job will then be to aggregate those estimates so they can estimate at a higher level than you, similar to a project management in a construction site. But each person needs some semblance of an idea of when their part will be done.

The same way if you were remodeling your bathroom and your garden and want to tell your kids when the construction around the house will be done, you'll need the gardener's estimate and the plumber's estimate, plus maybe some buffer at the end, plus some time to go with the wife to get a new shower curtain, etc.

If you tell your bathroom remodelers that you want green tiles after they have already done half of it with red and by the way you want the bathtub in a different place and you also aren’t sure yet if if you need a toilet bowl then their estimates will get out of hand pretty quickly. And your bathroom should be able to handle a 200 person family just in case although right now there are only two people in the house. And to save money you hire people from the street who don’t speak your language and have no experience but we all know one worker is one worker. They are all the same so when you hire cheap workers your cost is less without loss of productivity.

Why would you change plans and expect the construction crew's prior estimate to remain unchanged? Estimates change if the project changes. A commercial developer's job includes recognizing that and communicating changes upstream. Usually that leads to a negotiation, collectively determining how much to (try to) spend and which boondoggles are trutly important.

If a hired software developer told me up front, "I can't estimate because you will change things and then the old estimate will be wrong," all I can conclude is that this relationship is already dysfunctional.

In most companies it works this way "Give me an estimate based on what I am telling you now. I haven't really thought it through and there are a lot of unknowns I view this estimate as a commitment that should be kept although I will probably make a lot of changes during development"

I think this is close. IMO, the problem is that we're talking about a group of concepts using vague language. Is it an estimate, a target, or a deadline? "Estimate" gets used in place for targets and deadlines all of the time and it sets up situations where expectations aren't properly managed.

This is the trick, everyone in the room knows it makes no sense. You change the requirements and then hold someone accountable for an estimate that is based off of different, often simpler requirements.

What you do is extract an extra amount of economic output out of your worker, who wouldn't say no to that? Then when it hits the wall, it's the dev's fault anyway. If it works out, you get a pat on the back!

If the dev says NO, the dev will still have to do the work and may even be coerced into doing so. Many junior developers think it's their fault, but they're being manipulated. That's why nobody likes seniors, too.

> I can conclude is that this relationship is already dysfunctional.

Here's the thing, though: you wouldn't be wrong.

It may be different for contract development. For in-house development though, everywhere I've worked we've all always known that the relationship is dysfunctional. We also know that the dysfunction is everyone else's fault, not our own. So we just put up with it, because those assholes will never consent to fixing it, anyway, and just try to shift the blame to us instead.

Yeah I'm used to in-house, no material contract experience.

> the dysfunction is everyone else's fault, not our own

I specifically said the relationship. It takes two parties for effective communication to occur. An engineer who have given up trying because they assume experienced management will not listen, is just as bad as a manager that steamrolls their experienced engineers.

Yeah. What I'm getting at is that I think the relationship is doomed to dysfunction by a dysfunctional culture.

Of course. If management expects precise estimates then they need to provide precise requirements and be okay with adjusted timelines if they change the requirements. But that's fine. They can do that.

> then they need to provide precise requirements

> They can do that.

Yeah, no, they can't. Not at all.

While some, or a lot of that, is due to incompetence, it is also due to the nature of the beast: Software Development is an iterative process. You have to prototype and figure out what works and what not.

So regardless on how tight your processes are, whether you use agile or waterfall, it will never be possible to give perfect time estimates. You can only improve your risk-management.

No no no a bathroom isn't going to work anymore, we need a kitchen now.

"For competent people it should be pretty easy to convert the bathroom into a kitchen"

They best contractors I've worked with do just that, that is they don't give time estimates. They will say when they will start and how they will proceed but they never give estimates. They don't for the same reasons, they don't know. In their case it mostly revolves around sub-contractors and supply issues but the outcome is basically the same. There is to much noise to make an remotely accurate estimate so it is better to not do it.

I have a hard time imagining how such contractors would ever get a contract in the real world. I certainly wouldn't think of a contractor as highly skilled if they can't give an estimate.

Word of mouth. If someone you trust says they are good and get shit done then you hire them.

It's OK if you don't have money or time restrictions. Imagine that you have them.

Time yes, money no. They still give cost estimates.

We're doing applied mathematics, not bathroom remodeling. Try telling a mathematician to "story point" the conjectures they're working on.

I work in an industrial physics and mathematics domain, and in that domain specifically, I must say it is very charitable to describe a majority of software engineers as doing "applied mathematics", if we are to interpret that as meaning "software engineers are participating in the practice of doing research mathematics." Except perhaps in some extremely reductionist view of what it means to "write a program" (e.g., programs are proofs, Curry-Howard, or some other idea), I don't think "applied mathematics" truly characterizes the work of, say, someone writing out a GitHub Action or adding a new RESTful endpoint to list a company's product catalog.

There's a pattern amongst software engineers in these and other discussions. When it comes to discussing management and resource estimation, some people describe software as the most arcane, idiosyncratic craft to exist. In other situations (e.g., where actual research mathematics or computer science is proposed to be used to solve a problem), software engineering has to be "simple", "aligned with best practices", and almost anti-intellectual.

The pattern of discussion (here and elsewhere), to me, indicates a survival instinct of sorts, if that makes sense. To this end, there's a sort of hypocrisy if it means being a software engineer is as unrestrained as possible: software is purported to be like applied mathematics in situations where it's advantageous to describe it as such (e.g., "we can't estimate because proving conjectures is intrinsically unpredictable") , then when the proposition is brought forth to engage with software as applied math (e.g., "team, we can bound the 99%ile perf by proving this recurrence on resource usage, and that'll give us evidence to modify our search algorithm with an adaptive radix tree."), software engineers push it away for something satisfactory to a CS101 student (e.g., "we can just implement auto-completion of our product catalog by searching a sorted array of item names, why complicate it?").

I understand the troubles of estimating resources for tasks and projects, and indeed the requests can sometimes be unrealistic and overbearing—I don't mean to suggest otherwise—but even the most technically sophisticated pieces of software are flush with "ordinary" kinds of programming tasks.

In my experience just working by priority works better than endlessly estimating tiny chunks of a project, that doesn't actually give any real predictability. Workplaces that go heavy on estimation and "agile" methodology produce bad software and not very reliably.

Plumbers are doing applied physics, not rendering a button on a webpage. Try telling a physicist to provide an "estimate" for the cost of proving the big bang theory. Why should you expect an estimate for fixing your leaky bathroom faucet?

Half this site is crapping its pants over being replaced by AI being able to do their jobs with a prompt yet it thinks their work is sooo novel and unknowable.

If a computer scientist is doing completely routine tasks, they should be automating them, not estimating them every two weeks.

Most of software development is not computer science. Building a login page is a routine task.

Look man, even plumbing estimates are usually way off - but they know how long it takes to hook up two pipes that are 30' long for your new bathroom. They don't need to have meetings about it during the process. They don't need to talk to Home Depot about integrating with their broken API.

You're not seriously comparing research mathematics with software development, are you? You think you're expanding the breadth of human knowledge with your CSS tweaks of the shopping cart implementation? Give me a break.

It's not less research mathematics than bathroom remodeling, that's for sure. I never tweaked any CSS, did you?

> applied mathematics

Applied mathematics is a branch of mathematics. It's research. Someone studying solutions of the Navier-Stokes equations is doing applied mathematics. An accountant uses mathematics, but their work isn't "applied mathematics" any more than a plumber is an "applied physicist", or a veterinarian is an "applied biologist".

We're performing applied mathematics, but we're performing that task as engineers. The key is in 'applied'. Going from theory to practice takes planning and is not as unbounded as performing science in my opinion.

I think it's about balance, and a lot of software engineering projects would do better by steering more toward the academic research group model than the current fad of two week "sprint", plenty of non-technical pseudo-managers and endless micro-estimations.

What ever model works for a group of people producing software I'm fine with.

But as someone else said in this thread: If the question is more about who provides the estimate and less about why is there an estimate (also plenty of opinions about that in this thread) then I would say that time estimation is a integral part of an engineers job.

What makes someone an engineer is being able to provide a weighted solution towards the needs of a client or consumer. Not the best solution in the world, but the best solution for that client. If the requirements are that it should be delivered fast, you understand that the quality goes down. If the client wants high quality, it takes more time.

I’ve worked in construction. Estimates from contractors are notorious for being wrong. Almost every time.

But they are still expected to provide one, and to have some contractual requirements attached to said estimate.

So are software developers, especially ones doing contract work

Thanks for taking this avenue, the comparison to carpentry really annoyed me because all bathrooms or cabinets, or e-commerce/CRUD apps are the same right? (just because we're all tired of working on them doesn't mean our work is all copy and paste)

The most difficult part of any "hands on" job is time estimation - two people could give estimates of 3 days or 3 weeks on the same project and both be right.

I would assume the individual jobs could be fairly different, e.g. the plumbing doesn't come into the necessary places, shutoff valves not working.

Also, commissioner artists and authors are also asked for estimates. This is just basic professional courtesy when working with others.

When a contractor is remodeling your bathroom or your garden, the amount of time spent planning the design is insignificant compared to the cost of labor and supplies.

But in software, the planning is the entire project. The closer the plan is to covering every single possible case, the closer the software is to being done. In construction, the work doesn't even start until the plan is done.

Every one of these comparisons that involve a physical product whether it's contracting, bridge building, or manufacturing all completely miss the mark.

If you compare with other industries where the plan is the product, you'll start to see more similarities with software development. And the less the planning is cookie-cutter the more trouble those industries have with estimates as well.

Totally agree with you.

Looking at the comments only makes more obvious the snowflakeness and detachment from reality of the "software artists" bunch.

If they hire a guy to (re)model their bathroom, they expect it for free done yesterday. What I mean is that there's usually a complete turnover effect on the demands they have on people working on THEIR money and the lax and forgiving way they expect to be treated on OTHERS people money.

Well, much easier for a bathroom. It isn't like you have to have meetings, planning, interact with third parties, etc.. when redoing a bathroom!

If a client asks me for "an endpoint that accepts a text file and gives me a PDF" I can let them know how long it will take.

Most software projects can't be described like that!

What a funny analogy because these projects almost always run over the estimate.

I think a lot of people commenting are not giving a charitable interpretation to the question. Now why any estimate is needed, but why should the dev be expected to have the right prognostication? As a dev, I have to admit it's a fantastic question. The reason estimates go awry almost always come down to things outside my control. Why am I expected to make predictions based on that? The people getting paid salaries to have a bird's eye view and veto power over those kinds of changes that could screw with my deliverable should be the ones knowledgeable and accountable for that estimate. And if I agree to it and fail to meet it, the judgment should go above both of our heads: was I delinquent, or was I blocked because someone else didn't do due diligence?

I’m in precisely this situation right now. I’m an external consultant being asked by a dev team lead how long it’ll take for his team to deliver a project. He’s the one approving holidays, he’s the one allocating priorities. He’s the one that knows the skill levels of his staff.

I’ve been in the exact same situation a year ago, where another manager got mad at me for a delay. It was caused by him approving his entire team to simultaneously go on 3-month holidays when the COVID travel ban ended. Apparently that was my fault for not estimating accurately.

The people getting paid salaries to have a bird's eye view and veto power over those kinds of changes that could screw with my deliverable should be the ones knowledgeable and accountable for that estimate

But they are accountable to their stakeholders (clients, upper management, etc). They take your estimate and many others from other people in the team and work those into a delivery plan.

A good manager will know how to manage risks and remove blockers in a way that gives a developer the best chance to work within the estimate. A bad manager will usually have no plan and put all the blame on the developers if things go awry.

Agile frameworks often remove the developer as much as possible from the time estimation effort, instead relying on past performance, and attempting instead to have the developer focus on breaking work up evenly.

Sure, this helps by removing some of the bias, if done right. But many times devs end up perverting the original intent of the framework.

Eg: I've seen many scrum teams transforming story ponts into days or vice versa missing the actual purpose.

If you abdicate any responsibility to estimate the work how can you agree to any estimate?

I am responsible for telling the person promising how long the work would take as scoped out, given the competing or potential problems which that person is also responsible of informing me exist. It's not that I provide no info - it's that I can't be authoritative about delivery times in an ecosystem I don't entirely control.

I think this argument is not about engineer vs manager. It’s about junior vs senior engineer. As you gain seniority and tech-lead others you take on this responsibility. I agree that junior engineers shouldn’t do estimation, but rather senior engineers should set the expectations.

A key component of the due diligence you're expecting involves asking developers how long things will take. Savitha is building a new feature with about 2 more weeks to go, but it can't be deployed until a guy Jim on some other team deploys suchandsuch dependency upgrade. Should I commit us to a mid-April release date, or will Jim need more time? I don't know, I'd better go ask him.

Of course. What I'm saying is, I see failures IME as Jim putting something into the code base that makes Savitha's work consequently incompatible, which was not surfaced to her as even a possibility when she was asked to give her initial estimate.

It’s a very naive question even if I try to be charitable. I assume most developers would be more business aware?

A few obvious reasons:

- The buyer has to make commitments externally, for instance to customers, partners, finance, marketing, his boss;

- The buyer has dependencies on those external resources and needs to plan for them;

- The buyer has a limited pool of resources and needs to know when they are free for the next task or project;

- The buyer needs to get an idea of costs to complete the feature and secure the budget;

- The buyer needs to make priority calls. If feature X is significantly more effort than feature Y then we can prioritise accordingly;

- The buyer is paying and simply wants to know when he will get his shit.

I wonder if the person asking the question would be happy to let someone do a job in his home with an uncapped budget and timeline?

> The buyer has to make commitments

Not surprised this discussion devolved into this bizarro world point so quickly. The developer says “there are too many unknowns to provide any sort of estimate”. The manager retorts “but we need estimates!” As if they expect the developer to say “oh, you need them? You didn’t say that. In that case, the unknowns are now known and it will take four days, three if you remove this inconsequential detail that nobody cares about”.

Unfortunately this bit of illogic is so common that I know it’s a waste of time to argue, so I’ve learned how to figure out what they want the estimates to be, give them that, miss them (just like everybody else does), and then get to work on actually providing the feature.

This is why it’s a good idea to have ranged estimates. If a simple known thing with zero unknowns has maybe a 200% margin (1-3 days) then a more unknown task can have 10x as wide margin like 1 to 20 days.

As a developer I never get “that’s a useless estimate we can’t prioritize based on that/tell the customer that”. It’s a perfectly good estimate. It tells everyone there is a lot of uncertainty. The low lower bound indicates there is a chance it might turn out to be cheap (so maybe worth taking a gamble and trying it and giving up). The big variance indicates it could/should be given some time to prep/investigate in order to narrow the estimate.

When there are unknowns, make a wide estimate. Convey the uncertainty. If the unknowns are ridiculous, make a ridiculous estimate. Even “5-200 days” is useful because something else with less uncertainty can be chosen instead.

I agree with this sentiment.

I'm running through a similar situation now, where I'm somewhat adjacent to a client and a dev team. The client has a list of tasks he wants done, and he'd like banded estimates for the effort the dev team thinks it will take to complete each.

The developers are obliging, but one of them complained to me privately "why doesn't he just give us 2 months of budget and we'll get through as much as we can?" - which is an interesting statement. From the developer's perspective, this is ideal. They are very good and work hard and have integrity, so I'm sure something like that would work, from their perspective. They can just 'go' and get stuff done.

From the client's perspective though, it just raises a bunch of uncertainty. He wants to know what he could get out of his 2 months of development budget. If one task took all 2 months, and the others (15 in this case) didn't get done, is that OK? What if some important task wasn't that much effort? You'd potentially want to priortise that over a bigger item.

Just saying 'give us two months of budget' doesn't take the other party into account, and what they're trying to balance or plan for. I've sat on both sides of the table, and honestly just asking for a budget with no actual commitment to completion feels selfish in my opinion. Discounting their opinion because they're 'business' people and not developers, or treating them like an enemy to fight against won't win you any friends or long term relationships.

> When there are unknowns, make a wide estimate. Convey the uncertainty. If the unknowns are ridiculous, make a ridiculous estimate. Even “5-200 days” is useful because something else with less uncertainty can be chosen instead.

As a PM, I couldn't possibly agree more with this. When you give me 5-200 days instead of "we can't estimate this", you've started the beginning of a productive conversation. I can ask you questions about why the range is so big, and you can explain to me what steps we can take (research, narrowing scope, etc.), to reduce the range.

The SO question addresses this, though: after giving an estimate with a range, the developer is pressured to narrow that range, mainly by pulling in the farther-out date in the range, and then the developer is blamed when the project inevitably comes in late because of that.

Maybe the solution here is "stick to your guns", but that can have negative career implications.

I'm fairly certain that you aren't accounting for uncertainty properly, especially with long-tail events. Let's say you have a ticket to enable a config flag. That's low risk and relatively small. There's still a risk that there's the flag hits a silicon bug. Does your estimate reflect that risk and is it useful if 95% of tasks don't hit those issues? Consider also that estimates delivered by contractors or even other teammates will not reflect these risks and thus you look correspondingly pessimistic or even lazy.

For context, my team is in exactly this situation with several silicon bugs we found last month.

Yes - it's better to be accurate than precise. If you said "1 to 20 days" and it took 15, you were accurate. Being too precise gives you no space for the unknowns to show themselves.

> The developer says “there are too many unknowns to provide any sort of estimate”.

You are misunderstanding your job. You are a subject expert and paid accordingly. Answer with what the unknowns are, an optimistic and pessimistic take on them, what you could do and how long it would take to lift them.

If you don’t, your boss is going to ask you for exactly that in a roundabout way and micromanage you because you are forcing them to do your job.

It still baffle me than approximately half of developers don’t understand things which are obvious to any seasoned blue collar worker.

Thank God somebody said this. When I was working as an IC, I would be very communicative, highlight the unknowns, and get buy-in from my boss. If a surprise came up, I would make sure everyone knew the game had changed.

As a manager now, I get scoffed at if I even hint at a deadline. Excuse me what? We have a limited budget, a limited runway, and we need to prioritize which product bets we are going to make. This seems entirely logical, but a lot of devs I work with now find it impossible. They just enjoy tinkering away with whatever thing interests them, and delivering value only means what they personally find valuable.

Personally, I find it a bit pathetic and childish. I say this as a long time software engineer who knows perfectly well how this business works.

Blue collar vocational training is more mature than software development vocational training. It's typically taught by people who have professional experience, and there may even be an apprenticeship period where people are explicitly learning from an expert in their profession while on the job.

In software, on the other hand, we often have most training being taught by professional academics who do not understand and are not qualified to teach the parts of the job that don't have a straightforward academic analogue. Combine that with a business environment where things like budgets and cost control are often only theoretical, and all but the most gross liability for defects due to poor workmanship has been waved away by EULAs, let it marinate for several decades, and... yeah. Not only do people not know these things, but I'm not entirely clear on how or why they should be expected to learn them, either.

> You are misunderstanding your job. You are a subject expert and paid accordingly. Answer with what the unknowns are, an optimistic and pessimistic take on them, what you could do and how long it would take to lift them.

Add a small feature to an existing system. This estimate can be anywhere from several months to a day. Most of the error bars are org and system specific rather than anything to do with the feature. The exact same feature built by the same person in different orgs and different systems can be opposite ends of the range.

Seldom do people have the contextual understanding to provide an accurate narrow estimate, particularly since both org and system are in flux. They change constantly. Unseen landmines potentially await in both.

> If you don’t, your boss is going to ask you for exactly that in a roundabout way and micromanage you because you are forcing them to do your job.

And such orgs take forever to get anything done.

> It still baffle me than approximately half of developers don’t understand things which are obvious to any seasoned blue collar worker.

Software is probably the hardest kind of engineering you can do. All the system constraints are virtual. Just consider comparing a dirt ground for consistency and behavior vs libc... 304 stainless vs react.

You can go deep in both, but a shallow understanding of one takes you much much further than a shallow understanding of the other.

> Answer with what the unknowns are, an optimistic and pessimistic take on them, what you could do and how long it would take to lift them.

Yeah, most of the time my boss just needs to know if this job is a few hours, a few days or a few weeks. I never answer with exact numbers, but a range. If there are some major unknowns I let him know. If I made some important assumptions then I let him know.

Sometimes I'm wrong. Most of the time my estimates are sufficiently accurate.

Sometimes the customer really needs this done before some deadline. Then it often turns into "how can we make this happen before then". After a bit of back and forth with some added assumptions and cutting functionality, we usually end up with a solution that can be realized in time.

I mostly agree, but if it's a technology I've never worked with and I'm asked to do something I've never done before, I'm not an expert. That doesn't stop them asking for estimates, which turn into deadlines.

My optimistic/pessimistic takes at my last job would look something like "roughly a week to roughly 2 years"

Depends on how deep the yak shaving goes, and you wouldn't know until you got halfway through fixing the problem.

Often I could take an educated guess as to which side of the scale it came down on, but I've definitely started pulling on a string inside of the codebase and it just kept going and going and going. All of those baked-in assumptions that other developers unconsciously made over the course of a decade can pile up into a mess that you just can't implement the feature/bugfix without doing a pile of potentially breaking changes that need to be shipped in major versions.

Generally if I worked the problem for a week then I could give you a much better estimate of when it would be done, and often that estimate was "it's done now" after a week. But if I hadn't looked at that particular subsystem for a few years, I couldn't trust my memory of it up front and had to do exploratory surgery first.

(Really exploratory surgery is a much better metaphor for what you're going to be doing, and surgeons can routinely find surprises -- "oh look that barely perceptible shadow on the x-ray turns out to be a stage 4 tumor...")

Yes, the codebase was horrible, yes it needed to be cleaned up, no we didn't have the manpower to do that and it would have taken me 10 years to rewrite it all.

Quitting of course was an excellent solution.

Can you do it? Provide an estimate that's accurate and stick to it with the routinized scheduling of a blue collar job?

It's not an apples to oranges comparison. You give me an architect's blueprint and guys who build to the legal building code, and I could probably get a house built. Expensive and painfully, but it can be done.

I don't even know what the coding analog to an architect's blueprint is. They are fundamentally different jobs.

See in this case the manager has done a bad job, but the developer has also done a bad job, both in the exact same way.

The correct solution here isn't just to insist estimation is impossible and be offended that someone would ask for it - that just misunderstands how businesses work.

The correct thing is for either of those parties to say, "Okay, I understand it's not estimable at the moment. What are the steps that we need to take in order to get a reasonable estimate?"

A good PM will do this, but where there is a bad PM, a developer can step in and say the same thing. In your scenario, "The developer says “there are too many unknowns to provide any sort of estimate”." What the developer could say is, "There are too many unknowns to provide any sort of estimate - let me give you a few examples. <examples>. What I can do is come back to you after <amount of time> with a list of the unknowns and an explanation of the work required to evaluate them, so that we have enough information to estimate. Once you have that list, then you can decide if we want to proceed with the work to understand those unknowns."

I totally understand why developers get frustrated when they're asked to give estimates for things that can't reasonably be estimated with the current information available. But the flip side is it's also very reasonable for PMs to get frustrated and ignore developers who insist that estimation is an impossibility, because that's virtually never true - maybe it's impossible to do now, but there is a path to make it possible. If you work proactively to expose that path to the PM, it'll generally result in everyone being happier.

> There are too many unknowns to provide any sort of estimate - let me give you a few examples. What I can do is come back to you after amount of time with a list of the unknowns and an explanation of the work required to evaluate them, so that we have enough information to estimate

Sometimes we don't know what we don't know. If we don't know what we don't know now and if we don't know what we'll find we don't know later, then we don't know how long we will need until we'll know.

Uff. Sorry. If I had more time, I would have written a better comment.

> The correct thing is for either of those parties to say, "Okay, I understand it's not estimable at the moment. What are the steps that we need to take in order to get a reasonable estimate?"

Ah, that would be nice. In my experience it's more like, "Okay, I understand it's not estimable at the moment. Just make your best guess and we can revise it later. (Psych! I will hold you to this estimate like you were under oath in court!)"

This leads to things like the Scotty rule. Since management will always misuse, misunderstand, or misrepresent your estimate anyway, might as well look out for number one and inflate the estimate, just in case.

I don’t see how it’s unreasonable that someone would want to make plans based on the development velocity of software. Just because it’s actually hard to estimate some things accurately doesn’t mean it’s not valuable from a business perspective. The original post was someone saying that they should never be asked for estimates because every single thing they make is a unique snowflake of unknowable complexity, which is absurd. There are definitely features which an engineer can easily estimate delivery timelines on, and even knowing ballparks like “this is hard (as in weeks)” or “this is easy (as in hours/days)” is super valuable to communicate to actual business stakeholders.

In a healthy culture it should be possible to have a back and forth about how accurate an estimate is likely to be, but when engineers insist that all estimation is useless you can imagine why other stakeholders may lost trust.

> Unfortunately this bit of illogic is so common that I know it’s a waste of time to argue, so I’ve learned how to figure out what they want the estimates to be, give them that, miss them (just like everybody else does), and then get to work on actually providing the feature.

I do that and I've become exceptionally good at blaming the business persons for the outcome, making them admit their guilt in public and all that in what seems a friendly, positive and constructive manner and using business motivation and explanation. They never understood what hit them. Every programmer can do that if he or she learns a bit of psychology.

Can you elaborate?

The estimates are useful even if they're wrong. Not all managers realize that, so they ask the question badly. And they never teach it to programmers.

The estimates aggregate to a number more accurate than each individual. The manager cares more about the sum total than each individual one. Many can even slip without affecting the end date.

This isn't so hard, if management and developers didn't treat each other as enemies. Each can get wrapped around the axle concerning each deadline, rather than the real goal of allocating resources across the entire scope.

For these I like to say "by the time I can give you an accurate estimate, I will be done with the work."

Seriously though. It's always a time tradeoff. How much time do you want an expensive developer investigating how long this problem will take. Will they learn something they can use during development.

I always follow up my above quote with the unknowns and how hard I think they really are; what are the possible risks; _and_ I include the experience of the developers working on the problem; our ability to bring in other devs who understand this better etc etc. I like to make sure whoever is quoting this problem out understand everything as well as we can in the given amount of time.

Is this not what estimating and refinement means to developers? I’m always pulling a number out of my ass, but that number is generally informed by current and past work, unknowns, and a “gut feel”.

If that number is too high, then that’s a sign to management that the work needs to be changed. The only way to determine that is to estimate it then make that call.

App idea for tracking projects: Each individual developer can anonymously indicate how far along they believe a project is. For example, Developer A thinks the project is 60% complete, but Developer B thinks the project is 30% complete. Use algorithms to predict the completion date.

However, I don't think this idea would be very popular because it removes the manager's ability to control the presentation and play politics.

The original question was "Why are developers expected to estimate tasks at all?" @benjaminwootton gave perfectly sane answer to that question.

HOW developers should make their estimation is a totally different question, but it is not some arcane knowledge, and it should be an everyday craft for every mid-level engineer. First of all, we are doing those tasks by hundreds through our careers, so an easy way is to remember how long similar tasks took us last 10 times an produce a bracket, like in "from 5 to 8 working days, let's say 10 to be sure". Every business person will be happy with that answer.

OP says that every engineering task is unique as a piece of art. This is BS, IMO, except for the very junior engineer. Most of the time business wants some well-known and hundred times done things they see their competitors do. All the uniqness stems from particular codebase. If your codebase is a mess - you are in trouble, but it is you problem, not the general problem of software engineering.

In case I have some really unique task, what I should do is to say: "I cannot estimate this right now. Give me two days and I will get you an estimation". In those days I decompose the task into smaller, and divide those smaller tasks into "well known" and "unknown". Then I imagine (or even draw) a trivial Gantt chart and will have my estimation.

It is also totally normal to change your estimation during the work. You just should do that the moment you see a problem, not at the day of the deadline.

The response to "there are too many unknowns to provide any sort of estimate" shouldn't be exactly "well, we need estimates." But "okay, just do whatever" is also not a working outcome. Software is weird, yes. But other disciplines are able to provide useful planning information. Why are we unable to do that?

It’s monumentally frustrating to be forced to give estimates too soon in the planning process. The compromise, I hope, is to end up in a place where most of the problem is known and some is unknown, and you get there by breaking down the original problem. Leave it up to the “buyer” to digest your best estimate into their planning process.

Estimates are theatre. It’s not even theoretically possible to provide accurate estimates for anything but the most trivial project, yet here we are.

As an industry we just seem incapable of coming up with any other way of negotiating and running projects.

It gets even worse when you are in a line of work where the estimates are part of some kind of sales pipeline. Then you aren't just estimating based on effort required, but also against a likely number that the customer will ever agree to.

A general framework is that on each decision, including architecture, cost, time one at least adds the following:

1. Assumptions (including soft such as who will actually do it, resources are not equally relevant or productive) 2. Constraints (as agreed by stakeholders) 3. Risks 4. Clear scope 3. Alternatives and why they were not preferred

Business should be aware and own the uncertainties, or else the system gets biased towards sales people offering vaporware.

You can say this, but when a plumber or an electrician comes home, they are always expected to provide quotes and duration of work

The fair thing to say is that there's something like 80-90% chance that this will take X time, 99% chance that it will take less than Y time.

This would allow for planning, prioritising and is the most accurate communication we can achieve.

No, hard no.

In my experience good communication solves this every time.

MGMT: We need an estimate ENG: I have several unknowns MGMT: We really need an estimate ENG: This is my rough estimate, but be aware that the unknowns could change this drastically MGMT: Ok, what do you need to get a firmer estimate.

And so on.

As a project progresses, the communication between ENG and MGMT is crucial to ensure that MGMT are aware how those unknowns are affecting the project, MGMT will decide at every juncture if the project is worthwhile to continue to invest resources into it (assuming they avoid the sunk cost fallacy)

> If developers have to be the ones carefully calculating estimates and managing expectations then there's really no purpose to middle-management.

Op is not saying that having estimates is useless, but rather it should be a management task rather than a developers burden.

To me, that sounds even worse, but I do think devs are expected to do wider and wider tasks across the business. Their post reeks of resentment against management and PMs and I don't blame them. Especially in my experience, I have worked with PMs that I have no idea what they do as devs need to step in to do all project management related tasks themselves

Just to be clear, most of the time “management” comes to developers with a proposed timeline, there’s significantly more complaints than if the engineers are asked to estimate.

Asking the builders to estimate the time they need is a good sign of high trust in that team. It’s healthy. We should want more of it. The business’ side of the commitment is to understand that it is an estimate and should be treated accordingly.

>> it is an estimate and should be treated accordingly.

That's where it all breaks down. What does "treated accordingly" mean? In reality it means the estimates can't be used for anything because they are always unreliable. There is no such thing as a reliable software estimate, ever, unless what you're doing is so repetitive that you're about to be replaced by AI.

But that's not why people ask. Instead, half the time you give an estimate it gets immediately turned into a deadline, even if you were promised it wouldn't.

The top comment on this thread is naive. The reason devs hate giving estimates or point blank refuse is because it is meaningless, that's not how the software business works. Analogies to plumbers and builders just reinforce the naivety. Guess what, blue collar work is only predictable for as long as it's highly repetitive which being physical in nature, it often is. The moment what you're asking for is "build me an underground railway using the latest technology" it turns out construction estimates are worthless too (see: Crossrail).

One reason tech firms crush their competition so reliably is they don't have an estimates-deadline culture, because they're run from the top by programmers who understand intuitively why they're pointless. Instead developers are incentivized by equity, bonuses etc to do the job as fast as possible.

I don’t mind someone giving deadlines because I can then scope quality to fit those expectations.

If you've worked in the industry long enough you work with a PM who will ignore everything you say and ask for a delivery date, and then ignore everything you say until you give the date that the management or client wants. What they won't do is give you the resources and support you need to hit the date.

> Op is not saying that having estimates is useless, but rather it should be a management task rather than a developers burden.

Count how many people are also complaining about their managers giving timelines to clients. Damned if you do, damned if you don't.

One thing I find frustrating is that non-engineering people in tech companies don’t get held to as high standards of productivity because it can be harder for especially engineering founders to evaluate whether they are doing a good job. This can lead to people thinking those roles are “worthless” and then Continuing to have low standards for those roles.

The best combo project/product manager I had didn’t have either official title, but we were on an internal team and somehow she always managed to make sure people around the company knew what we were doing and delivering, pester people so that no tasks were dropped, etc.

Interestingly, she did most of this by putting tickets for projects into Google docs, and otherwise didn’t buy into any of the standard project management dogma. She had an ear/nose for who might want to use our stuff and was relentless at getting us on internal presentations of any teams/groups that would be interested.

I don't think there's any question that people always want this estimate. The problem with most software is that there really isn't one, since it's usually an open-ended search for a solution.

Also, a job on his home is exactly what it should not be compared with, since that's the sort of repeatable job that can be estimated.

>since that's the sort of repeatable job that can be estimated

To some degree and depends on the job. Start opening up walls in an old house and tell me how cookie cutter it is. When I used to run shipyard jobs on offshore drilling rigs, I can tell you that every single job ran into all sorts of problems that were individually not predictable.

It's also just nonsense to think that most software is this uniquely open-ended snowflake. Most engineering jobs require problem-solving that can be affected by all sorts of things in the physical world--plus you sometimes just have to build and try out designs and maybe they work, maybe they need to be refined a bit, and sometimes you have to toss them out and start again from scratch.

I find it kind of funny how both sides take an "it's just not realistic" approach to each other's complaints and then the compromise is usually "You pretend you can make an estimate and I'll pretend to believe it".

That begs for supporting evidence. I doubt most software is open-ended research.

Even if the job was incredibly small and predictable, my point is that he would likely not be prepared to sign off an open ended project with an uncapped budget.

If the expected ROI on a particular project is so razor thin that accurate estimates are needed, don’t even start. Be sure to do your due diligence and aim for at least 10X (ideally > 100X). If this is provably true, then accurate estimates will not be needed. If this cannot be concluded, simply live with the limitations of off-the-shelf offerings.

Tom DeMarco expressed the same exact point:

> https://www.computer.org/csdl/magazine/so/2011/06/mso2011060...

I think this is a big part of it. For every manager with an unrealistic schedule demand, there's a manager above them with an unrealistic ROI demand. The projects with accurate schedules and accurate ROI's don't get approved.

That sounds great and all, except that estimates are usually wrong. It’s a real problem when estimates are treated as deadlines, which is exactly what you’re describing.

The problem is the writer of the question is conflating two things. Their quote: "A. Give a very wide estimate with a lot of padding. B. Get pressured to be more accurate and to bring the estimate down..." The real problem seems to be getting pressured to reduce their estimate, they only summarize things as the problem being that they're asked for an estimate at all.

I’ve known people whose version of A was “between a year and five years” for something where five years was clearly a ludicrous amount of time to spend and a year would be stunning. In those cases, the problem is that they aren’t really bothering to give an estimate at first, so the pressure to give one continues.

This is also hard to agree with:

"Software development is not carpentry. Almost everything a developer writes is unique, they have never built that particular thing before. We are not cabinet makers repeating a variation of something we've built hundreds of times before."

Some software development fits this description. Quite a lot of it does not.

Yeah I’d hate to be working with this guy. Your coworkers care about when things get done, too. Nothing more frustrating than to sit on your hands because of a dependency on some other team.

Agreed. Having human dependencies who have competing priorities or unknown competencies on the critical path is such a big risk on timelines.

In some cases I’ve got agreements up front that if the dependencies aren’t done or being worked on by particular dates my team gets access to the codebase or systems and can take over. That type of consequence/potential embarrassment can set up potential adversarial positions but it has helped a few times.

You've identified the problem but missed the solution: don't bet your business on a dependency that doesn't exist yet. Be useful enough to work on a different project, or take a furlough until you are needed.

Sure! We agree - though it would be nice to know if I can come back to it tomorrow or in six months.

Give them a break, they're venting but they're raising valid problems within the workplace (don't go by the headline alone). I've personally been in this spot and had to seek outside help to identify issues within my workplace!

They key is negotiations, and both parties being able to balance a moving target of requirements and scope. Large software development is not a determinable thing as much as we would like. Perhaps ChatGPT will solve this?

>> I wonder if the person asking the question would be happy to let someone do a job in his home with an uncapped budget and timeline?

Even this is not always accurately determinable! If you've had work done in your home, or heard anecdotes from others, you would realize this.

I'm not sure why you feel the need to falsely frame the point as a request for a blank check.

I get all the reasons you provided, but at the end of the day an honest discussion needs to ask is estimating tasks effective.

Do developers get more done or less done when estimating tasks vs not estimating tasks? The only claim estimates would increase productivity is the extent they are deadlines. But I think you will see that when developers take meeting their estimates seriously their estimates tend to get more conservative. And then there can be a desire to not over deliver on the estimates to lend those estimates legitimacy. Without estimates, you still have a performance metric to use, individual velocity. If one developer isn't getting stuff done as fast as others, you don't need estimates to know they aren't performing as you would like.

Are estimates reliable? This of course varies greatly by the project and the team but I my experience estimates are typically no more precise than 50% of their length. So a 6 month project probably shouldnt take longer than 9 months. Your mileage may vary, but is there a way to crudely estimate a project without asking for estimates? I would say probably. Why not just measure the teams velocity and project it?

I would argue estimates were a first guess at a way to run a development team which has survived mostly because of a refusal to consider the first guess might not have been the best guess.

I do wonder if developers would accept the same from people they "employ" to do a task.

I imagine sometimes I renovate my kitchen, an expensive task, within my budget but with not a lot of leeway, I can't renovate the kitchen twice for example. If the workers say they don't know how long it will cost and they won't provide an estimate to when ... I would not be happy.

Even creatives, writers, musicians, graphic artists have deadlines and must estimate their work.

But we feel that our work is special and should not be asked to give an estimate.

Hear, hear. Anyone who has worked in a quasi-management or leadership position before will understand the need for an answer to the question: "We don't have unlimited finances. Tell me how much money I need to put towards this."

Having said that, most developers refuse to respond intelligently to these questions. The answer to almost all of these question should be something like "I think delivering this goal might take X resources, but here's the smaller goals we're going to deliver along the way to the big goal so that we can reposition when we go off-track."

Instead, developers say "Well we have to make sure that EVERYTHING can fit into this timebox, and our manager is going to treat this estimate as a deadline, so estimates suck!"

This question is supposed to be a conversation starter. Don't shoot the manager for doing their job. And even when the manager does a poor job and treats your estimates as deadlines, call it out! It's up to the developers to put up a bit of a fight and make more intelligent decisions too.

I've only met a handful of developers who are able to have a conversation like this. It's what the 'Agile' movement was attempting to fix. But it turns out conversations skills are tricky things to impart.

> I wonder if the person asking the question would be happy to let someone do a job in his home with an uncapped budget and timeline?

Huh? Every major contract job on my house and my friends' houses that I have heard of has taken longer than originally "planned". More often than not unexpected expenses come up along the way. The same is true for commercial construction/remodeling. That's just how these things work; pretending otherwise is going to be a huge source of friction.

The difference between the buyer of a house and the buyer of software is that the house purchaser is clearly and succinctly able to communicate his / her desires and expectations. Where as the purchaser of software is making some vague request for functionality with poorly defined boundaries and requirements. There is little meaningful comparison of the two scenarios that can be made.

And giving people a precise and inaccurate estimate makes this better? A capped time and budget won’t make it take that amount of time or money. It takes what it takes. The complexity and amount of randomness in these processes mean any expectation has wildly large variance, with a long tail in the wrong direction. If you want home builder like estimates, build a home, not software.

It's not just about external factors, you can't let you dev go dev for an infinite amount of time. Task should be done at some point and you should roughly know how long it should take.

> I wonder if the person asking the question would be happy to let someone do a job in his home with an uncapped budget and timeline?

The author made the claim that software engineering is fundamentally different from carpentry ("Almost everything a developer writes is unique, they have never built that particular thing before. We are not cabinet makers...")

But does that hold up? Many software projects do in fact have a similar analogue, and each house is unique and carpentry projects will thus run into unexpected snags.

He/she hasn’t been around enough to see these patterns yet. The amount of times I’ve had to implement security, rbac, rest crud, modeling, service integrations, etc are too many to count. I rely on packages that help reduce the boilerplate and because I’ve done that so many times, I can give you an estimate on roughly how long it will take.

The fact that the OP doesn’t see this is fact that they are too junior to be asked for estimates at all.

Or they’re very senior in a pioneering domain that doesn’t have all these packages written.

It really depends on what domain you are programming in.

> - The buyer has to make commitments externally, for instance to customers, partners, finance, marketing, his boss;

Usually a business person makes some promise to the buyer or his/her boss that a certain feature will be done in X amount of time without consulting the team who does the work. The result might be overworking, resentments, stuff not being done in time and technical debt.

You can't have it both ways. Either you ask the developers for an estimate and you get ridiculed because how can developers possibly have any clue how long it's going to take them something? A day, a month, a year, who knows how much time this CRUD service will take! Or you don't consult with your developers, but then you've made an unrealistic promise with a number pulled out of your pants and you're the cause of all the suffering in your team.

No. Realistic is talking to your team exposing all the knowns and unknowns, refine the feature as realistically possible, while keeping the client in the loop, and once the feature is decently refined you ask the team for an estimation, take the higher limit and add a buffer for things that are not development related.

In the best case the feature will be done earlier, in the worst case you won't look like an idiot which promised things and didn't deliver and you won't compromise your relationship with the team. Compromising relationships with either clients or tech teams aren't good for your career.

More simply put, try to be as honest as possible. Not over promise, not under deliver.

One of my former girlfriends went to a business school and worked in banking. She thought people in IT have better salaries and she thinked tech people are a bunch of idiots and it's easy to push them to do things and extract a fat paycheck. So she did some courses on IT management and applied to lots of product owner and product manager role. After tens of interviews she hadn't had one success. To add insult to the injury, I showed her I make more money as a team lead than the product owner. Of course, the reverse is true and I've seen product owners being genuinely useful and having lots of business knowledge, some even programing knowledge.

Some people are just after the money, no matter what role they occupy in an IT company. And they tend to be the least useful persons who are blocking others to reach full potential.

Absolute estimate done right force scoping decisions which can be very valuable.

Relative estimates like planning poker can reveal hidden assumptions.

Commercial estimates can reveal risk appetite, desperation and more.

All estimates create insights but imho. it is best to burn the paper with the number after writing.

And yet at most companies an army of "facilitators" exist, with access to extensive tracking metrics and tools. They should be able to at a minimum ask the right questions (i.e. not "how long will x take") to create and maintain the estimates.

I wouldn't hire someone to do a job in my home that they've never completed before.

this assumes that there’s a buyer of software, and that the buyer decides which software to build. In those cases estimates might make sense But in many other instances (product/engineering led companies, start-ups) software isn’t built for a buyer, it’s build for a user - and a user doesn’t care if it took 5 days or 15 weeks to build their favourite feature. What they care about is that you build features those users actually want.

In those cases estimates form part of a whole basket of mechanisms and rules designed to ‘shield’ developers from users and vice versa, which imo is among the biggest mistakes one can make in such software companies.

A user doesn't care. Your boss cares if the competing company built the feature in half the time as you and snatches half your user base.

Every project I ever worked on was more like “okay we already building it” and estimate was the last step after spending hours of discussions. My manager was typically had enough clue to estimate themselves too.

And your response naively suggests the business people are being above board.

The real reason is to provide a scapegoat for the 50% of times it fails (ratio from Mythical Man Month)

"But but but but OUR organization isn't like that!"

Sure.

- The manager needs someone else to blame.

> Software development is not carpentry.

I found that assumption to be ludicrous too.

agreed with this. it's almost like this person has never had to pay for labor before

You’re all acting like he does know, he’s just not telling you. Step back for a moment and consider how things change if he actually doesn’t know and not even a water boarding session will get him to tell you how long it’s going to take. Now what? You either live with the uncertainty or fire him and replace him with somebody who can conjure up estimates - which, presumably was what you were trying to do when you hired him in the first place.

Basically the author is lamenting that estimations are NEVER accurate enough. I've not known a dev who made accurate enough estimations to avoid a crunch.

It does seem like the responders have a little bit too much sunk cost in the whole famously fraught agile thing to understand critique of it unbiased. It's fine in theory but does have flaws.

One flavor of agile (XP) says that estimates longer than 3 weeks diverge from reality too much. If you want to be able to estimate accurately, you must break it down until every task's estimate is less than 3 weeks.

That doesn't give you an accurate estimate, but it does usually give you a less inaccurate one.

I'd probably argue that any accurate estimate almost has to have some slack built in. For any task of any complexity, especially one requiring coordination, things rarely go more easily and quickly than you expect. Far more likely, you run into problems or you really need to talk to a person about something and they're on vacation.

> I've not known a dev who made accurate enough estimations to avoid a crunch.

And yet after three years at my first job, I could easily do a quick breakdown on a project, assign a three level complexity to each step (easy, medium, hard), put an amount of days corresponding to the selected complexity (to be reviewed after each project), add 25% for unforeseen events and contingency and my estimates were accurate enough most of the time.

It is not difficult to properly do rough estimates. Most developers just refuse to invest the time necessary to do it properly and sadly for some as the answers here already illustrate I do believe it’s because they think of themselves as special snowflake rather than team player.

> Get pressured to be more accurate and to bring the estimate down

Of course people want it quicker, but this is the problem, stick to the estimate and the problem goes away for the most part. Negotiate on scope if it is too long.

Communicate as early as possible if there is a mistake in the estimate.

At that point the problem largely goes away.

Edit: When communicating about the mistaken estimation, include which assumptions or surprises came up.

> stick to the estimate

If you were right. I’ve never met anybody who could accurately estimate a software project, and I’ve been doing this for 30 years.

It is a rare combination to have the technical skill and experience along with soft skills like humility that make it somewhat rare. But useful estimates are absolutely possible.

The question is one of range, with humility you can hit estimation ranges defined by how familiar the project is and how well defined the project is.

There are many developed techniques that we just don’t implement. By in large, most estimates I have seen were some weird gut. Not defined no milestones. “Yeah, about two weeks boss”

Often estimates come before requirements. Those will be wrong more often than not.

Many times I will estimate the estimate. The “bigger” the project the longer the estimate takes.

People are afraid to admit any knowledge gap.

People switch jobs every 2 years, I have been at the same place 10, and before that 8. You learn the cadence, you learn the pain points. You learn the traps. You learn the domain.

These can be accurate.

But this is not the OP’s issue as stated, he is lowering estimates in the face of pressure. This will nearly always fail.

Yeah, all models are wrong, but some models are useful. As far as I understand, the goal of estimates is not to pressure you, it's to promise something to someone. As long as you are good enough, business decisions can be made. And that includes figuring out when something is off track, which is one of the primary goals of management.

Whenever I read something like, "Software development is not carpentry. Almost everything a developer writes is unique, they have never built that particular thing before. We are not cabinet makers repeating a variation of something we've built hundreds of times before." I just assume that the person is trolling, or a child.

Virtually all software developers are building minor variations on thing that have been built hundreds if time before. Sure, maybe not by that particular software developer, but by a thousand others at some point, and half of those have probably stuck their code on GitHub so you can look at how they did it.

This idea that software developers are somehow cranking about beautiful snowflakes of such uniqueness that it is impossible to ever estimate how long something will take is nonsense. It is delusional.

I really wish as a profession we could get over ourselves and acknowledge that, hey, it is possible to learn from others, that much of what we do is repeatable, and that if you're really struggling with estimating it probably isn't because you're engaged in ground breaking work that has never been done before, but rather that you haven't bothered to think things through.

I generally think the plumbing analogy is a pretty good one.

It even works for estimates! We're often asked the equivalent of "how long will it take you to hook my sink up to my toothbrush for my Thursday home parties" and...

well it's nothing new. It's just pipes. Anyone can do it. But estimating it requires an absurd amount of context which is not always available. How far apart are those two things? Is there enough water flow to power the toothbrush, or will you need to upgrade the piping to the sink too? Sometimes the house has everything made of marble and it'll cost millions of dollars to replace the holes you need to make to put those pipes in, though piping itself is only like 30 minutes. Sometimes the toothbrush is a legacy model that only works with deionized water from Uruguay that has been flown within 50 meters of the south pole in the middle of March, do they have a reservoir available for that or can you convince them to just buy a modern one from Walmart to replace it? Does a DWUSPM50 water supplier even exist any more? Oh, you meant only during those Thursday parties? Is there a way we can tell when those are happening, or can they just pull on a lever to change the system? Does GPRD require the pipes to physically separated by 1m when turned off, or is a valve good enough? Why in the world does the customer require all horizontal pipe runs to only go North unless they're underground?

None of which is hard, and the plumber doesn't have to build it all in many cases. But how long is it going to take? .... somewhere between an hour and the end of the universe, most likely, unless there's proof the request is impossible.

It's not a perfect, beautiful snowflake, but it is a snowflake.

Scrolling through a thousand others' code on GitHub, is not what a cabinet maker does.

You want a standard for all programs you could build, put it in the CS degree or a 1 week training course that gives you a license at the end. Like a forklift guy goes through and happens to come out the other side knowing all the basic rules. The forklift guy doesn't have to scroll through thousands of GitHub projects to infer from studying, what could be taught in a week with some decent management and/or training.

When there's no standard, literally of course everybody's a beautiful snowflake who comes to their own conclusions. We have trained people to be that.

Of all the suggestions I could make why not cut to the deepest one. We're not going to have standard programming patterns understood by your average-joe-coder fresh out of university, until those patterns have been integrated into the English language like Greek drug terms and Latin legal terms.

You can learn and build a entity component system for a video game in a week, come back after a month of exciting holidays later, and discover you've forgotten what an ECS even is. But the biological concept of evolution (or contract law, or anything else) is learnt-once and stays with you forever.

It's time that the basic programming concepts have fixed names that don't change. That are embedded in the English language, taught at school from grade 0 and are distinguished from normal English words with some ancient-yet-familiar dialect. Ancient Americana if it were possible.

I'll make a standard. The next generation of children won't even realize they already know "how to program" by the time they are 9 years old, and they won't argue about the existence of these concepts, nor forget them, they'd just argue the style of them. We'd skip generic English words like "game loop" and acronyms like "ECS" and have real identifiable phrases in a familiar-but-old language that refer to powerful concepts.

/soapbox

> Virtually all software developers are building minor variations on thing that have been built hundreds if time before.

Speak for yourself. I build distributed systems and unique solutions that you can't find anywhere open source, which makes estimating very difficult.

Exactly. Many commenters are making up rules for software engineering and think the rules ought to apply universally.

> Virtually all software developers are building minor variations on thing that have been built hundreds if time before.

I've never written a piece of software that has already existed that could do the job -- why would I -- that's why we have a thriving software market of purchasable software. The market of reusable software is massive.

> This idea that software developers are somehow cranking about beautiful snowflakes of such uniqueness that it is impossible to ever estimate how long something will take is nonsense.

Your argument is that you can estimate minor variations. But how do you know the consequence of adding feature X is a one hour job or a one day job? If you have 50 minor variations or 500 minor variations that are each maybe one hour or one day jobs, how long is that project going to take? I've got a unique snowflake here.

In most other industries, they can estimate quite accurately because they've spent the time planning all the details. The cost of construction and labor absolutely dwarfs the cost of planning. With software, the planning is the entire project. The closer the plan is to covering every single possible case, the closer the software is to being done.

You're saying we've been calling it "Software Development" in error, and it should really be called "Software Production"?

OP is working in a dysfunctional workplace and has conflated that with "developer estimates are bad":

> 1. Give a very wide estimate with a lot of padding > 2. Get pressured to be more accurate and to bring the estimate down > 3. Get pressured to work unpaid overtime to meet that estimate > 4. Watch management get congratulated by upper management for running a tight ship

Imagine I had plumber at my house and he gave an estimate of 3 to 4 hours to replace a rusted pipe. And I responded with "I think it should only take you 1 hour". He would look at me like I was crazy. That's because trying to substitute my layman's estimate for a professional estimate is crazy.

OP is providing estimates - that's a reasonable ask. The problem is management is ignoring the professional estimate and using their position to substitute in their preferred estimate. OP is being used to launder management's expectations. So of course they're patting themselves on the back. If the plumber ignored his own professional experience and only charged me for 1 hour of labor I would think of highly of myself as well, even though in reality I had no idea what I was talking about.

The solution is to set estimates that you can explain and justify. If management ignores your professional opinion you need to make it clear it's their decision and not yours. If you don't respect your own opinion no one else will.

I think what is hard with a lot of tech is that developers are often not plumbers. We are not encountering similar sets of problems nor are able to use similar solutions or even refer to familiar tooling all of the time. It's more akin to when the plumber crawls below your house with nothing more than a flashlight, then recoils in horror that your sewer is plumbed in 200 year old rotten wood pipe, then they start digging and hit more pipes that aren't on any map and shouldn't be there, then things get expensive and timelines enter the unknown as the problem gets seemingly bigger the more of it that you solve. Of course the client never sees this, they think you just twist a wrench, so we have these deadlines and talented people burning out over the stress they cause them.

> We are not encountering similar sets of problems nor are able to use similar solutions or even refer to familiar tooling all of the time.

Frankly, yes we are. There’s only so many ways to solve a problem and there’s only so many problems to solve in a particular field.

A senior engineer should have had enough experience to at least have a high level understanding of what is going on, and to correlate that experience with approximate work. If that’s not possible, the work is just not scoped down enough.

A team of engineers, given their combined experience and business knowledge should pretty easily come up with a story point estimate based on existing completed work. And that’s the entire point of the refinement and estimation process: getting enough information about a piece of work, then working together to come to a decision on an estimate.

Will that be right all the time? No. But that doesn’t mean the value is useless, it provides plenty of information on if a piece of work is small, medium or large, as well as if it’s expected to take a few hours, days or weeks. And who better to come up with that number than the people actually working on the issues?

I haven't seen anyone talk about the elephant in the room - only part of the reason for giving estimates is practical. Most of it is political/psychological:

https://news.ycombinator.com/item?id=23093789

The political solution is to give an estimate assuming nothing goes wrong, and then when things go wrong blame those things.

This lets everybody from you up the chain look good while not changing anything.

As someone who moved from a practical engineering field into software engineering, this separation in tasks between management and developers seems weird. Engineering in general is about finding a local optimum in a multidimensional problem space, one of the axes could be time, another one could be complexity or agility. For me, an engineer is not just tasked with outputting code, but also working out these optimums.

Can you expand more? This seems like an important insight to me.

Firstly I think the most important thing in engineering is nothing is free. If you change a parameter in your design something else has got to give. If you increase the cargo capacity of a plane it is probably going to have a slower cruising speed. The hard part is finding the sweet spot that exactly fits the requirements of the client or use case. And also! now a dosen of other parameters have shifted to, like weight balance or climbing rate. How do these fit in?

Now for software, let's say you have two clients with almost identical requirements for a piece of Software. The only difference being one of them thinks time to market is important, the other one has less focus on time. As a software engineer you should produce two very different pieces of Software for these clients. You will have to make decisions based on the time it will take to implement something and also understand the effects of this on other aspects of the end product.

If this person worked on my team, they wouldn’t any longer. Not even addressing the question at hand of why we give estimates, the idea that you just want to code without any repercussions shows me that you don’t understand the job.

Companies (most) don’t hire you to perfect the art of software. They hire you to deliver business value. Estimates are them trying to frame how long it takes to get that value. Coordinating the delivery of that against other company projects/plans or customer project/plans is the business you are in. If you want to just code and experiment, go pursue a PhD and stay in academia.

The people who work for me, with me, report to me, or interact with me know that I’m always focused on value. I could program the best platform I could design but without customers that see value in it, it’s worthless (or worse, costs you money).

You use a platitude(creating business value) to reach a non sequitur(that's why developers must estimate their tasks).

To say that a developer is there to create business value is a truism that's completely irrelevant to this discussion.

The fact that you see creating business value as a platitude means that you are in a position where any ask of you to help the business succeed will be met with argument. My non sequitur is only a non sequitur to you because you don't understand the statement. You wouldn't say that if you were in a position where delivery was dependent on deadlines and in order for you to capitalize on it, you must deliver before/on the deadline. To be able to tell the business "This is what it's going to take" so that you can execute and deliver is called estimating. Something every developer should learn to do.

Deliver but deliver what?

What does value mean? Unless you can ask for precisely what you want (none 1 can in 1 go even if they think so) then if you don't ask how do you deliver?

What's the point of providing an estimate that's useless?

If you talk about focusing on value then estimates provide no value to that value. All they do is serve a reporting purpose - not delivering in any way to make money so you've just contradicted yourself.

So let me ask what value are you asking for? Do you want a website? Well ok you'll get a blank page that says "WEBSITE". Clearly that's not what you want so how do people estimate nothing? Even the owner likely doesn't know what they want. You may feel like it.

As others said you'll just lose value because if I come up with x/y/z that DO add value I'm no longer going to mention it because it's outside of estimates and clearly not valued. If I see anything that's wrong with what we're trying to deliver again I'm not going to mention it because it doesn't add value!?

Here’s a concrete example. A customer of mine wants to process some data, this processing of data is needed for their fizzbuzz widget they are developing and they need the data before they launch the widget. They want to know how long it will take, at what cost, because I’m bidding against others who also want to do the job. In order for me to continue to operate as a business (and pay staff), I need to sell, in order to sell, I need to give an estimate to the client on how much effort it will be so that they can plan accordingly or select a different vendor.

This is an engineering task. Estimate the steps and time it will take. Deliver it. Plain and simple. You don’t know what the data looks like so it’s going to be new. You don’t have a framework. You have some semblance of a schema but it’s written by business folks.

Like all the other commenters have said you're not really asking for an "estimate". A fixed cost project does NOT have estimates. Estimate occur when you do time and materials i.e. you get charged per hour and it can always get extended.

There's competition and you roughly know where the bid will land (assuming you're already experienced) and likely talking to each other. Instead of giving a real estimate anyone knows you pad any real estimate 3x and then drag it out if you're under time.

I once tried to give a real estimate, finish it under time and got told for it because then we can't bill the client (time based).

Again, no it's not an estimate. It's called I have $x and y time and want to do <z>, how many corners can I cut whilst the client won't notice?

> If this person worked on my team, they wouldn’t any longer.

So, literally, if you found that a member of your team was the one who posted this question, you would work to get them out of your team? Fired, managed out? No discussion? No, for instance, conversation about how things could improve?

No, not in the slightest. I’m not one to hunt people down. I would tell the author that here’s what we do, why we hired them, if that isn’t in alignment then we wish you the best and will gladly offer a referral towards your next endeavor. Jesus. I’m not toxic. I’m also not going to let toxicity ruin my team(s). Such as baulking at requests for timelines. If you don’t know, say you don’t know to your manager and let the manager, manage. The author doesn’t understand their place in the business just like you don’t understand that you can have hard conversations without playing office politics or being a toxic manager. Hunting people out that don’t believe in the mission. I never work to get people fired. They usually do that on their own.

My point about them not being on my team is that I don’t run a daycare. I also don’t run a research firm with endless cash. I pay you a ridiculous salary to deliver business value. I get paid a ridiculous salary to deliver business value. Anyone in our profession needs to understand this. $150k+ salaries aren’t necessarily normal elsewhere. I’ll pay top dollar for top value but you have to deliver. Does that make sense?

> I don’t run a daycare, etc.

Non-sequitur. These are obvious statements that don't have bearing. I hope you never actually say "I'm not running a daycare" in front of your team. It's like having a sign in your fancy restaurant that says "No spitting". Your clientele will slowly but surely become the kind of people who needs such direction.

Empathy, transparency, psychological safety are critical for software engineering teams, as well as daycares, to operate well. This is well known in contemporary practice. OP is obviously suffering under some kind of mismanagement. Whether it is fixable mismanagement depends on whether his managers respond well to feedback.

As for estimations, the most accurate estimations I've ever seen are when the team goes over the task together and then each member estimates with blind votes, and then discuss outliers together if any. Better are scrum "stories". The least accurate estimates are when a producer wanders over to the desk of the distracted developer and secures a commitment. Which does happen unless there are explicit mechanisms in place to prevent it.

Is this Twitter where janitors get fired because they don't add "business value" and we have to all bring our own toilet paper?

Maybe there's something wrong with this concept of business value then. There's more to it than just a feature = $x. Can you add a new window to your house just by adding $x?

The people who work for you, are their estimates accurate? What fraction of their time is spent on them? What have you as a manager done to improve the accuracy of the estimates?

Mostly, there's still some slippage of some but for the most part we estimate accurately and pad a bit for testing and unexpected side-effects. I've used retrospectives to get better at estimating things and are honest about how well we did with respect to the timelines. Either we missed the timeline because we didn't understand the lift or we shaved the features of delivery to match the timeline.

We monitor our velocity and adjust accordingly.

Dude. Come on. Maybe start with asking your reports how you can help them, rather than jumping straight to quit/fire. And if that's not in your (literal, written) job description, it should be.

I do, I don't let it get to this point. But once it is, there's no recourse but to let them find their ikigai.

> you just want to code without any repercussions

Am I reading this right? You — a manager — just want to be able to assign blame for your own failings to the developers under your command?

And then fire them, like sacrificial lambs?

No, you didn't read it right. I was referring to the OP that justs wants to code without giving estimates or any kind of confinement to their exploratory software engineering. I don't assign blame. I don't have sacrificial lambs. This isn't a meat factory.

If I worked on your team, I wouldn’t any longer.

That’s fair. At-will employment laws…

Would you like to elaborate on why?

Well, if it's a one-track project, estimates can be treated casually.

However, I spent the majority of my career at hardware manufacturers, where software was actually kind of an "annoying extra requirement" (I'd like to think that's changing, these days, but not so sure. Contempt for software seemed to be a fundamental pillar of hardware engineers).

Our projects were always done by fairly massive, interdisciplinary teams, and everything had to "sync up," at the correct time. If the hardware would be ready at a certain date, the host drivers needed to be ready then, because the SDK people and the QC team needed them, etc.

And even though engineers love to rag on marketing, they have to have very complex projects, to generate buzz, advertise, ship, distribute, etc. Hardware is a lot more difficult to distribute than software.

Also, Quality was really important. A recall of hardware is a nightmare. Sometimes, we could be heroes, and save customers from hardware defects (like having an image processing filter to correct some chromatic aberration). Other times, we could brick the unit, or wipe out customer systems (I once wrote a SCSI driver that wrote to sector zero on hard disks -oops).

Estimates -not just software estimates- were critical.

Managers don't actually want estimates. A genuine "estimate" would be a probability distribution over a range of dates, with some point identified as the most likely delivery date and a roughly even chance of being early or late relative to that point. What they actually want is "the earliest date you cannot currently prove to be infeasible", which essentially is the near end of that date range, so there's a roughly 100% chance you'll come in after it. I've had exactly one manager who (after I explained this) admitted that was actually what he wanted, but I couldn't convince him of the utility of asking for genuine estimates.

You cannot prove anything either way so that's also a moot point. If someone can prove you wrong on it they should do the estimate and if they can't then anything works.

We're all in the world of microservices in large orgs that don't talk to each other. Who knows what will be impacted...

Reminds me of bureaucracies deciding that staff should have the “Worst possible equipment they cannot prove actually stops them working.”

Bingo. And not only is it a probability distribution, it's a constantly changing one.

Most of us are carpenters since carpenters don't build literally the exact same thing over and over either. They build similar stuff like decks and houses over and over. But they're all different to some extent. Programming is the same. The exact functionality is the different but they're all accomplished using more or less the same techniques.

I can see of course why clients want deadlines, but I can't help but imagine that deadline-based work is somehow antithetical to who we are as humans. We are supposed to go out and make what we can of the daylight, then retire satisfied that any work towards foraging or the homestead is good work. You don't set out to forage x amount in y days; you forage for a period of time and see what you end up with by the end. There is no deadline or quota to meet for this sort of work, nor can one provide any estimates because things are often well out of your hands, controlled by the forces of nature.

Not all that many jobs grant you the opportunity to work like this in the modern knowledge worker domain, unfortunately, so we are constantly stressing ourselves out with these self imposed "deadlines" that go against the sensibilities of how our species evolved. Of course you end up grey haired and stressed out before long, we aren't built for this. Maybe a job that allows you time to venture into the woods and come back at your own pace so to speak is the kind to have for a low stress life, perhaps one a little better suited to our underlying biology.

To set expectations (to sales people, PMs, managers, etc), i.e. to allow marketing to be aligned to a feature launch, or contacts to push things backwards to accommodate reality

To ensure alignment and a shared understanding of the scope exists, i.e. to prevent doing more than necessary or less than expected

To hold people to account, i.e. some people really are low performers and seem to evade being accountable for their performance

They shouldn't be deadlines, but it's not unreasonable to expect someone to know how they're going to approach something and what amount of effort and time that may take

> To hold people to account, i.e. some people really are low performers and seem to evade being accountable for their performance

Maybe I'm lucky, but in my experience this is quite rare. It is true, however, that if this is the expectation (and you matter, e.g. you're their boss), people will lower their capacity to match.

If you want to see an engineer squirm, ask them to estimate how long something will take. If you want to see a manager / product person squirm, ask them what the net present value of the deliverable is.

We should just accept that significant uncertainty exists in both situations. If you can’t handle risk, liquidate company and invest in t-bills.

I am amazed how my team mates are trying to always severely underestimate tasks. To the point that our manager has to intervene and raise the estimation a bit.

Why are SWE and programmers so afraid of bussineses persons, product owners, scrum masters and higher management?

Whenever I thought my estimation is good, I fought for it with the teeth. It could be the CEO wanting it done by tomorrow, if I said it would take a week, it took a week.

And if an user story is not well refined, is subject to change while it is in implementation phase, is unclear or have some uncertainty, then it's absolutely normal to add time for that.

And I hate estimating in points instead of days. The bussines is always pressuring towards doing more with less people, and as a result velocity rises, we are expected to do more points and people stay after hours to do the work.

> Why are SWE and programmers so afraid of bussineses persons, product owners, scrum masters and higher management?

I've seen this many times and I used to be such a person.

It stems from expecting to be at peak mental shape all day (because that's what you've shown in your job interview) - ultimately overestimating one's capabilities.

I had to wait to my thirties to understand that nope, I can't sustain this and I should acknowledge that I'm not really putting in 8h of focused work daily.

I see this as a rite of passage for a senior developer - you can't be dependable if you're not true to yourself.

That makes sense indeed. It's more of juniors or middle programmers who usually underestimate. And the poor folks end up working nights and weekends to try to meet that estimation.

But it's not that they will have more respect, recognition or salary raise if they work more than they would sign for.

If I were a tech or people manager I wouldn't expect my team to overwork, just do a decent job in a decent amount of time, i.e. no slacking. And our current people manager which used to be a programmer does that. When he feels a task is underestimated he asks to raise the estimation. And the PO and the scrum master comply even if they don't like it.

I have found engineers' estimates to be accurate and straightforward when the team owned all the resources needed to complete their tasks. Whenever it depends on some unreliable factor (often some unavailable expert) estimates are useless. Estimating tasks without estimating each team members capacity is basically lying to yourself.

Software is a vast field and we do different kinds of work. Some work is easier to estimate than others.

Estimating TTC for "make a reputation system for a social network" is probably going to be harder than the same for a typical Oracle app. Mostly because the former involves some science and engineering and not just programming. But that's the point I'm trying to make: SWEs are asked all the time to solve hard problems, not just code from a detailed spec.

At various times I've been asked to design a CPU, write a C compiler, invent technology to extract simple factoids from text, and make a spellcorrector for web search. Incidentally, the approach to schedules was sometimes reasonable and sometimes unreasonable. Smaller companies tended to be less reasonable.

Such tasks are very different from UI heavy work or a typical database application, and are more difficult to estimate. Those persons writing compilers or improving web search indexing are not the same people talking about user stories. If you believe developers need to be professional and schedule their work, you're in the majority but you may not be doing truly interesting work.

[Edited to say "Software" at the beginning instead of "Software Engineering".]

The one reason I have seen it useful for developers to estimate tasks is that it helps spot misunderstandings early on. As in: wait, your estimate is 3 days and mine is seven, are we talking about the same thing?

yes and then we foolishly just take the average or something instead of saying “oops clearly this needs to be broken down further”

I expect a plumber to give me an estimate of time and cost for his "tasks" before starting work. Even if he doesn't tell me the exact cost upfront he'll still be able to tell me roughly if he'll be done in 2-3 hours or less/more.

If I have a big project and I'm dealing with a plumbing company I might not be talking directly to the plumbers but to their manager. The manager will usually ask for time/complexity estimates from his plumbers before sending me a quote. The manager will usually factor into the quote any risks plus the company's profit margin.

There's a lot of talk about agile, scrum, no-estimates and so on, but a big part of the software industry still works with time estimates and budgets communicated up front.

Why are plumbers/carpenters/roofers… expected to estimate their tasks?!

And if you say “oh there is a BIG difference” there really isn’t (other than that overwhelming majority of developers are terrible at their craft, not their fault there is just entirely too many developers)

The best manager I ever had used this process:

- Manager asks for programmer for time estimate

- Manager writes time estimate

- Manager gives time estimate to client.

It might seem easy but it had the consequence that the developer thought carefully the first time. There was zero pressure and things worked fine.

How much will the team have to learn as individuals and as a team? How many components can you enumerate and in what detail? How many tries will each component take?

How many interconnects to "external" systems are there? Risk is time, how much risk is there?

What are the non-functional requirements? These will be dropped when time is short.

How motivated will your team be? What will make them depressed about this project? What will de-motivate them?

=====

Don't ask a developer for estimates in terms of time. It's not useful to them, and it's not informative to management.

Money now is more valuable than money later.

As someone who has both built and sold software, it's simply to be able to sell software before it's complete. Without estimates, you can't sell something that doesn't exist yet.

This doesn't change the fact that requiring estimates is a bad idea if you want great software. The best software is built well, then sold.[0]

Great software later is more valuable long-term than bad software now.

[0] The best case is actually to sell software without a timeline. But most organizations are not able to operate this way.

Still, some level of estimates can always be given. I have never embarked upon a programming task, even a very novel one, where I couldn't say whether it will take two hours or two years. Now, whether it would take two weeks or two months has sometimes happened.

The only time I get irked about giving an estimate is when I don't get time to think about it. I need to be able to think through the request, jot some questions (and jot down what I anticipate the answers to be and how it affects the estimate), make some notes and think about what impact that feature will have.

It's not a good look for a manager to ask for an estimate in a meeting and expect a reply immediately. But, it happens.

> It's not a good look for a manager to ask for an estimate in a meeting and expect a reply immediately. But, it happens.

The only time I ask this question is in meetings where a project is late and we are talking remediation or someone is proposing to do something new. I don’t do it because I expect people to deliver estimate instantaneously. I do it because I expect professionals to come to meeting prepared after having done their job.

> I do it because I expect professionals to come to meeting prepared after having done their job.

Can't do your job if you're not given time to.

I managed developers for a long time, prior to retiring. Generally, except when I worked for a company filled with PMPs, I asked developers when they would complete an assignment if things went well and if they didn't when. I didn't ask for task level estimates. I did ask for descriptive write up of the direction/approach and any intermediate milestones that told them they were on track and how well the project was going. Over time, I learned who was good at this type of estimate and who wasn't, who sandbagged, who was overly aggressive, etc. It took some time to get my bosses happy with the lack of detail, especially since I usually gave the pessimistic estimate or something close. I hated being micro-managed when I was writing code everyday and wasn't going to follow at practice that I myself had hated. This process worked great until I worked for a PMP-driven company, where projects routinely crashed and burned. Slack by Tom DeMarco should be required reading for all PMPs and their managers.

Estimations are useful for developers and management. It helps developers prioritize their time and propose more efficient approaches. It helps management track projects and make decisions more efficiently.

Even an open-ended problem has to be scoped into a time-boxed "spike" first that would result into concrete stories which would be sized in story points. Nobody likes to give estimates because it feels like an unfair commitment. These are the rules I follow myself based on my own learnings and advise from others.

1. If I can't feel sure about an estimate in story points then I prepend a time-boxed "spike" to solve that first.

2. Always consider "extra" tasks like tests, data migrations, etc. in the estimate.

3. If my estimate looks bigger than what I can do in half of a sprint if you do scrum then I break it down to smaller stories so I have less risk to have the story rolling over.

4. Add about 20% of padding to all estimations.

5. If you take notes in the story/tickets about all the things adding to the estimation besides common discussions about the work, then you'll get more and more comfortable with your estimations over time.

I’m a big fan of AMZN’s approach:

- make an operational plan (OP) for the entire year, which sums up all the HC on your team

- figure out your goals for the year

- have some senior engineers estimate those goals in # of HC (usually 0.5 HC is the lowest granularity)

- add up the HC, prioritize the goals and figure out the cut line against your budget for the year

- when the new year rolls around, start execution. Launch dates are usually set by quarter, with the majority launching by Q3

- individual teams have complete leeway to use whatever project estimation techniques they want. It really doesn’t matter at all. Even waterfall is “fine”. All that matters is whether they can deliver the goals they signed up for in their OP.

- if a goal goes off the rails, report a “traffic light status” (red/yellow/green) and path to green, and engage leadership accordingly to reset expectations.

People accuse this process of being too waterfall / unagile, but it’s actually really helpful in centering the deliverables with business value, while giving teams extreme autonomy to achieve those goals. Mature businesses think in terms of quarters and bottom lines, not sprints and story points.

Why are you referring to the company by its stock mnemonic?

In my experience, the estimate is almost never about an actual, literal target anyone should put on a calendar. It is more of a way for management to slowly develop an understanding of how much cost or anxiety is associated with a thing. The more often management samples you for estimates, the more likely they will start to grasp reality, even if you are wildly-off in most cases.

It is traditional for developers to look at this like a binary thing where they're being expected to provide a deterministic outcome at an arbitrary future date. Unless your management is actually incompetent/abusive, they understand how the game works and aren't actually looking for this kind of magic. Once you learn that missing targets is totally fine & acceptable virtually everywhere, life improves dramatically.

Another way to look at this: If your work is so easy to predict that you can set detailed targets 12+ months out and deterministically hit them, how much value are you able to provide to your customers?

It is true that scope management through batch theory delivers consistent results, yet the quality is almost always below what would continue a products profitability, almost always initially falls below user expectations, and almost always includes a tradeoff for engineer sanity in the form of maintainability.

What the PM is not saying, or the quiet part, is that from their perspective "it is your job, do your job." Frankly, that's a piss poor take as well. People are griping about the relationship between engineers and businesses because it's taking a toll. Telling people to shut up and get back to work, worse, alluding their gripes are narcissistic at best is a woefully attrocious take.

Businesses can be better, but part of being better will be taking bets on how to make things better that the engineers (workers) don't solely shoulder or take blame for. Hell, it'd be nice to see the business aspire for internal change where the engineers aren't asked to change at all.

Perhaps a solution is to distinguish between an estimate and a detailed plan.

One is a quick guess with high risk, the other takes time and research to prepare and the risk of running into a delay is lower.

And often the best solution is to have none; perhaps part of the problem is that people are often too afraid or have not enough trust to work without plans or estimates when needed.

Context is everything.

1. How important is it that we get an estimate? Things take the same amount of work with or without the estimate, so how bad is it actually if we don't know how long something will take? And no - a feeling of general anxiety among management/leadership around "not knowing" DOES NOT count as an actual cost! If I can't handle uncertainty, that shows a lack of maturity on my part. If the estimate is not critically important due to factors external to the company, consider using an appetite instead(https://basecamp.com/shapeup/1.2-chapter-03#fixed-time-varia...).

2. If I truly need an estimate, the next question is how difficult is it to provide an accurate estimate? Nobody can estimate how long AGI or flying cars will take. If we need a new field on a form and in the DB - something the team has done many times before, I would require an accurate estimate from the engineers, and hold them accountable if they miss the target. However, if I NEED an estimate for a very difficult to estimate task, it is a RED FLAG that the company is at significant risk. Rather than push the team to provide a more accurate estimate, leadership/management should instead focus on strategic moves or contingencies to mitigate risk. If strategic moves/contingency plans are not realistic, the next preference would be to play it safe: rather than waiting for the estimate, start work immediately and redirect/focus additional resources on the task until the risk has been lowered(get progress updates instead of estimates). If we can't actually start work, we need to have a long and hard think about our core business and work practices because the process is putting massive stress on ICs by making them guess at things which may be outside of their control, leading to decreased performance and increased attrition.

As a developer, I love being able to give you an accurate estimate. It means I know the weight of all of the tasks and likely have done them all before. So creating an estimate is basically a rough outline of the tasks I need churn out. I can go on autopilot and crank out some solid work that improves on previous well-tested experience.

Producing estimates is a necessary evil for multiple business reasons. A fundamental skill of any experience Software Engineer is to be able to make decisions under lots of uncertainty, and that includes allocating resources to a task for which a lot of information is not yet available. Yes it sucks but it needs to be done.

It's been this way for over 60 years. You'd think we'd have formalized an estimation process by now.

Here are some tips I've developed over the years:

- Estimate in days, not hours

- Don't forget testing, especially end-to-end testing

- Don't forget project execution overheads (PM's, BA's, stakeholder meetings, etc.)

- Don't forget deployment overheads, if you have them

Finally, all stakeholders should understand that initial estimates are a Rough Order of Magnitude estimate, i.e. the actual cost will be -50% - 150% of the estimate. The investment decision needs to be made at 150% the cost, for that's what they project may actually cost and if that doesn't make sense to invest then you can nip the project in the bud right now. That's the whole idea behind a ROM estimate.

Death Marches are a symptom of organizational incompetence, and yes, many organizations are incompetent and destined for eventual failure.

As a product manager I often ask for and estimate for a piece of work but it’s not so I can hold the engineer to their estimate. It’s often so I can understand capacity in the team and plan accordingly. I often just need to know; is this a week or two of work or a few days.

Sometimes I might need an estimate if another team is dependent on the work being completed but i understand that most estimates are very rough and that I might likely need to go back to external teams and manage their expectations.

I think the problem that OP highlights is not necessarily a problem with estimations it’s a problem with management holding engineers to account. This sounds like a company with poor culture.

The solution to this is not to give estimates any more, the solution is to fix a toxic culture of management requiring accurate estimates and then holding engineers to account

This is a question i have often asked. Just over 20 years ago I was renting a flat in Edinburgh where just opposite a steel-framed office block was being built. The construction really interested me - i saw several guys and a crane operator swinging up a huge horizontal i-bar and bolting it on to the vertical bars, with no obvious problems. Compare that with adding a new interface to something.

And do you think these guys were asked how long the bolting would take? No, they were told how long they had to do it, by the engineer in charge of the site, and they did it.

This is why all software project managers should be highly competent programmers and should do all task (a bad word, imho) estimation themselves. After all. they are hopefully more experienced!

Here's a casual thought: if you, as a developer, were able to generate code instantaneously, what time would be required to 0) gather initial requirements and create Jira tickets, 1) re-work your code based on changing requirements, 2) re-build and re-package your solution for testing, and 3) have your dedicated QA or acceptance team evaluate your packaged code in a staging and or production capacity?

Because those components are, fundamentally, a significant portion of the time estimate required to make plans from a business perspective–and nearly all of them are largely outside your control. AND-most importantly-your ability to gauge those timelines, without a single Git commit, is what separates highly-paid engineers from n00bs.

I agree that time estimates are a necessary evil, but I disagree that story points are useful for anything really.

This comment from the article was interesting and it matched my experience:

> The law of large numbers only applies if a) the population is sufficiently large, otherwise outliers will dominate b) the estimates are at least mostly independent, otherwise systematic bias will skew the result and c) the estimate is in fact the expected value (mean average) of the probability distribution of time taken. If the estimate is a mode of the probability distribution, then you can not meaningfully add the estimate directly and be able to meaninfully say anything about the resulting distribution.

Reasonably accurate estimates aren't that hard to get if engineers and management REALLY want them. Just remember that estimating a project is a project unto itself. You just need to break things down into small bits that can be accurately estimated and be honest about the bits you don't know enough to estimate and dig down into those bits until you know enough to accurately estimate them.

It's not easy and it takes time. I used to do a lot of fixed price work and you'd live or die based on how well you could do this. Let me tell you, nothing is worse than busting your ass on a project for six months only to lose money on it. Ugh.

Since no one mentioned it, I have a good news for you: it's a sweet spot between what a project manager needs and a team member needs, it's called a PERT estimate. When asked to do an estimate, give your optimistic estimate, realistic estimate and pessimistic estimate. This allows for a better idea of the risk and padding needed for this task (you don't need to add a buffer in any of those numbers) and the manager can use that to calculate a weighted average (very simple formula that you can find online). It's in the PMBoK, so if you have a project manager, he/she should know about it.

“…completely unmoored from the purpose of business, which is to make money by providing a product or service within a given schedule, scope, and cost.”

Or perhaps to solve an important problem in the world, within a given scope and cost!

In all these discussions, what I pick up the most on is the adversarial type of relationship people have with their managers. I don't get it. Everyone is pursuing the same goal; aren't we partners in this? I have only ever worked in big companies (20k+ employees), and its rampant there. Perhaps I have been incredibly lucky, but I have thrived in my career by actually forging a partnership with my managers and leaders. Work acquaintance does not have to be a cold, cutthroat relationship. There is usually enough to go around, and you win more as a team.

I wish I was asked to estimate a time deadline. I'd prefer that I think to the current situation where we pretend to estimate 'complexity', and then say we can only fit X amount of complexity in a two week sprint, or it's not a very complex ticket why's it taking so long.

Not that I want to be held to time-estimates, I just think at least call it what it is. You can't schedule your sprint on time, and the amount of points in it on complexity, you have a dimensional error.

Toyota, where Lean came from. Doing estimates to put together a car sounds like it would be straight-forward. Doing estimates for R & D, not so much.

I think software development falls somewhere in between those 2 extremes. We have some knowable quantities, but also some unknowns. We can't really estimate the unknowns reliably, but we can do something like a "timebox" to see how much we can figure out. Meaning that we need time up front, just so that we can come up with an estimate.

My take is the comoditisation of developers, pushing as much responsibility to them in search of efficiency. Arguably they are best placed to know how long the implementation of X might take, but often the lines are blurred, i.e. developers also become business analysts to spec out the work to determine what the business really needs. Personally I find it exhausting and is one of the aspects that makes me dream of doing something else. Maybe GPT-4 will put me out of my misery soon.

Because Software Engineering is much more than blindly typing code.

Yes, business and domain knowledge is part of the job.

The only developers I don't expect estimations from, are juniors and trainees.

You can reframe estimation tasks to identifying where there's uncertainty, and what the PM should prioritize to ensure the project is successful.

Your estimates are prioritization inputs and tasks for the PM to do, managing said PM.

Gannt charts do this well, and you can apply rhetoric to your arguments by setting some tasks to be much much longer than others. The reader will see those items as the most important and influential for affecting the total time

> It would then be the manager's responsibility to extrapolate my likely completion date, learn my accuracy over time, adjust to it and manage expectations of upper management.

That is something that should be happening for job evaluation anyways. I've also never seen it happen.

Seems to be an unfortunate trend in management in general. My mother worked as a nurse and complained the managers didn't bother to walk the halls to check up on things.

Perhaps by coincidence, there were a couple of front page threads last week about the cost and duration of major government infrastructure projects. But actually it's a recurring theme. People speculate about things like government bureaucracy, NIMBYism, and project complexity, but ultimately nobody knows the answer. And some of the easy answers break down if applied to the management of software projects.

In my current company we don't estimate, and that's the best perk I've had, ever.

It removes so much pressure and anxiety and negative feelings.

Where is accountability introduced? Who makes the judgment that something is taking too long?

We're accountable on an individual level with our direct managers. The team lead, his boss, and product managers make judgements of priority when something takes longer than expected.

The work developers do varies, so this question in general makes little sense.

Sometimes, developers literally do R&D without knowing, whether they will be able to solve the task at hand at all. Other times, they use proven solutions for typical problems.

I saw the most issues arises, when management starts to confuse between the two.

From the business perspective, you definitely have to balance the ratio of the two.

Aside, I utterly despise the recent popularity of the "You Have an X/Y Problem" answers in StackExchange.

Is that a trend on StackExchange? So obnoxious. I stopped paying attention to that site years ago.

The actual answers to OP's question is that management needs to know:

- how much resource to allocate to the task

- variations of the estimates in function of variations of the scope

In my experience, "ballparks" estimates are sufficient for this exercise, and precise estimates are just resource-tracking leading to the issues described by OP.

every business exist to make money or something of value to its shareholder.

You can hire contractor to modernize you kitchen or paint wall.. you need some understanding of exactly what is scope of work, what is $ cost, what is the quality, and any other constraint.

the contractor cannot just say i have no idea how long it will take. there has to be some start and end date.

why some programmer think they do not have to provide any estimate at all is perplexing. is your paycheck an infinite money stream? does your employer not have commitment to deliver something by specific date? or do you want someone else to go create estimate of how long YOU need to complete some work?

this is why tools like scrum come in. estimates not a perfect science..so break down scope of work into small and small chunks until we can say “ok…this piece take 3 days”.

I get the feeling they're asking why can't management estimate for me based on current velocity.

Because the budget is not infinite - there is normally a limit of time or money and it will determine the decision making process.

This doesn't mean the project will finish on time or within budget it just enables that part of the decision making process.

Why should you be expected to estimate how much time it’ll take you to get across time for your doctor’s appointment? Just head over there whenever. I’m sure everything will work out in the end.

I've been on all sides of this issue. Producing code, managing teams, providing estimates, negotiating deadlines, pre-sales, small teams, large orgs, the works.

Deadlines and estimates are all smoke and mirrors, all bullshit. Sometimes that can be fine, but it becomes a problem when the stakes are high (for some definition of "high") and when negotiating power is removed from some group -- and frankly that group is always the one shovelling coal in the engine room.

I'd love to see the coal-shovellers in a position to apply as much pressure to the sales/whatever teams and upper management. I mean, it would surely be a disaster, but an entertaining one. Milgram's prison experiment all over again.

I rarely see projects finish on time and on budget. I don't think I'm unlucky. I wonder who is at fault for this, developers and engineers or project management. Or both.

I feel I could pick out exactly who the developers are, the managers who used to be devs, and the PMs from these comments with a pretty good success rate.

Because estimates show that you’ve actually thought through the task. It’s a forcing function.

Estimates aren’t the issue. People using estimates against you are.

The question is certainly naive, but the answers read like satire of the worst project managers I ever had.

Time roughly estimated to complete 1 task - 2 hours. Time it took - 22h. Sometimes I just hate my life

Cuz proj managers need some fall guy when they estimate a project timeline and submitting it to upper

If only the task that the estimate is for would not change much after the estimate has been given.

'Walking on water and developing software from a specification are easy if both are frozen.' – Edward V. Berard

And remember - the people asking for estimates want the estimates right now. I could give you an amazingly accurate estimate of how long a software project will take - it just might take me longer to produce the estimate than it will to produce the software.

"Tasks" are the lowest-common-denominator directives that developers should refuse to estimate as a matter of self respect. If management wants an estimate, insist that they do their job and write a proper story instead.

Hmm this could be the perfect task for GPT: estimate projects.

sometimes estimates are useful for deciding what not to do..

“oh, that’s gonna take 6 months? we won’t need it by then, let’s do something else.”

Because magical thinking and lack of boundaries.

Because of non-technical micromanagers.

Worked in a position as a nominal 'team lead' for a couple years. Small group, as they got funded and grew, more management and ceremony kept creeping in, and... we had more planning/estimating meetings.

I was asked a few times "when will this be done? We need to know when X will be done".

So... already, I'm bristling because we don't have a shared definition of 'done'. My pushback was... probably seen as not helpful/aggressive/whatever, but I responded with something like:

"I can't tell you specifically, because I don't have control over all the people and processes that get us to 'done', meaning it's tested/reviewed/deployed".

At that point in time, I already had code developed locally with tests, and had pushed up something sitting on an ephemeral dev branch for others to poke/test/verify.

"What do you mean you can't be specific? We need a date".

"OK... Oct 28" (at the time, it was 2 calendar weeks away).

"Well... I can't trust anything you say at this point - I'm going to say Nov 9".

Pissed me off to no end because... why ask if you turn around and say "I do not trust anything you say"?

I tried explaining further:

"We have to have an internal code review, some internal testing, some testing with the client on this feature, some testing of it merged in with everything else, then a deploy. All of these involve other people. I've asked multiple times for the people responsible for these to pick up the task and give me any necessary feedback. 3 of them have told me 'Not now, MrX told me ABC is my highest priority', so... short of you, MrX, telling them to cooperate with me ahead of the other items, I have no ability to move this forward. It's been ready/sitting/waiting for 3 days already, but without some explicit authority from you telling other people to take this task seriously, and prioritize it and cooperate with me, I can not give a date when it will be 'done', because I don't control 'done'. The entire group does, and they're all politely telling me it's their lowest priority, on your order."

I got some reply along the lines of "now you're just being argumentative and pedantic!"

Me: "I could roll it out now - I'm relatively confident in the code, docs and tests I have in place, but our process is steps ABC, then XYZ, and other people who are not me are tasked with that. If I roll it out now, I'm violating the group process/agreement, and this will be seen as aggressive, or passive-aggressive, or 'non-team-player', etc. So... it could be 'done' tomorrow, but not through normal processes."

"No, we have processes in place for a reason, we have to follow those".

Me: "OK... so, if my code is dependent on people who can deprioritize or skip over or ignore any requests I ask of them (many were documented in tickets as 'waiting on personX')... how can I both follow the process, and give you a specific date when something will be 'done'? I can not."

I was so frustrated here because the 'process' was brought in to 'make things better', but it spread everyone too thin. We (group of 4-5) had routinely been missing estimated delivery dates for months - it got worse after new processes, not better (but we put more in jira, so there were nicer charts showing how bad it was getting week to week!).

We just went around in circles for another 5 minutes or so then I left the meeting. It was a constant state of feeling like I'm being talked down to, and "I can't trust anything you say".

I get it - people need some degree of estimates/delivery dates. Extra 'agile' processes along with spreading people too thin, siloing people, then ignoring the day to day impact - none of this was good.

This was a few years ago - I've thought some more about how I might handle that differently if it comes up again, but I keep coming back to "avoid projects/teams/processes like that again" as the easiest path. I'm not sure there's any productive way to get out of the logical 'rock and a hard place' that ended up being (from my POV).

It boiled down to "give us a specific date on something which requires multiple other people who you have no control/authority over, and whom I've also told to ignore you and/or deprioritize your requests". The only options seemed to be

a. give me control over those other people needed to follow the process

b. bring in more people to help follow the process

c. allow me to bypass the current process

FWIW, someone else got wind of this on the client end and we collaboratively did option C a few days later, and it was just rolled out (to the client's satisfaction) well before the end of October.

Lots of good answers which add a lot more value than mine. I'd like to answer it with an xkcd comic - https://xkcd.com/1425/ that hopefully gets the message across as to why we estimate.

Let's take a step back to the real question: is the developer upset because

1. they don't know how to estimate, 2. because their estimate is always wrong, 3. they are being held accountable for an estimate that was incorrect, or 4. they feel that it's not their job to estimate the duration of their work?

Most people here assume that the issue is #4.

However, it's not like they teach estimating skills in school. And the abilities of people vary so much that you'd think estimating would be impossible.

But let's go back to the old apprentice/master model for a minute. How much time would it take a master builder to build, say, a classic roll-top desk with a bunch of little drawers out of oak? I think if you surveyed 10 furniture makers their estimates would be within a few days of each other. Then you'd ask them how long would it take for an apprentice to do the same? And I'm sure their answer would balloon tremendously.

What's the point of that exercise? Experience matters. When you've done lots of things, and you've paid attention, it's easier to estimate how long it takes to complete things - even if you've never done those things in your life. How long does it take to build an OS? With the right team, it should take about 4 years to build version .9. How long does it take to build a telemetry back-end that scales to a few million clients? Maybe 3 weeks to a month. For someone new who's never touched any of the technologies before? Maybe 4-6 months at a minimum, and that's assuming they're good at integrating things together.

So let's get to #1. If you don't know how to estimate, well, you estimate by first trying to figure out the amount of work it takes, then looking at how long it took you to do something of the same complexity/work, then adding some extra time because it's new. You can use your bug tracking system to figure out how long it takes you to fix a bug.

What about #2? Well, if your estimate is always wrong you need to find that delta and exploit it. If it took 2 weeks and you said it would take a week, well, try and figure out why. Were you waiting on other teams? Ran into some problem? Couldn't get resources? Or it was harder than you thought? You ran into some unexpected weirdness? Next time, double your estimate.

There's a PM rule of thumb that says take your developer estimate, double it, then move it to the next time unit. It actually sort of works when you move it up to the next level ie: include testing/QA, documentation, training, deployment.

#3. Are you really going to get fired because you gave a bad time estimate? Then you either need to get better at it or leave. The fact is, anything of any complexity is going to require more people to do, and more people = more time = worse estimates. But it's really up to you to keep people up to date. If you estimate 3 months and you're not even close after a month, well, maybe it's time to tell people and reset expectations. But what the heck are you doing that requires 3 months of work?

the top voted answer lectures OP about ackshually having an X/Y problem, coupled with with a cheeky boomer pull-yourself-up-by-your-bootstraps “you're part of the problem; be part of the solution” to downplay any responsibility on the part of management

this is such hilariously distilled peak stackoverflow that it belongs in the hall of fame

I agree that the top-voted answer is obnoxious, and right off the bat:

"Most of your question is really a rant about how things work at your workplace. Discussions about toxic workplace practices per se are out of scope for PMSE."

Not a friendly, charitable answer, and it is right at the top.

As to the question itself, crowd-source the estimation, is the best approach, in my experience. Get the team together, each member write down an estimate, and reveal all the estimates at once. If there are outliers, have a discussion, otherwise round up and move on to the next estimation. If you're in a scrum shop, you can abstract the estimation by one level and call them "stories" and you're estimating "complication" not "time", which is even better.

There is some reason why so many people upvoted this comment and not yours to the top. Many readers here are highly experienced developers and know that there is always some responsibility on the part of management but this question also strongly tells a lacking responsibility on the developer side.

I agree with you on this being peak stackoverflow. It's a shame how these people install themselves as mods and just run amok. What a shame. The first year or two of stackoverflow was the best times. You could actually have great conversations with professionals.

This didn’t read cheeky to me; how I understood it is: it’s pointless to convince the management to prioritize dev needs in this scenario, either throw the management some bone or find a better workplace.

Reply on news.ycombinator.com