I like the cynicism, but this implies that you can give accurate estimate, which I also take exception to. To give an accurate estimate, you'd have to first have a complete specification, which I've never seen.
https://learn.microsoft.com/en-us/sql/relational-databases/s...
You hit the nail on the head.
difficult to scope some projects without starting the build
The original 3 man dev team (Rails/JS/IOS) did a bit of skunk works, and built the original idea in 2 days. We had a good head of tech at the time so we deployed it and just kept quiet when we estimated new forms.
The problem is that (a) most organizations don't track and record the time to do anything and (b) people don't want to take the time to break down tasks to units similar to something they did before. Estimation is itself a time-consuming task.
This is the sort of thing you do when the unthinkable outcome shifts from idle workers to late delivery: Today, it's unthinkable to management for workers to be idle. If instead it were unthinkable for the project to be late, they would have to accept some risk of idleness to create that outcome.
Of course, you can always start doing prep work to de-risk the project, such as spikes, POCs, et cetera, within the planning phase. This might allow you to pad less but retain the certainty of your outcome.
You might get reprimanded if you give accurate estimates - that doesn't change the fact you mostly can't give accurate estimates even if you wanted to.
Ah, an optimist.
I long for a world where software development estimates and those who expect them are perceived as the unfunny jokes they are. Why do naked emperors make such wretched despots?
That being said, accurate estimates are usually not needed, but the order of magnitude is. Knowing if the change is days, weeks or years of work is important - and while we're bad at estimates, we're rarely "I estimated 2 weeks but it took 6 months"-bad.
To assist you, let me paint a picture to put you in the right frame of mind:
You have never worked with superconductors of any kind and you’re not even a physicist. You’re one of those “scientist types” that are indistinguishable in the eyes of account managers.
You’re in a thirty minute Teams meeting with a disinterested project mismanager that wants not just a finish date (on a specific date), but the milestone dates on the way there.
You haven’t even met the team you’ll be working with. You haven’t yet spoken with the customer. Your “requirements” (lol) is literally just three words.
Your “obstinance” at refusing to be professional and “do your job” is being thrown in your face by the PM and is being recorded for your next review meeting.
Replace “room temperature superconductors” with any one of dozens of IT technologies or tasks and you have a nearly verbatim replay of my career and my challenges with estimation.
Here’s the thing: if you’re doing something for the first time, you can’t estimate it. If you’re doing it a second time, you can’t estimate it either because it’ll benefit from reuse in a way not experienced the first time. If you’re doing things three or more times in IT, you had better automate the process… another unpredictable first-time activity.
You’re either a meat robot doing repetitive work best done by scripts or LLMs - or by definition you are doing things for the first time and can’t estimate accurately.
I don’t do the equivalent of putting down roof tiles in my work.
Do you?
OTOH, it's not the culture where I work currently. We are expensive, we know we're expensive but the reason we can get away with it is because we deliver. That's on several fronts like technical capability but also because when we say "It'll cost X to get you what you want" it usually ends up costing them something close to X.
Part of doing that is that if we don't have enough information to build the thing, we might do a "Phase zero" for a fixed period of time looking at the biggest question marks and that means we're much better informed to quote the main piece of work.
So, it can work both ways really.
While I'm at it there's another thing that boggles my mind. Leaders will say this is the most important project at the company and so much is riding on this project blah blah blah and then they give you a max of 10 minutes to explain all the issues you're facing before they seemingly move on to the next thing. I don't doubt it's importance. I genuinely believe them when they say this is really important but then put your money where your mouth is, roll up your sleeves and actually start driving shit.
It's so far from reality that it's almost satire.
In my experience many good and experienced programmers are able to do that. The reason why you get different observations is that in many cases, if you do such a project, you do it because the journey is the destination, i.e. you do such projects to try out new interesting technologies, i.e. the central gola is not finishing the project, but to get exposure to programming topics that you consider interesting. Finishing such small scale open-source projects to a given maturity is just rather a desired side effect and not a goal.
Ask your stakeholders how accurate each estimate needs to be and how much they want to spend on estimating. Where I'm at now a simple complexity score of 1,3,5,8,13 works fine and gives us +-30% variance and predictive power for new work when translating points into time spent after about three months of work and collecting the velocity.
We spend about five minutes answering how complex is this for each story. To get a 10% variance takes the team a half day for anything non trivial. You have to go through this exercise in both extremes with the stakeholders and then you get buy in for the fast estimating.
Google Mike Griffiths cone of uncertainty for the real take on this.
The problem in most organizations seems to be that no one is willing to absorb the cost of real estimation.
I think thorough planning and estimations usually makes for both better and faster outcomes, but I'm unlikely to push for it simply because a full day of planning (or more) every two weeks is pretty damn boring. Doing some light high-level planning and diving into things is less efficient, but way more fun even if you end up having to backtrack more etc.
OTOH, it's part of a quality process and if your organization is driven by the need to produce high-quality output, accurate size & effort estimations are a part of how you get there.
And you can probably work together to see what can realistically be completed by whatever date they had in mind.
Depends on the workplace I guess. Haven't seen this where I've worked. quite the opposite. Some managers I've worked with taught me to multiple estimates by 2X before writing it on paper.
The reason planning poker exists is to create a sort of prisoner’s dilemma between developers to stop this getting out of hand.
Planning poker creating a sort of prisoner's dilemma is an intriguing thought. Mind explaining a little how it leads to something like prisoner's dilemma. I'd like to grok the connection between the two things.
If you ask developer A how long it will take, the best outcome for them is that they AND EVERYONE ELSE go high. So they should go high.
BUT if everyone else goes low, but they go high, then they look bad. So they're forced to go lower. But then not too low, or they won't be able to deliver in time.
So perhaps this settles on an estimate that's "as low as possible but no lower"?
But it's really not. The best outcome for them is that they are 100% accurate.
Now, I understand that that's generally not feasible, but assuming that you need to pad everything by a huge amount is part of the problem.
As a general rule, I've found that the kind of answer that gets the best response is on the order of "about X time to complete development, Y to get through testing, Z if you need any documentation, release notes or user instructions, I think A, B & C are risk items that could delay completion. If Sally and Jared aren't around for us to ask questions, that will also delay things so we need timely responses from them."
Give good answers and you'll find people take you a lot more seriously.
I've successfully talked stubborn execs out of bad ideas by framing things in terms they understand (risk, costs, reputation). And by doing so earned more respect which I can further leverage in the future.
Planning poker is a sham. Between the abilities of variours sw engineers, their experience level and the knowledge in the area that needs to be touched almost any estimate can be given. In really it's just a way to create peer pressure and extract more value from people.
People are definitely trained via bad managerial incentives to do this and end up doing it even without thinking.
It's also funny how people talk agile but then want full project estimates with milestones down the line because that's how they get to pretend that: - they can give real assurances to the customer - they're valuable PMs with a tech contribution (when it's mostly the M&M customer management and meetings, they're good at)
There's all flavours of situations and people, of course, but it's definitely a common trend in companies, even those that have high renown and are seen as leaders of an industry.
Every project I've ever been on that had wildly wrong estimates was because management didn't actually plan jack shit either due to inexperience, laziness, malice, or all of the above. They then play the blame game to try to keep their job, and man it's so hilarious to witness the sheer carnage as everyone tears them a new asshole nowadays with the rise work from home (group chats/calls, DMs, etc.)
We seriously need to drain the swamp of inept management and nepotism within business ranks.
I've experienced better and it is absolutely possible to be off by less than ~25% of the total time estimated.
i.e. it took 6 days instead of 5, or it took 5 months instead of 4, which is way less depressing.
I agree, but this runs directly counter to Agile methodologies which are all the rage these days.
I don't think so. At most, planners plan for the happy path and then add slack to help them manage expectations. Sometimes stuff emerges and projects adapt. Sometimes your slack covers it, sometimes it doesn't and you get delays and extra costs. That's a Project Manager's dish of the day.
> then lock it in via sunk costs
You're presuming that clients are oblivious to issues of delays and cost overruns, and the negotiation stages don't include accountability specifications.
If it's any consolation, the estimate would have been pointless even if they never changed their minds about anything too.
Did anyone tell them this?
None of these had any access controls, not even for which cells were editable. When I queried about the obviously misaligned pastes and no-content graphs rendering the reports useless, a couple people responded like "Yeah, we know the reports are crap. They want them anyway.".
This was at a company that "was continuously growing at ~3% since the founding" (some decades) and "had never laid off any worker, not part-time or temps".
But somebody had given up. Maybe the guy who came before me, who quit with little notice. A week? I don't remember.
I was given three months for creating and documenting a physical filing system (previous guy left no documentation), and physically filing the backlog. I made it take two weeks, including going back through old Excel workbooks and physical files to repair, correct, and apply access control on the workbooks.
For my next two weeks, I spent a tiny bit of each day keeping things updated, and the majority of it helping out in another department.
Then they suddenly laid off all temps, no appeals from supervisors accepted.
Fast-forward a few years, and I heard from an active employee that they very nearly foundered, before returning to profit, thanks to one of their largest customers taking some of their debt (calculating that their reliability and precision was worth more than the cost of trying to get any other supplier tooled, schooled, and proper rigorous).
I like to think I worked myself out of a job their, by making the estimates not pointless.
I have never actually experienced this personally, but I have no doubt that it's a thing in many companies.
But it's really inept management. If management has some sort of hard date in mind, then the way they should pose the problem to dev is: we need this completed by xx/xx/xx -- what can be accomplished in that time?
I'm not implying you're wrong. Frankly, I wouldn't know enough for or against. And if I had to choose, I'd be inclined to agree.
It just feels bleak.