Hacker News Reader

> Software estimation is basically a useless craft.

It depends on how you're thinking about estimates. Just saying they're useless is arguably missing the benefit of the process.

Estimates are not generally useful in and of themselves (as in, the estimate value isn't particularly useful), but the process around agreeing on the value within a team is. If you're on a team and estimating a task during refinement, and there are wildly different estimates from the different team members, then the definition and/or understanding of the task is wrong and it needs more clarity and discussion. Another thing is that date estimates are problematic, and estimates should more generally be used to estimate complexity, and while they might be linked in some way, they're not the same.

The issue with estimates, in my opinion, comes down to the process after the value has been determined. If it's used as a benchmark or metric in some way, then you get into problematic situations.

I also do disagree with this:

> The reality is, unlike building a building, often the software being built has a ton of unknowable elements that are due to the software being bespoke in many crucial ways.

Honestly, software development is not as complex as people like to believe and there aren't as many unknowable elements as people like to think, and in my opinion this line of thinking is just an excuse to not actually think about what needs to be done to complete a task.

> Honestly, software development is not as complex as people like to believe and there aren't as many unknowable elements as people like to think, and in my opinion this line of thinking is just an excuse to not actually think about what needs to be done to complete a task.

Funny, I've been hearing this constantly over the 24 years of my professional experience as a software developer and I've yet to see it proven true.

Unless we're talking about toy examples like Hello World, developing quality software is devilishly complicated and, in my opinion, downplaying this complexity as an "excuse" is in itself an excuse to compromise quality.

EDIT: Fixed a few words my phone auto-"corrected" for me.

I’d say just discussing how a task will be implemented susses out whether it’s understood. Estimation isn’t a useful part of that discussion. Estimation is almost exclusively for the benefit of management is determining when something will be done.

I would say if this is true in your work:

> Honestly, software development is not as complex as people like to believe and there aren't as many unknowable elements as people like to think, and in my opinion this line of thinking is just an excuse to not actually think about what needs to be done to complete a task.

You’re probably working on fairly rote easily producible stuff. That’s not a knock for sure - buildings are complex projects but are easily producible and rote, and contain some but not many unknowns. I imagine in modern times (I’ve not tried to do it for about 25 years tho, and it definitely was complex then!) making modern websites and e-commerce stuff is fairly rote and predictable and mostly a design process.

Developing something innovative is full of unknowns that can only be determined by trying to do it. Building arbitrary scale SaaS data services, complex cryptographic platforms, more recently AI stuff, all are replete with unknowns. Developing complex business systems as well.

> Building arbitrary scale SaaS data services, complex cryptographic platforms, more recently AI stuff, all are replete with unknowns.

I think we can all agree that the vast majority of software work does not belong to the categories above, and is therefore relatively predictable.

I don’t think I agree, I think most software is written in the context of megacorps, and it’s mostly complex business logic. Further, an awful lot of developers work in government, military, scientific, big tech, and a bunch of very technical software development endeavors. I think some work that can nominally be called software, which I classified as mostly graphic design and related plumbing in my response is relatively predictable. My web design folks claim they’re very productive in scrum, mostly because it’s mostly about pinning down customer requirements and churning out known patterns. This is not something you get a degree in computer science to do, and I don’t really consider it software development. Likewise, setting up SAP integrations or Salesforce integrations isn’t really software development. It’s something like it - but there’s no development of software - just configuration of already written software to customize it to a context.

I’m sure we can disagree here, but I would note we don’t all agree.

Sorry but your entire point sounds like a no true scotsman to me. The work of developing useful software which most people get paid to do is, in fact, relatively predictable. Maybe we are all truly peasants who do inferior work that you don't dare classify as software, maybe what you do is truly groundbreaking (I already accepted that as a valid exception), or maybe you have been working with people who just suck at estimates.

I’m not saying design based work isn’t real work or has no value. But I don’t consider it software development in the way I don’t consider magazine layout software development, even though it’s complex and difficult work. But it’s fairly rote and mechanical, and doesn’t require any software development.

For instance I consider developing react to be software development. I consider making a chat system in react software development. I do not consider making a brand layout using a web framework to be software development.

As an example, generally in business you classify software development as R&D for accounting purposes. It’s speculative, has uncertain payout, uncertain development times. Website creation on the other hand can’t be treated as R&D. It’s capitalized under a different amortization schedule.

I’m sorry if it sounds like I’m making some qualitative distinction in terms of value or whatever. That isn’t my point. My point is when we discuss software development and software engineering they are about development of software. I think most people can distinguish between front end web development, and say, developing a web development framework as distinctly different pursuits with very different characters in terms of skill sets, riskiness, predictability. You can adopt an expansive view, but you took issue with my statement without taking into consideration my measure.

In my experience pinning down user requirements (and then getting feedback on your understanding of their needs, and iterating on that) is one of the most important parts of software development, not just in "web design". It's also the hardest part of "complex business logic", and the main subject of books like Domain Driven Development.

And it's the hardest part to estimate. To make estimates you must first get the requirements clear, but that's a major part of what you were trying to estimate.

I agree, and that needs to be done in both cases. A complex business logic system has an awful lot of unknowns because it’s never been built before. While systems that talk to similar problems might have, businesses are highly complex in themselves and even the users usually can’t articulate how things actually should work or why. They usually can only explain what they like or dislike about the small part they do in their job. That’s much harder than a brand managed layout rendering.

Edit: the hardest business logic is actually brownfield development - where the users who knew why don’t exist any more, and the developers who figured it out have moved on. I think these are the hardest to predict efforts with the least management margin of error for prediction mistakes. It also, IMO distinguishes between a good engineer and a great one. A good engineer can write code. A great one can improve code that been improved by hundreds of different people over decades.

> I’d say just discussing how a task will be implemented susses out whether it’s understood. Estimation isn’t a useful part of that discussion.

I see your point, but I would argue that estimates are more valuable than just discussing the task because they give you a shared language to quickly define how complex you think a task is, and can also help avoid getting too into the weeds about specific implementation details when talking about the task in general. There are definitely problems that can arise from estimation, but if you're able to avoid them, it can be a useful tool for a high functioning team.

I’d counter further than this sort of estimation naturally arises in any discussion of a problems solution. The challenge arises from the recording, accounting, and aggregation of estimations for any purpose other than the development of the solution - which is important for the purpose of allocation of talent and sequencing of immediate dependencies. The thing is the error bars in these estimates are large and positively skewed so you need to make your estimates about the smallest possible piece of work, and attach as little importance and dependency on their accuracy as possible.

The problem comes when you string estimates together and compound the mistake by assuming a precise estimate is an accurate one. I generally refuse to give precise estimates for that reason.

So many +1s for you.

I want to add that if the team is focusing on the present, it becomes clear that estimation is a passive, descriptive (rather than active, proscriptive) act. No ticket has time pressure on it. The manager predicts the future timeline based on how long similar tasks took in the past, on average. No idea if one task will take 5 or 15 days, but 10 of them will take about 100 days.

Once you understand that, there are many very simple ways for a competent manager to get development velocity out without adding work for the developers... and all management actually wants is average velocity and remaining work in similar units.

If your team tries to keep tickets about the same size, ticket throughput is your velocity. At most the devs can slap gut instinct Easy/Medium/Hard labels on the tickets so they don't have to "right size" everything. Either of those (or a thousand variants), when coupled with the pile of work left to do (in the present understanding) can give a competent manager high precision long term estimates with confidence intervals. No need to make it more complicated (or time consuming!) unless the devs tell you they NEED more.

In my experience, teams that focus on the present, and which are in control of their own process (ie adding or changing only when DEVS feel the need) typically end up evolving their own agile process... which is way more effective than OOTB processes like Scrum, and doesn't feel like a burden (the team drops anything that feels like a burden). Teams who have a process imposed upon them, on the other hand, spend years ranting online about how terrible agile is, and waste a lot of time in unnecessary bullshit.

> Teams who have a process imposed upon them

In my experience, this is the vast majority of cases. Hence:

> spend years ranting online about how terrible agile is

Sad but true. And to make matters worse, those seem to be the same environments where businesspeople make target dates without engineering, and then get angry when the deadlines aren't met.

Put another way, poor management is a human layer problem. No technology (whether process or tool) can solve that for you.

I agree with your opinion. The problem is that managers have the notion of software estimation as a "craft". It should not be considered a craft at all, and once you held estimation to the importance of being a "craft", it is going to be a massive burden. It should be just a numerical representation of your gut feeling and past team experience.

Software engineering always has to account for uncertainty. You may try to deal with it before starting the project (waterfall) or in the process (agile). Management seems to think that by pressuring the team they can just do away with uncertainty and make their strategic planning seems more concrete, and make questionable decisions like publishing untested code. It's a power struggle all the way from the top brass to us lowly devs, but management need to understand that this is always going to bite them in the ass on the long run.

Right. The problem is, as usual, us. We imperfect people being who we are, and doing what we do.

And part of who we are is that we blame Jira, because it's so much easier than to fix ourselves.