Hacker News Reader

Comment by jakubp | Hacker News Reader

If someone can conclusively teach inexperienced programmers good approach to estimates (methodology) + help embed this into sales process of a software house-type company, I know some folks who'd love to have this :)

My own experience has been this: people make estimates, client has expectations based on some variant of those, and something later happens but so much change is introduced during the actual software development, that there seems to be no sane way to compare what happened with original estimates. (new features, changed features, lots of new information, market shifts/product vision shifts, quality assumptions change, etc. etc.)

But at that point nobody cares! People go on with further projects, and no data is ever collected.

Nobody learns.

When f*ckups happen, i.e. a gross over- or under-estimate, this is often not shared with the broader organization (ashamed/afraid sales people/PMs/devs say nothing/hide it/sugarcoat it). Client walks away.

Sometimes project is severly underestimated but usually not just because of the software part. Again, no decoupling and estimation of contributing factors is done.

It's insane.

View on HN

Apple executes massive feature releases on a yearly waterfall-like schedule across hundreds of teams. It requires honesty, transparency, strong leadership (ruthless prioritization), strong cross-team goodwill, strong cross-team collaboration as well as a solid body of experienced top-tier engineers. Other than that there's no magic. While there are teams within apple that does scrum/agile, none of the core OS/frameworks teams does it AFAIK, and I do think it's incompatible with shipping on a hardware-aligned schedule.

Adobe executes (executed) massive releases on a well-defined cadence (1.5 yr per "creative suite"). Sure, I won't say they don't have collaboration, or top-tier engineers etc. But in the end, the success of this well defined cadence boiled down to the simple strategy of "we release what is ready". Everybody knew the deadlines well in advance, and everybody strives to 'make it' - but, (not so) occasionally, some teams didn't and you could see entire features get ruthlessly cut out. Simply no other way of synchronizing across the entire company... you cut scope to make it in time, and if you can't make it, that's it, your feature gets cut. With enough teams working, you still get features to showcase at each release.

That's pretty much how Linux kernel releases work. Just having a regularly scheduled release train that always runs on time – that you can catch (or not) is great.

> "we release what is ready"

this really is key, if you can do it

alot of companies though, are "business driven" which means releases and schedules are set beforehand, which makes this hard... (but if you can do it, its great!)

What happens when you can release whenever. This question is probably quite relevant to most developers, because CD (continuous delivery) is popular.

How do you know they are thoroughly waterfall? How do you know they don’t use agile, time-boxing, etc.? Apple is so secretive, it feels like we know of little other than what they are confident about releasing. The things that weren’t done on a schedule compatible with a hardware release, how often are they simply postponed? And for those postponements that they do admit, how many are waved away with a creative explanation? I think you need a good deal more information to back up your point about Apple’s work management.

I worked there 6 years. It is 100% Waterfall. Things that are too buggy by July 1st are delayed by 12 months.

> Apple executes massive feature releases on a yearly waterfall-like schedule across hundreds of teams. It requires [...]

And no doubt a great deal of crunch time, their fair share of delays, some major messes and the occasional outright cancellation. And if I'm not wrong their roadmap is not public information.

I personally find it hard to believe that a large company can exhibit all those adjectives you list. Everyone finds this hard.

When you have hundreds of teams you can cut a bunch of features and it still looks like you are massively productive each release :) Not everyone has that luxury unfortunately. Some of us are a single team of a handful of developers.

Have you noticed their declining OS release quality in recent years? I have, and I now wait 6 months to upgrade.

The triangle is real[1]. Something's gotta give when a deadline can't move, and as we know from Brooks, software development speed doesn't increase linearly with more people. So it's primarily quality or scope that must suffer to hit a software deadline.

[1] - https://en.wikipedia.org/wiki/Project_management_triangle

People have been saying that since OS X first came out, and OS 9 was so buggy it had a famous cute bomb icon for when it crashed.

I'm using MacOs/OsX since Lion, and I can say that it was very stable (even on my Hackintosh) and I observed stability issues increasing with all the following versions (on real Apple HW) and I'm on the same ship as GP never upgrading to newer versions until at least 6 months pass.

Yeah, first few OS X releases were famously buggy as they were figuring this UNIX stuff and the OS 9 compatibility, but they stabilized about 10.4 Tiger.

Honestly the OS X itself (uhhh macOS now) is still fine, even with Big Sur and its weird iOS controls, however the other software is sometimes weird; but - let's say Numbers was never good and iWeb was always downright bad. And iTunes was bad in Jobs era and is still bad (as Music).

I however still kind of miss when native Cocoa apps were sort of good and not just ported iOS app via Catalyst or something. But whatever. I'm old I guess

Don't the bomb icon and "Sad Mac" go back to the very beginning?

When OS X came out 20 years ago, it was the star of the show. Nowadays it's just an annoyance that Apple has to keep around. The quality has demonstrably fallen, and the internal struggles it's had within Apple have already been documented.

Honestly, Apple's decline traces back to losing Steve Jobs. The strategic changes were immediate, (and aimed at shorter term gains, things that people besides Jobs thought were being left on the table) the effects just took a long time to really add up to something meaningful. IMO, they never really recovered, they just are massive and have a HELL of a lot of momentum. It takes time to undermine that all even if you do everything wrong. They definitely are still doing some things right, but IMO, the type of thing you are complaining about is a leadership issue.

That doesn't imply anything about estimates.

Do they tell you at the beginning of that cycle what features to expect? Or do they constantly work on features, and then at presentation time do a freeze?

How do you know it's waterfall-like? If you ship working software every 2 weeks and every 26th one you release, hooray, you have yearly software that's super likely to work well.

Apple produces a new build of iOS or MacOS every 24 hours. For the many years that I had to suffer through that, about 40% of those builds were dogshit and unusable. The public gets the 360th release, and hopefully by then the PMs have driven enough burnout in the engineers that it is mostly acceptable to users.

Makes sense!

The macbook range had years of delays

I think it's because inexperienced programmers don't think about the "soft" costs of programming.

Requirements gathering, general communication, testing, production rollout, coordination, research, final approval - it all adds up. For a 2 week project you might only actually spend a day of ass-in-seat coding. And that ass-in-seat coding is what they tend to estimate.

I think this is a big factor in estimation inaccuracy for really new programmers - but it is pretty well fixed by project managers following estimation rules of thumb - like double it and add five days... The bigger issue that I am less certain about being teachable is the experience side of things. I've worked in industry for a while and I know how long different sorts of patterns will take to write and what sorts of complications different sorts of add-ons will add to those numbers. Some projects are about where my naive brain fifteen years ago would put them and other ones are wildly different due to complexities that aren't visible in a surface level examination. Learning to sniff out those complexities seems to just take experience (not with every individual scenario - but it's a pattern analysis thing).

Programmers can also try to come up with reasonable estimates, only for management (product, middle or upper - or often all of them) to come back and assert that the estimates are off base and to "reconsider." Not much to do at that point but to give a lower estimate and hope it works out. If you make that your hill to die on as a programmer you will find yourself out of a job with only your integrity to keep you warm at night.

I do a fair bit of writing at work of various different types of things.

For stuff that I'm doing on my own? I can estimate pretty well. I know more or less how long the writing will take based on length. I'll allocate more time for research if I need to do more than a base amount. And then I add some slack in case I'm struggling with something, priority interrupts happen, or I'm just not into it on some given day. (Underpromise. Overdeliver.) It only gets harder with bigger projects. It helps that I'm quite fast compared to most people so even with significantly underpromising people are usually very happy with the estimate.

But where it's really hard is work that involves getting buy-in and input from a lot of different people. Those always take forever. (And at this point I pretty much know they'll take forever, a lot of which is just latency rather than actual active work.)

> I think it's because inexperienced programmers don't think about the "soft" costs of programming.

That's what the research has previously found[0] decades ago -- engineers will generally correctly estimate the part they are best at, but will either neglect or under-estimate other work. The old-school ways of dealing with this are canned Work Breakdown Structures and checklists.

This is not unique to software.

[0] van Genuchten, "Why is Software Late? An Empirical Study of Reasons For Delay in Software Development" https://web.archive.org/web/20160914040151/https://lab.cs.ru...

> ...nobody cares! > Nobody learns. > It's insane.

Estimation is really hard, especially if you're dealing with new challenges, new people, and new expectations. Throwing moving targets (unplanned changes) into the mix, as you noticed, makes estimation even more difficult.

The thing is, it very rarely is the case that a wrong estimate will sink a project. Things get delivered late all the time. No one dies. Most stakeholders aren't like two year olds throwing a tantrum because an unforeseen problem delays a deliverable by a few days (admittedly some are close).

The actual definition of failure also needs to be considered. Is it considered a failure because a project manager ended a colored gantt bar at a particular date and the actual time was longer? Or is it a failure because the customer cancelled the contract because a deliverable was not available on the agreed-upon date? Or is it something far more nebulous, like a slow death march towards a misguided goal that burns everyone out and compels them to sip artisanal coffee and daydream about leaving instead of grinding onwards and hitting targets?

Ultimately, people "don't care" because accurate estimates are not the most critical problem on their mind. Most of the time an accurate answer to "When will it [the deliverable] be done?" has about as much gravitas as guessing the number of jellybeans in a jar at a kid's birthday party.

Doing a retrospective and analyzing what went wrong when a project delivery slipped expectations is certainly reasonable but it so easily slips into a blame storm and/or people making commitments that aren't realistic. Getting straight answers from folks for this kind of analysis requires a level of psychological safety that just isn't present in most workplaces.

Failure is a tricky thing.

There is a story that Microsoft Word was a staggering failure in terms of actual time and cost to implement versus its estimates.

It then went on to be among the most lucrative ($ billions) pieces of software ever created.

Was Microsoft Word a failure?

Internal estimates are a political tool, not a technical, financial or scientific one: First, decide if you want to do it or not. Then do the estimate, high if you don't want to do it, low if you do want to do it.

Any kind of reality doesn't factor into it.

In the Word example, somebody really thought this project was worthwhile for some reason, so they gave a very low estimate to get things started. As soon as money has been spent on a project, it isn't likely to be cancelled as easily, so you are safe to continue even if you exceed your original estimate. In the worst case, you need a few excuses.

External estimates, i.e. estimates you use for writing up a contract with an external customer, are different. Those actually do matter, depending on the wording of the contract. But a smart contractor will try to get the contract into a shape such that the above for the internal estimate does apply.

The problem is knowing how fast they learn. For a n00b this is basically impossible, and takes cost monitoring to notice their rate of learning. For a person with the basics of the language that they'll be using, and a tiny bit of experience in the libraries they'll be using, things get easier.

For everyone (n00b to senior) I would always recommend designing & breaking things down into no smaller 1/2 day blocks, where nothing went longer than 3 days (for inexperienced), but longer for more experienced. Senior devs should be able to be able to be trusted (to be accurate) for week-long blocks.

Use a Gannt chart (ms project or similar) and monitor the inexperienced, gauging their real-life times with their estimates, adjusting accordingly. Give semi regular feedback at the accuracy so that they (& you, the manager) are working with.

In one of my companies, the most inaccurate we were for any project was that we were late 2 days over 2 years. Build on trust, build on meetings and other time sinks. Include testing, sign offs, etc.

So, absolute n00bs need close mentoring so that the (senior) mentor can begin to gauge the growth rate... Which will be steep... But different for everyone.

It's very useful to break tasks down so instead of a month long "upgrade elasticsearch" story, there's dozens of day-or-less tasks to work on. Often it's not noticed how much effort goes in to creating these new smaller tasks. If you aren't extremely familiar with the targeted version of elasticsearch, you'll have to spend significant amounts of time investigating, planning, and writing.

You can get very high accuracy on estimates by spending lots of time creating smaller stories and encouraging people to over estimate. This may not be in the best interest of the project or the business.

That means the business is taking safe bets. Depending on the industry this makes a lot of sense. If you are in a small business this probably won't help you.

More important than estimating is knowing how to sequence the work. I often see teams try to build the entire DB schema first, or built the interface out one wire at a time. The agile approach of building small, vertical parts of each feature tends to work much better in my experience.

For example say you have a CRUD application with a page for editing and searching complex objects. Rather than building the entire search page with all of its filters, and then building the edit page with all of its fields, you build a simplified version of the search page with only a few filters and a simplified version of the edit page with fields that correspond to those filters. You also make sure the functionality integrates with the backend to form a cohesive, vertical feature that can go through QA, and possibly the client. Once that is working you expand the list of fields and filters.

This approach tends to surface design issues much faster, and is more adaptable to change requests from the client.

Although when working like this there can be a "Good enough" inertia

agree with the adaptability of change requests from clients, but i would say only insofar as the design issues you're mentioning are actually addressed as quickly as they are surfaced. Otherwise this type of building eventually grinds to a halt.

another commenter sort of hinted at this, but this type of process has a downside of prioritizing short term design decisions that lead to working code. It takes time and sometimes a lot of thought to think about the system and the problems deeply and ask yourself if you've created the right primitives.

Make a complete list of things to develop in a spreadsheet, "complete" means every single item the customer wants to see in the product, not only every single button but also every single label, that should be definable by reading the project specs or mockups. I think "forgetting things" is the first big mistake leading to under estimations.

Add an estimate that you multiply by three in front of each, ie. if the dev thinks 1 hour then they should put 3. From my experiments: no multiplying factor turned to nightly and weekend work, multiplying by 2 turned to long (10 hours) work days, multiplying by 3 turned into comfortable work hours and quality work.

I've had success delivering quality code and respecting deadlines since 2012 or so with this system, but YMMV

Only problem: the customer might not like what he gets, even if it works exactly as planned.

> "complete" means every single item the customer wants to see in the product,

Ah, but the customer does not know everything they want, due to the fractal nature of reality. The closer you get to the end product, the more detail is resolved, and more work is added.

Oh man, thank you for the fractal nature of reality. That is the perfect term to describe what I saw about ever-changing requirements. No matter how closely you define something, things will always keep changing on all levels when closer to actually "delivering" / launching. Adding time, costs, confusion, frustration. Fractal, yes, that's it :)

> No matter how closely you define something, things will always keep changing on all levels when closer to actually "delivering" / launching

Xeno's paradox [0] doesn't apply here. Some projects do actually finish.

[0] https://en.wikipedia.org/wiki/Zeno%27s_paradoxes#Achilles_an...

Although this sounds flippant, the design process is where unknowns should be uncovered. It is easier to change design before a line of code is written. Design is precisely what is needed in order for the dev team to break down the projects into tiny chunks where nothing is unknown.

Your time estimates should always include time for research and design. The customer can sign off on what you're about to write at the end of that (whereas beforehand you give them a wild ballpark figure made by senior devs - not management!)

Yeah but too many customers change their mind even after they've signed off. Then you're left with strictly enforcing a contract or making adaptations.

You're right, design is very very necessary and should be given plenty of time. But it's not a magic bullet.

In terms of contract, I have found this works best (recommended by my lawyer (a former ASF lawyer)): 30% upfront, 50% at first delivery, then they have 2 months to find bugs, final delivery is when the bugs are fixed that's when you take the last 20%. This way, you're certain to always get 80%.

If they want the liberty to "change their mind on the fly" then they should buy man-days which is a completely other thing. Or, they can buy supplementary mandays for whatever they want at the daily rate without changing the contract, or, they can just wait to finish this contract and start another one.

Another way of looking at this is that (unless there is a complete disaster) as you get closer to the end the uncertainty in the estimate decreases. When you get to the end, you know precisely how long it took.

it's pretty a well known saying that the last 10% of any project takes 90% of the time.

I forgot to add this to my other comment. Yes, a multiplier works wonders. (I typically added 30% to my own estimates, which covers all the little incidentals that trip you up, and coffee breaks). I've not been forced to work longer than an 8hr day since the 90's.

I thought your comment stating that time should be added for both research & design was actually a reasonable explanation for why I need the *3 multiplier!

I've done backlogs before; both very high-level and very detailed; with and without mockups (even final graphic designs) and detailed description in the form of multi-page spec or user stories (depending on the fashion of the decade I was doing it in :).

I don't think having the detailed requirements made estimates much more accurate. However it made creating those estimates take an order of magnitude longer - developers needed much much more time to absorb all the details.

At that point I decided I only ever want to work with senior developers who have good imagination and can give reasonable triplets (solution, # of devs to do it, approximate time) for requested pair (idea, quality level).

Back when I was a product manager, there was an engineering manager who would drive me crazy. He had this idea of a 90% schedule, by which he meant a 90% chance of hitting if everything went as planned. Everything never went as planned.

He probably read Goldratt's follow-on to The Goal, Critical Chain. Critical Chain Project Management is the specific term. Goldratt tried to sell software to companies that would help them make schedules like that, but a large component (often ignored) was collecting historic data, and running models. Most companies might do one of those, but rarely both, and of course models with only speculative numbers are absolute garbage. In my experience, places that tried to apply CCPM never actually grokked it, and it was some manager's obsession because they were buzzword crazed (like many big-A Agile offices).

I think this was a little before that (mid-90s) but maybe something related.

ReRe's Law of Repetition and Redundancy [4] somehow does not apply to you:

  A programmer can accurately estimate the schedule for only the repeated and the redundant. Yet,
  A programmer's job is to automate the repeated and the redundant. Thus,
  A programmer delivering to an estimated or predictable schedule is...
  Not doing their job (or is redundant).

[4] https://news.ycombinator.com/item?id=25797519

I think the explanation was made in another comment where they say "don't forget to estimate time for research & design", which explains why the *3 multiplier works for me. And yes, I feel that most of the work is redundant, there's always like network services of some sort or web pages, databases, linux servers, a humain interface, other services depending on the project like a message queue or a caching server or both, background workers and scheduled code execution... things I had been doing for ~10 years back in 2012. Data structures, workflows, they change between gigs, and each gig has its own particularity, but 90% is the redundant from a gig to another: HTML/CSS/JS/Python/SQL/Networking/Storage/Linux stuff

Software development is the process of writing a complete specification, including every single button and label. If you have a formalised complete design, you could write a compiler to do the work for you.

If someone can teach _any_ programmer how to estimate accurately they will literally print money. Its inherently a hard problem (I'd argue a siren song). To truly succeed you need to perform a massive stakeholder education on software development, hidden costs, and the highly dynamic nature of the beast.

Then you need to train the IT professionals to build the dynamic nature into the conversation and process. You dont quote a number, you give a range. You communicate confidence levels and uncertainty clearly and continuously. You update assumptions as new information comes available. You _push back_ when the uncertainty is frowned on.

It can be done successfully on a micro scale sure, by working on it within an organization and/or team. But its not scalable and its not a 5-step plan. Its hard.

As long as humans are involved in defining requirements, inaccuracy of estimate will be proportional to the cusomters' incompetence and level of bad politics of the organisation as a whole ;-)

Estimates don't give you the time when something will be done. You can just be almost sure that the thing will not be ready before this time. And that's still something.

The principles behind the Manifesto for Agile Software Development mention only retrospectives, not estimates. You could say they value learning over guessing.

When people ask me how can they learn, I tell them to find old project schedules and other project documents. Looking at past projects makes the learning more objective, and it is not happening under time pressure.

Reply on news.ycombinator.com