The Last 1%
jaredramsey.com
jaredramsey.com
If a project is successful, it isn't 99% done; essentially it will never be done so long it is alive, supported, and/or used by anyone (I won't get into semantics here).
As others have pointed out, many of those things should be done way earlier as part of the development, AND as part of the ongoing development/maintenance after launch.
On another note I'm also skeptical that a project should ever be 100% done even if that was possible. I had the privilege of working on a project during the whole lifecycle, including when it was deprecated and eventually decommissioned. It was very pleasant to go through all of the open Jira tickets for this product and close them as Won't Fix forever. All of those features, bugfixes and enhancements were never implemented. The project launched, lived, and died without them, and it was fine.
On the other hand, I've seen projects get to 99% and they provide value for a few months and then because they never get to 100%, every time there's an issue no matter how small, it's very difficult to debug and the original owner has moved on to a new project or left the company so users are left holding the bag or they just end up deciding to stop using the product or build a new one.
Software is a living organism, or it is dying. Just as a practical observed matter of how it actually works out.
There are a few exceptions, but few indeed.
You can pretty much play any game pre PS3 era still enjoy it to it's fullest.
Whatever shenanigans were done late in the project, if it passed QA, that code base was effectively read only, with little concern for ongoing maintenance.
On to the next one.
Think BIOS/vacuum cleaner/shaver/toothbrush/alarm clock/dumbphone/non-smart TV/airplane kitchen equipment/medical diagnostics devices/PLC's that control most of the entire world's infrastructure and so on.
It's entirely possible and very common to have an actual "final" software release and be just fine (at least up until recently before the whole "everything needs to have internet for telemetry" hype).
Updateability is the defining advantage of software over pure hardware solutions. If you don't use the advantage then you are stuck with just all the disadvantages.
Some products with software continue to "live on" successfully and thrive, without updates. Think of a digital alarm clock who's goal was to help typical users to wake up on time most of the time. If you ship a product that does that and it isn't being updated, is the product really dying or doomed to failure?
No alarm clock will ever wake everyone up on time, but we can always strive to get closer to that goal if we chose to set that goal. An unreasonable goal could cause unnecessary bike shedding, etc.
But a simple pacemaker for the heart, the goal is closer to the idea of helping as many people as possible, rather than most. Hopefully we write good software and we go 15 years without needing an update. I think that is better than bad software that has to receive more updates. Which software is more "alive" and "thriving". Is the good software with no updates for 15 years really "dying" since it isn't "improving"? Again, thriving and improving shouldn't be conflated.
So, a product setting appropriate goals helps determine how much maintenance is actually necessary and some goals can be met without requiring any future maintenance. Other goals may benefit from frequent maintenance. Some products can thrive without improving. A product's goals determine's the importance of improvements.
The best code I've written is the code that's still running 15+ years later and no one even thinks about.
(The projects I work on typically spend over 50% on automated tests, but I'm in an area of the software industry where we really can't have and bugs escape into the wild. I wish more areas would work like that.)
"Minimum and yeet" works surprisingly well if you have unsolvable debates on customer value, options to yank the feature if it sucks, and generally competent engineers. If you are really good at yanking the feature when it sucks - you may even be able to get away with incompetent engineers. However if you repeat this cycle on the same code base 100x over, every feature gets harder. Eventually you hit a point where no one really knows how to do anything anymore as a feature that used to take 2 weeks now takes 8 months.
"Completion or bust" works really well when you can't take a feature back... ever. However the definition of completion tends to grow over time. I've seen launch checklists which amounted to 6 month projects on their own (for both good and bad reasons). Sometimes completion becomes an excuse for architectural astronauts to enforce a change resistant paradigm on the code, eliminating any gains from completionism. Other times, the engineers and managers become convinced that any change will take N months and start ignoring everything that doesn't look like it will provide N months of value.
In an ideal world, I'd love to see organizations better adapt their standards to the needs of individual teams or invest in tooling which makes it "cheap" to do the right thing. However this is not easy to pull off.
Small add: "generally competent engineers" can sometimes (almost) inline most of the code completeness details for cheap, reducing their cost greatly.
Writing documentation is always a time sink, though, in my experience. Or maybe I'm just not good at it :P It's usually an additional day of work overall, though.
For end-user products with a user interface, less so as it comes down to being flexible... similar with dashboards, as a developer I often don't know what is wanted up front... when something is asked for and you learn the domain more, it can come together easier. Larger projects with a front end component and many developers, nearly have to forget it at the dev level.
And it saves you days and days of re-discovering truths about your code weeks/months/years down the line. Sometimes those rediscovered facts are not even true which will come bite you big time.
Integrate your documentation with your testing. That makes it easier to create and maintain. You don't have comprehensive automatic tests? Then that's your problem right there.
* comments in code
* team-based comments
* project design docs
* knowledge-base articles for handling on-call rotation around feature
* how-to guides for customers
When developing medicine you wouldn't consider safety studies as "extra". When you fly an airplane then the take off checklist is not "extra". Arguing that automated test suites or documentation are just "extra" on top of making software and could be skipped is similar to arguing that you could fly a plane without any checklists or releasing medicine to the public without evaluating its safety. That's just unprofessional nonsense.
What does that look like in practice?
Automated testing FTW. Then you can change and refactor and extend all day, as long as the tests are all green, you are golden.
This is why having a solid BI and Data science org is underrated. To assign these to eng teams is disparaging for every org.
Somewhere around 2010 a sort of product development pseudo-science started to take hold of the industry. Telemetry, A/B testing, surveys... oceans began to be boiled to avoid the uncomfortable fact that good product is developed intuitively by good product people. This "last 1%" is somewhat akin to waterfall development.
Stay lightweight, hire people proven to design and ship awesome product and iterate as fast as possible. Use your own product. Talk to people who use your product. Take chances with your product design. Don't let organizational turf wars shape the product (something telemetry driven development is notorious for). Good old fashioned craftsmanship and creativity can go a long way. It's how new industries are born.
Admittedly there's a big difference between web apps and products that are meant to last or even contain hardware components.
When you are experimenting with consumers and a feature might end up being a failure, then yes some of those items could be cataloged as tech debt of sorts.
You said what I was thinking but couldn’t come up with the words. These aren’t the difference makers.
Every “successful product” is held together with spit, glue, and an ocean of tech debt. There is no utopia.
But curious, can you elaborate on how telemetry driven development leads to organizational turf wars? I think i've seen that too but interested to hear your thoughts.
Does more time on a page mean users are more engaged or are they struggling to find out what's going on? Depends on which product manager can make a better case, probably involving even more telemetry which has to get prioritized.
In the creative IC model, the designer and developer work to solve a problem based on their experience building product. Or someone has a kick ass idea one day and just implements it creating a step change in usage. That type of environment requires freedom and trust, it also puts a lot of control in the hands of the ICs which is why it's not popular with product managers, directors, vps...
This is great for helping to build products, not companies. Companies are larger than just their products; they also have obligations to existing customers, stakeholders represented by auditors, etc. That's not an argument that new products should be anchored down by these other concerns; it's an argument that "creative IC" should only be launched with clear alpha/beta/preview-style labeling, and that there should also be engineers who are not paired with Product, whose job it is to "fill in the rest", so to speak, because it's also important.
Which is why the article says: "This last 1% isn't just what separates a great product from a good product, it's what separates a product that might not eventually fail from one that will eventually fail."
A feature can easily be successful while being impossible to maintain because when it comes time to pay the cost, all the main people involved have already claimed the required benefits and jumped ship.
I'm not sure if you mean you should hardly ever, or almost always, do the "1%" things.
And that attitude is why so much software just sucks. Spending some time to contemplate and test and find out what's good and throw away what's not before it ships, all that goes a long way with quality. "Iterate as fast as possible" is one of those immature ADHD approaches and I'd run if a manager forced my team to do this kind of rushed nonsense.
Why would you build something to throw away before it ships? Hire good people and they won't build un-shippable crap. "As fast as possible" doesn't mean you rush things, it just means you don't waste time with all these peripheral activities.
That's because you learn things along the way. About underspecified requirements, about design choices that looked good on paper but ended up brittle, about tech that promised something but couldn't hold up to it when tested out thoroughly. There are many reasons and in every field you see this effect.
> Hire good people and they won't build un-shippable crap
Ever heard of a prototype? Those exist for a reason.
This, together with the ever-increasing complexity of well, everything, and the increasing number of "things developers should know about X", together with the notion that developers should always work fulltime and learn in their own free time, is non-sustainable.
It's a painful fact that in this "modern" environment, we just can't build anything anymore. There have been three critical vulnerabilities just today (Downfall, TunnelCrack, Inception). If you make a website it's probably hundreds of kilobytes big and you get people whining over accessibility and how it breaks dark mode of version 23.42.23 of their obscure browser.
Have you ever noticed how productive people like Fabrice Bellard just don't care about that stuff? The last procent is just a trap to suck you into the tarpit of spending time on useless shit. Choose a stable target and reasonable feature set, release, and never touch your project again. Bliss.
In the home builders example... that would be lawyers and lawsuits.
Stopping at 90% [https://news.ycombinator.com/item?id=36967594]
I thought this was a direct response to my post.
It's a nice coincidence tho, you and OP should collaborate on a joint research to get the bottom of it.
> This last 1% [is] what separates a product that might not eventually fail from one that will eventually fail.
Then proceeds to list a dozen different types of instrumentation, dashboards, and documentation.
I don't know about anyone else here but I've working on many, many successful software projects that have made their owners many millions of dollars with out-of-date or missing documentation, minimal instrumentation, and minimal or no dashboards to speak of. Most projects I've worked on have achieved at least some level of commercial success and the vast majority missed at least one of those major categories, usually several.
Yes you'll be in a much better place if you have it, but if your competitors and building features and iterating the product while you're building out instrumentation and dashboards, you're likely going to lose over time.
* churn
* turnover and nothing of theirs is documented, everything is tribal knowledge
* their former "star performers" are now trapped working exactly on this single project because they're the only one that knows how it works.
* customer loss because fixes take weeks since you don't even get the metrics to know that their systems are down and their customers have come to believe them to be unreliable.Since we're being pedantic, plenty of software projects that have made their owners many millions of dollars have subsequently failed. Who uses WordPerfect or Lotus 1-2-3?
Considering automated testing as "technical debt" is why there is so much crap out there.
Ideally your documentation is tightly integrated with your tests too, then it will be done as well and won't go stale.
Interestingly neither post seems to be at 100% (not a criticism!), which kinda relates to what I said in the other discussion: https://news.ycombinator.com/item?id=36971378
But actually this blog is apparently talking about a single feature of a larger product. In which case the larger product should already have the necessary infrastructure (metric collection and dashboards), and so perhaps it is just 1% of the time.
For the kind of indie style product I’ve worked/am working on, I think these are more like 20% issues, but I don’t think that you need all - or even most - of the items listed until after your product gets a bit of traction. Better to launch an 80% product now than a 100% product in 6 months.
Until you’ve got many customers, you can get most of the usage, performance and error instrumentation from watching the appropriate logs, perhaps adding a bit of dedicated perf logging in performance critical areas. Building all this other infra too early is just a waste of time.
In fact I’d say that, in the early days, continuous deployment and a bit of testing is more important than instrumentation. It’s way more difficult to retrofit CD, and it saves so much time, especially when you’re pushing lots of updates, i.e. at the start of a project.
But like I say, my comments apply to a new product. Not a new feature of an existing product. In which case I’d expect engineering standards and infrastructure to be well defined.
Good software developers do know the benefit of those things. They resist the management's efforts to ignore those things. In the end, the software developer often give in because they know who has the power.
There's balance to be had, and cases where's it's not worth it to put in more effort on certain things... but the boss rarely has any visibility into the tech side of things.
Internal (maintenance) documentation
External (how-to/FAQ) documentation
Performance metric instrumentation
Easy-to-decipher performance metric dashboard
Usage metric instrumentation
Easy-to-decipher usage metric dashboard
Error metric instrumentation
Easy-to-decipher error metric dashboard
Alerting
Automated testing
None of the above are even important in launching a product, much less an MVP.
Startups have raised hundreds of millions and get millions of users without any of those. In fact they'd probably just slowed them down.
Hence an MVP is a MINIMUM viable product
> Startups have raised hundreds of millions and get millions of users without any of those
A very narrow definition of what makes a good product or long term enduring company. Unless all we should care about is VCs and founder exit sales
Mimimum public features-wise. Not minimum because it doesn't have a performance dashboard, alerting, and automated testing.
>A very narrow definition of what makes a good product or long term enduring company.
None of the things listed are essential for "a good product".
In my recent experience, the biggest point that applies to the vast majority of software projects is his second point: user documentation. As often as not, you get pointed to a website, on which it is impossible to find anything useful. Maybe there's an FAQ, but the questions were never asked by actual users.
If there's a problem, or something isn't properly documented? Maybe there's a support email, or an online ticketing system. Whether you'll get an answer is a lottery. For one piece of software, I submitted a ticket in February asking for clarification of their poorly documented API. No answer, so in March I submitted another ticket requesting an answer to the first ticket. Promptly answered: I just need to be patient. To date (August), nothing.
Development schedules for anything "online" are always crazy: You've got to get something out there, fast. Decent documentation never happens, because the company is already off on the next project.
I tend to use a lot of headerdoc-type of stuff, and the last documentation is often running Jazzy on my codebase, and uploading that to the "docs" directory in GitHub.
Error handling should not (IMNSHO) be put off until the end. It should be designed in from the start. We can add "do-nothing stubs," but they should still be there, for when we need to go back, and hook up the reporting.
Localization is also something that I don't think should be left out until the end. I design my code, so that every string is a placeholder token, and is replaced at build/run time. This also allows for easy integration of marketing "talking points," and a "corporate glossary." These may not be a big deal to the coders, but people who sign checks like them.
Etc.
I have a little screed on how I do documentation, here: https://littlegreenviper.com/miscellany/leaving-a-legacy/
That's the difference between professional software engineering and just hacking something quick in your free time.
You wouldn't ride a motorbike at 150mph on a German autobahn if it was put together by your neighbor with parts from a junkyard in an afternoon. Unclear to me why you'd treat software any different.
Revenue is not the only metric to consider IMHO.
If we had spent an extra four months or so getting this 'last 1%' done, it wouldn't have mattered. It would have just been even more of a waste of time and resources. At least they realized it before we polished the hell out of it.
Actually, I would even go with the 20-30% threshold.
Many great products got traction from the start. Think Twitter, Stripe, Facebook, Craiglist, and etc.
Even the first version of iPhone was far from done.
More practical: have a well defined checklist of must-do's. And a separate list of significant nice-to-do's.
Then wind up a project any time all must-do's are done, with however many nice-to-do's actually got done.
So the last 1% (10%? 25%) are optional nice-to-do's that can be scaled back without concern. The optionality of nice-to-do's pads schedules, making targets dates easier to hit.
seriously tho, that stuff is very important and should get some priority but only after you have customers. if you are not in a position to worry about revenue (maybe you won the lottery or this is a project backed by Google), then do as many of those things beforehand as you can. but your hardest sale is the first one, which will be even harder the longer you wait to put it on the market.
There is no finished. Only more done. Or less done.
If you are not pushing for automated tests from the start on a project that will be released to users, I think you have some professional maturing to do. It can be small and simple but it must exist.
Recently while refactoring I realized I was breaking nearly everything. Most code didn't have tests -- I'm working on an experiment.
So, I coded 3-4 tests of the "run the main program and if it doesn't crash then it passes" style. These tests are silly, but they catch lots of refactoring bugs! And it only took 30 seconds to write the test.
If later I decide the actual _values_ matter, I can add another test, but this "break glass" test is already giving me benefits. I can refactor bravely and see how much stuff goes up in flames. Very useful!