Towards an understanding of technical debt (2016)
kellanem.com
kellanem.com
But that's not what we're doing when we create technical "debt".
What we call technical debt is really best analogous to an uncovered short sale. We sold something we didn't own (i.e. technical sophistication and proper design) to get something of value (a shipping product) in the hopes we'd never have to cover the asset (fix the bug, upgrade the dependency, etc. - i.e. cash out before the code hits the fan).
If you win the bet and you never have to cover the position - it's progress for nothing! Awesome! But, if you do have to cover it - disaster! You're forced to pay any price to cover the short or abandon the feature - maybe the whole project. This can happen at any time. Insanity.
In finance we generally don't permit uncovered short positions for a reason - they destabilise markets, and quickly. Transitive dependencies, i.e. systemic risks, abound.
Companies have to start talking about software projects and practices in an actuarial sense - and get insurance to cover their naked shorts and wild technical bets.
Also, most debt ( especially fractional fiat debt ) is an uncovered short position. When you are lending something you don't have or own.
Debt instruments and fractional lending are not an uncovered short position in that one generally is insured in commoditised bond markets against losses due to the non-systemic default of one or a few parties. The money exists, it just doesn't now, but absolutely will exist in the future. That's the difference between debt and naked shorting.
It's illegal for cash instruments - actual shares and government bonds - but not for futures and options.
- it has interest: you're paying extra in maintenance, debugging and manual testing for all the shortcuts you've taken
- it can be paid back by refactoring the bad module. The price of such a refactoring is usually known and, unlike in short sale, won't get to infinity
- it can be used to grow. Instead of working five years on a perfect solution we build something plausible right now and then keep refining it
Take, for example, something like Mozilla's Facebook Container extension [1]. It's fine if they take some shortcuts that allows them to launch that faster if it turns out there's another Facebook scandal in the news - launching faster allows riding the waves of publicity, resulting in more users. Even though the total development effort is more when they have to clean up those shortcuts after launch, the additional effort has been used to grow.
[1] https://addons.mozilla.org/en-GB/firefox/addon/facebook-cont...
I see examples of the latter all the time in the experience of developers and how they approach decisions. I have recently worked on largish (or at least in the 10s of millions of dollars of investment to create) codebases in which developers (recently - like 2013) wrote switch statements that became God classes because they simply were not familiar (at least not in practice) with patterns that avoided it. Consider that one example and imagine the type of havoc that lack of experience creates across the entire codebase. You can say this is a case of ignorance, which it is, but it's the same as the ignorance of some to handle money in a sustainable way. It's like giving a credit card to a kid.
As I see it, tech debt isn't enough of a descriptor of the kind of engineering shortfall one is experiencing. To understand precisely what one's talking about one needs to characterise the problem more.
My main problem is that the blanket term for the issue uses a best case analogy, when the worst case is so much worse; it's not funny.
Steve Freeman famously described it as an unhedged call option:
https://jaxenter.com/bad-code-isnt-technical-debt-103059.htm...
The key difference between a short position and a call option is that the former is linear, whereas the latter is not: if things let a little bit worse, a short will lose a little bit of money, whereas a call will cost nothing, but if things get a lot worse, a short will lose a lot of money, and a call may lose an enormous amount of money.
Someone who actually works at some dodgy trading company argued that some kinds of debt are like options, and some are like debt - and so potentially, some are like shorts:
http://blog.jayfields.com/2011/03/types-of-technical-debt.ht...
So now we just need to calculate our technical debt portfolio's delta, beta, vega, theta, rho ...
If left unchecked, it will compound and get worse.
Adding 30 floors on top of this building with no time to reinforce the foundations first? Sure, I'll call our team of developers, they do that all the time ;)
We can use the same terms we would use to define any standard debt you owe to a bank. Let's use "no documentation" as an example. The principal is the amount borrowed, or the time spent writing feature X instead of writing documentation. At some point you'll have to write that documentation, which would be your principal payment. And finally, the interest you're paying on that debt is the time developers have to spend figuring out something that could have been more readily available in documentation.
> In a sense, debt is a good thing - you can timeshift revenue. Businesses are comfortable with debt. > But that's not what we're doing when we create technical "debt".
I disagree. In the example above, there is plenty to be gained by establishing that debt. That new feature (or however that time was spent) might bring on more customers or generate more revenue, or influence more investment in the product, or whatever.
My only wish is that technical debt were more easily measured; the net interest is highly unpredictable, and the overall monetary gain from establishing that debt is also hard to measure, which is why I might personally hesitate to call it a debt.
Hopefully that made sense.
Edit: Sorry I just realized you got like 20 similar responses. I had HN open for a few hours and when I sat back down I decided to respond.
> When we program we’re writing down a very explicit and exact description of a solution to our current best understanding of the problem we’re trying to solve. Over time both the problem we are trying to solve and our understanding of it will inevitably change, often dramatically. When that happens some percentage of our very explicit and exact description is now wrong, and a liability. Taken to the logical conclusion, given time, every line of code is wrong and a liability that will need to be dealt with.
> A little discussed and poorly understood design goal for code is disposability.
Criticisms:
- I'm not sure I agree that these five buckets are very distinct, nor that they encompass all things referred to as "technical debt". They're good examples worth thinking about, but I don't think they're as archetypical as the author does.
- I've never seen it as an assumption that "technical debt" implies "the people before us did a bad job", something the author spends a lot of effort arguing against. The people before you lay out a runway, which helps determine how quickly technical debt will build up, but even the very best codebases accumulate it eventually.
It is possible to create very extensible software architectures, I’ve met many engineers that will spend a great deal of time anticipating future needs, but in many cases this constitutes premature optimization.
You can address issues that are damaging to productivity, operability and morale, but only way to “fix technical debt” is “rm -rf”.
I think he was saying this ironically, but this is a very real possibility sometimes. There were often times I’ve found (and wrote) code dealing with cross cutting concerns that weren’t core to our business and I ripped it out and used a popular third party (open source) package.
On a higher level, I champion using managed services whenever possible over doing “undifferentiated heavy lifting”.
You should throw out old code when any of these apply 1) The code becomes irrelevant 2) The code becomes redundant (you found a way to generalize things) 3) The code is so bad that maintaining it is simply not worth it any more.
These things happen frequently enough that removing code is usually a reasonable action.
There’s also some level of people skills involved here and convincing your stakeholders that it’s better to sunset a feature vs rewriting it.
I don't want to speak for Kellan but I would be very very surprised if this was ironic. From my conversations from having worked with him at Etsy, and from my conversations with people who reported to him for a long time, he absolutely believes it when he says all code eventually becomes a liability.
Item 1: Security, Performance, Stability
Item 2: Fix, Update, Upgrade, Improvement
Examples: "Security Fix", "Performance Improvement", "Stability Update", etc.
Apt-get update doesn't update the system at all, it updates apt.
I like where this framework is going.
It also seems like markets and disasters can seem random from a SWE point of view. You can build a great system and people just might not take to it, or the whole thing could topple over because of a third party component, a really determined hacker, etc.
This left me open to being ousted (e.g., the Product Manager "had a friend" who could surely do everything much faster.) Seems easy from the outside. They got rid of me, hired the friend, who both increased the team size and then re-built from scratch.
Selfishly, it would have been better to more steadily deliver features and balance speed with maintainability -- because sometimes higherups dont understand the tradeoff they are making.
Anyone who has refurbished a house will live and breathe examples of technical debt embedded in the workmanship, and the material / design choices around them.
As you discover past shortcuts made by hurried builders, asbestos components or electrical cabling no longer safe to use, you quickly realise that you have to address these issues in a way that doesn't come back to bite you down the line.
As a regular homeowner you are transformed into the perfect dogfooder for House 2.0, and this perspective should inform your (day job) approach to technical debt as it is traditionally defined.
We don’t do this normally in most homes around the world, but families of multi-generational wealth seem to be able to do something similar with their homes because they both have means and will to build an estate that lasts centuries.
Could be a sign that things got too toxic in software development if developers can't even talk openly about experimentation and research.
For example: you started a few feature with a few scripts, added a few new things here and there ad-hoc. Now it's growing and full blown and nary impossible to maintain. A more clear understanding of the problem space now exists, and also that a refactor would make sense.
The continued inability to really move forward due to the rag-tag system is a kind of debt, and the trade off/decision has to be made as to wether or not the team needs to take a few steps back before moving forward.
This is really common.
I worked at BlackBerry during their heyday. If you are old enough to remember, there used to be international headlines when the BB email service went down. Everyone in the press was using BB and would harp on it. This is partly because BlackBerry had made a bunch of acquisitions and gun-taped the systems together, as opposed to properly integrating them. There was a real 'cost' to this bad press. The only real and fundamental way to avoid those systematic failures was to do a proper integration which in and of itself was a risky and unwieldy proposition for the company at the time.
I'd argue that was a fairly material and validated form of technical debt.
There's quite a bit of published literature on technical debt, which appeared quite recently (~5 or 6 years ago). One distinction is between normal technical debt (code level), and architecture technical debt a.k.a ATD (system level) [1]. So far, whether you like the term or not as a patient, it seems useful for scientists to think collectively about the phenomena in these terms.
Some commenters in previous thread say an ideal (reference frame) doesn't exist, from which to calculate the delta of TD currently. The symptoms do exist. That extra few hours it takes to onboard a new developer, fix a bug or whatever are observable. If there exists a possibility to reduce that observable, measurable effort, then that possible future timeline should be the reference frame.
I've risked an appeal to authority in order to transfer better understanding as a random stranger on the Internet. Technical debt seems a topic too fuzzy to fully understand, and that's why I'm pointing out the way scientists are thinking about it. I don't see any other clear path we're going to get industry wide consensus or understanding, if not for the scientific method and institutional support.
That being said, the article did a good job of communicating at least 5 different ways of thinking about TD.
Disclaimer: I'm building an architectural technical debt research tool called RefactorKit.com
1.https://www.researchgate.net/profile/Terese_Besker/publicati...
The amount of technical debt can be described here by identifying the quickest shortcut as well as the most correct solution, choosing the solution between those two on the spectrum that makes the most sense for current time and staff, and taking the difference between where you landed and the most correct solution. This difference is technical debt.
Taking on the debt does most times get you further faster, but in turn it slows you down in the long run. The longer you wait to pay it off, the more time in the interim you spend making minimum payments that get you nowhere but consume time.
Some companies have gone light on this because the liability/cost of failure was not so much.
But at some point, some of those risks become real.
The costs aren't always apparent, because that's the nature of 'black swans' - but the systematic risk is.
STechnical debt is not a universal thing. You can have a debt free code, then new hire will argue that whole thing is going to collapse and needs to be rewritten. Even when rewriting it brings 0 ROI and it actually good as it is. I’ve seen fair share of engineering initiatives that were complete waste of time, caused by attempts to fight “tech debt”.
The only good use of technical debt is to manipulate business folks into approving projects you want to work on. And I’m guilty of using it that way.