Bad code isn’t technical debt, it’s an unhedged call option
higherorderlogic.com
higherorderlogic.com
Call options are a better model than debt for cruddy code (without tests) because they capture the unpredictability of what we do. If I slap in an a feature without cleaning up then I get the benefit immediately, I collect the premium. If I never see that code again, then I’m ahead and, in retrospect, it would have been foolish to have spent time cleaning it up.
So porque no los dos?
Specifically, it is because the managers I am attempting to communicate with understand the concept of "debt" and do NOT understand the concept of "uncovered calls". I work at a bank and VERY few of the managers I need to explain things to would really understand uncovered calls. I would guess that people at other kinds of employers may find it even worse.
The analogy totally holds. If you don't pay back financial debts (e.g. airlines going through chapter 11 and shedding their pension obligations), then you ALSO come out ahead.
In a perfect world, we'd be guaranteed that our code will never need to change. In this world, abstraction is worse than worthless - it is a cost with no benefit.
In the real world, there are risks. Bugs will be fixed, requirements may be changed, back end systems may be replaced or scaled, and so on.
Building things the "right way" is exactly like buying insurance against those future risks. Business-savvy developers know how to estimate the probability and cost of the risk this code will face over its lifetime, and weigh that against what it would cost to build in extra code to mitigate those risks right now.
Thus, choosing the right level of abstraction and engineering overhead to add to a particular feature should be a series of decisions and tradeoffs, not just a determination that it's always better to build the "right way".
In my experience, developers tend to be overly confident in their ability to predict and mitigate future risks, and suffer severely from hindsight bias. They rarely lament the extra abstraction layer they built and never needed, but are quick to point out when modifying a feature will cost more because of weak abstraction.
Abstraction has a lot of benefits besides being able to adapt to changes.
A hypothetically perfect immutable system does not need to be testable, maintainable, or understandable. It becomes a black box.
That being said I agree that it's always a trade off; I actually did many refactorings where I threw an abstraction level out instead of adding more of them, resulting in faster and easier to maintain code. It's definitely not good to invest in abstraction blindly.
In this perfect hypothetical world, you wouldn't eschew abstraction; you'd just know exactly what abstractions to use to perfectly model the problem you're solving. Those ideal abstractions would be different, since the problem is immutable.
Ask yourself this: in this perfect world, would you code with a magnetized needle and a steady hand? Why not?
As to the rest of your point, yes. Anything in service of pure development speed would be desirable. If you could abstract and reuse something faster than you could implement it in some other way (eg copy/paste), then it would make sense to do so.
What would the engineering equivalent of that be, I wonder?
Quitting and getting hired back as a consultant?
If the market value seems greater before you've actually shipped: congrats, you just made money shorting your own company.
Invest in your competitor.
After 6 years of refactoring we now have a framework in place to start on the financial logic. It shouldn't take us more than 6 years to get the rest rewritten.
One of the hidden costs of bad code is that it is contagious. After staring at bad code, it is very hard to write very good code. So it takes several revisions to go from bad code to good code, to get the patterns right, etc.
That's why the debt analogy fits so well - because technical debt compounds. If your codebase is already heavily in technical debt, when developing new features or fixing bugs you are faced with three equally unpalatable options:
1) Do it properly (takes: 1 month)
2) Don't make it worse; don't make it better (takes: 1 week)
3) Just do a quick hack to work around all of the other hacks (takes: 30 minutes)
As a developer with a pushy boss who doesn't understand technical debt and a tight deadline, the rational choice is pretty obvious: take 3, go home at a reasonable hour to your family, still meet the deadline and drown your professional sorrows in alcohol.
The problem is this. Let me put it clearly:
If you are maintaining bad code, it will take you several tries to come up with a good architecture to replace it. You will make better progress having someone entirely outside do the architecture themselves.
OTOH, if employers are going to use credit scores to decide whether to hire you, then technical debt may be an even better metaphor if you factor that in ;-)
OTOH what one gets by going this way is a community of users, and therefore it will still beat Hurd 1.0 to completion. Starting an ERP from scratch without funding as a multi-vendor effort would lead to something that would never be released.
But you should see the legacy code. The code we forked from was a textbook of how not to write (secure, maintainable, bug-free, etc) software.
With tech projects there's almost always a de facto "abandon" option – if not for the firm then at least for the individual engineers. So whenever the cost of proceeding is higher than the expected value, you exit. That clips the downside, more like some sort of combined option position.
Or more like a specific kind of debt: nonrecourse loans secured by collateral. You get the money up front, but if it proves impossible to repay, you simply surrender the collateral. In this case, the collateral is the project itself: either the option to continue, or the IP rights, or the enclosing firm. And, in the event of failure, those may be worth nothing, so you aren't losing an unbounded amount.
For monetary debt, these fragmentary artifacts may in fact be surrendered to the creditors. In the case of metaphorical technical debt, you surrender up those hopes and dreams and mental (sunk) costs, to the reality that there won't ever be the time and budget to fix the system.
And so this leads to a different conclusion than the article, which ends with a near-religious stance against the inflexible evil of debt. Because the downside – project abandonment – is capped, sometimes technical debt is worth taking on, when acceleration-to-market (and thus market-feedback-learning) is of paramount importance. You're borrowing from the future, but you only pay it back (with interest) if there's wild success. If you fail for any other reason – perhaps things having nothing to do with the technical debt – you don't have to pay it off, you just surrender the (essentially worthless) collateral.
That's a hard thing for people with an aesthetic or craftsperson mentality to accept. And it still sucks when it's the technical debt itself – the cost of fixing old rushed choices – that occasionally makes a project no longer competitively viable. Your monetary credit report is unblemished, but your self-conception can take a hit.
I don't know in which context the writer is writing, I'm a developer and have been mostly working on startup-style projects. If you fail, you don't make a buck, the product does not work or does not sell. The downside is the lost time and money.
I have also known couple of startups, which have had really good engineers who have invested lots to testing, maintainability etc. In the end however the business hasn't succeeded selling the product, and all that investment was worth essentially nothing.
Don't take it so literally; it's just a short-selling simile.
Or destroys the users' financial well-being: software they use to run their small business, or invest their retirement?
https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dha...
Whether or not this can be attributed to technical debt, I don't know. But it is an instance of a software failure that resulted in death.
I suppose net across the system, you could inflict more downside to others exposed to your default, but it's not unlimited unless you can wipe out the entire planetary economy (financial and non financial) with your bet.
A bit like learning to fish is better than giving a guy a fish.
the term "technical debt" is fairly clear; "technical credit" or 'technical lending" is fuzzier but also more descriptive. Maybe the compromise is "technical debt with interest"
actually, "technical trading" might be a good contender too. You might make out in the long run, or crash and burn. It's all about trade-offs anyway; not writing tests now means future you will have to write them. Sometimes this is fine, sometimes this is a truly horrible idea, and sometimes it could go either way.
So if you think we need better differentiation of technical debt (which I do) then finance is a good source of those differentiations[1]
Personally I've heard tech debt used to describe everything from tightly coupled UI/DB code, to design decisions the person disagreed with, to code without tests. Even though those are obviously 3 different problems. So I'm for increasing the size of our vocabulary
[1] I actually wrote a whole article using finance terms to think about tech debt. https://medium.com/@the_ajohnston/questions-about-the-debt-i...
And technical trading as a term is just completely wrong because the term technical trading already has a meaning; and it has nothing to do with understanding the unknown future benefit or gain. It has to do with making decisions (and predictions) based in extrapolating past results into the future. If anything we should be calling it fundamental trading since fundamental trading is making investment decisions based on a holistic view of the security in question. So, the decision to incur a technical "debt" is made with an understanding of the entire system and the cost-benefit analysis of paying "cash" now versus paying an unknown amount of cash later -- or not having to pay anything at all.
One could call it gambling, but a more accurate description would be risk management. Using a risk management decision matrix one could say that the risk of someone dying in a flood at a shopping mall would be catastrophic, however the likelihood is exceptionally low, so it wouldn't make sense to have life boats spaced every 10 meters. However, it could happen, but the shopping mall owner chose to incur that "debt" of not having lifeboats because they determined that the cost wasn't justified given the profile of the risk. A debt analogy would say that a shopping mall absolutely will be flooded eventually and the debt would exist until such day that the owner bought lifeboats.
Premature optimization is the root of all evil (or at least most of it) in programming.
The problem of making something worth maintaining has priority...and the software upon which it depends is often a second or third order priority. Facebook was built on PHP. That using PHP created technical debt was a nice problem to have on the way to the bank. None of which is to say that writing bad software is ok. It is to say that bad software includes software that wastes time trying to anticipate and solve the wrong problems at the wrong time for the sake of an ideal rather than current business needs.
What I've actually seen in practice is that "technical debt" mostly means "things I wouldn't have written that way." It has very little to do with the code and more to do with the philosophical leanings of the commenter.
Technical debt is a great metaphor for explaining this to non-tech stakeholders. They understand that choosing to rush a feature out now will lead to slower development (paying back the debt) later.
There is:
1) Code that is covered by tests
2) DRY code
3) Loosely coupled code
4) Code that fails fast
5) Code that doesn't reinvent the wheel
Most technical debt I've seen violates one of the above principles. It is crystal fucking clear when you are paying it off and the benefits to paying it off are pretty obvious to those working on the code base.
It is impossible to avoid violating them completely, but low technical debt means small and rare violations, whereas high technical debt means large and extremely common violations.
>What I've actually seen in practice is that "technical debt" mostly means "things I wouldn't have written that way." It has very little to do with the code and more to do with the philosophical leanings of the commenter.
In practice I find that most people know terrible code when they see it, but if given the chance to rewrite from scratch most will end up digging themselves into the same hole.
Bad coders will not realize that they are doing it. Good coders can also end up doing it simply for expediency.
I find that the best way of paying off the technical debt is simply to work my way through fixing things 1-5, roughly in that order. It is a slow and largely thankless process.
Selling a naked put - Integrating a 3rd party library for a feature vs building it yourself. Immediate benefits, and probably unlikely to go sideways, but if the framework turns out to suck you suddenly incur a large unexpected body of pain. However, unlike a naked call, there's understood limited downside, limited to the functionality the library provides.
Buying a put - Building in any kind of protection from "very low probability but high damage" events, such as provisioning a completely separate infrastructure in the case of a massive DC or network outage, or having a completely separate monitoring system in case the first goes down in the middle of fighting a separate fire.
Buying a call - Basically anytime you make an engineering bet that costs a bit of time and is unlikely to pan out but if it does, you win big. Like a spike to try some state of the art algorithm or bringing on a short term consultant to solve a really hard problem. If your whole team is always doing these in the long run you lose, but strategically doing these when they make sense in the long run can result in huge gains.
Selling a covered call - Focusing on consulting services vs building a product. Steady income, but in the unlikely case you build something that strikes gold, it won't be you who becomes rich overnight, it's the person who paid you to build it.
You can build a shanty village without a plan that supports a million people, but the first fire, storm, earthquake, etc destroys the whole thing. Also, for some reason when Bob flushes his toilet, the power goes out briefly in the capitol building. Nobody knows why, but routing power through the sewer last sprint to save time probably wasn't a good idea.
Another good one is, "Well, I can build you a 20,000-foot fire-breathing chicken made of balsamwood, if that's what you want, or if you just want me to parse a CSV file, I can do that too."
As a developer who has to hear the "tech debt" thing all the time from my co-workers, I really hate that concept. Metaphorically or otherwise, I don't like being in the position of a bank demanding high interest payments from a poor sap who didn't realize what he was getting into. That's not how I view myself in relation to my employers and clients.
But anyway, I agree - naked call options is a much more accurate term. But it is a difficult term for folks to get their heads around (incurring debt is something the post Marx Brothers world has gotten used to. Outside specialised areas of finance, not so much with Options)
Chico exclaims "Did you see that - the horse cleared the car !" Groucho - "well, I wish I could clear mine!"
And 80 years ago we have the consumer debt crisis still writ large.
Comparatively very few people have a useful intuition about "unhedged call options" that using it as a metaphor for poor code quality would leverage.
Also, I think predictable ongoing support cost is a big result of poor code quality in production systems, so that aspect of the debt metaphor isn't completely off-base (there are also unpredictable potential costs in the future, as well, so its not a perfect analogy.)
Options are tricky to understand but essentially, think of it as just a piece of paper or a contract between you and the person buying it from you. It's a buy/sell market that exists on these "papers". Options trading essentially revolve around buying these papers which usually have an expiry date of between weeks to years and at what price you can buy the underlying good before the expiry date. The profit is made from the valuation of this paper going up and down based on the underlying value of what this paper represents. The paper can be contract about beef, corn, oil and share price of Apple. Just like the stock market, you want to pay cheap price for a paper and sell it when it goes high, but the beauty of options is since you don't actually trade the actual good itself, you can create a dizzying array of strategies and combinations to make money in any type of situation, but with the condition that you have to be right about what market we are in (trending up, down, sidways, volatility etc).
Writing an option is like selling a piece of paper that says the person buying your paper will have the ability to buy the stock at the price written on the paper. If the price is low and the stock price goes up, well you now have to buy X number of shares that was written on the paper at the high price to give to this guy, causing great deal of loss to you (since you sold the paper for some cash earlier).
If the stock price falls, then you could buy the shares at the low low price using the money you made from selling the paper and give it to the other guy, who most likely won't ask you to do this and just think about the time he paid you to write him a piece of paper that is now "expired" or worthless.
I think the uncovered call metaphor is a brilliant way of framing this issue. It doesn't matter if we know finance or not, it's the way of looking at the problem that provides us with a good framework with which to make development decisions. If we are aware of the unlimited downside and manage that risk, we can actually be more aware of when it makes sense to incur technical debt and when it doesn't. In my opinion, it, just like options trading, is all about risk management and certainly not about traditional debt management.
So I think it needs to be judged in context - who are we trying to communicate with?
This book, is not specifically about coupling but the first chapter is an excellent overview that all developers should read:
Well, not really. For example, if you took usd-based debt in russia, you are now quite royally fucked.
The whole thing is about risk, if you hedge away all your risk financially there is generally no profit to be made (you're left with arbitrage), if you hedge away all your technical debt then you've essentially (re)implemented it properly.
Explaining that doing things "bad" creates "technical debt" or "unhedged call options" I think sets us up for not being taken all that seriously because these things themselves are abstractions that we have to work hard to reason about effectively.
Is it more useful if we say something tangible--that programming is like working on a house? Imagine you just started working on updating a house last month. The owner comes to you with a request that they be able to see what's going on in the kitchen from the living room; you _could_ just knock a hole in the wall. That would satisfy the basic requirements, but it's going to make a mess, it's going to look like shit, and it may disrupt the electricity/plumbing/hvac or even the stability of the house itself. But even if we make a proper window or door between the rooms without disrupting other systems in the house and clean up our mess, it's still possible we make a door or window that is a little different than all of the other doors and windows the past builders have put in the house.
In 2 or 10 or 20 years, after several maintainers have come and gone, the owner discovers that they're being bled dry by the house's energy inefficiency. The new maintainer may go measure a window twice and get the dimensions down precisely and then count all of the windows in the house that look the same. Then they go place an order for N new energy-efficient windows. When the windows arrive and they're getting ready to replace them, they come to the terrifying realization that every window in the house is actually custom, and only a few of the windows will fit. They install the windows that fit and try to return the others only to find out the vendor charges a healthy restocking fee. Down 40% of our budget for replacing the windows but only having actually replaced 15% of them, we no longer have enough money to replace the rest. The owner decides to just use the remaining 60% to cover the increased energy costs instead of fixing the issues.
The new maintainer goes to put the diagonal blinds back up on the windows that did get replaced and discovers the custom mounting hardware has gone missing in the shuffle, so the blinds get left in the corner with a note that someone needs to find a way to put them back up when there is time and money. It's the middle of the winter anyways; the sun is always under the horizon when the owners are home. Because this is a _funhouse_, these were of course the blinds with the greatest role in reflecting summer heat, and despite the improved window technology, total energy efficiency is going to be in the shitter come summertime. For now, perhaps quixotically, energy efficiency is buoyed by the extra solar heat; the owner seems happy enough with the updates, and the metrics all show good performance.
Houses need maintenance. Things on or in them break and can't be fixed because either the parts aren't made that way anymore or the knowledge of how to fix it has eroded. They end up with empty telephone nooks and doors that were sheet-rocked over instead of being removed. A major appliance stops working and you discover the house needs infrastructure updates before a contemporary version of the same can be installed. The maintainers want to add a floor and some new rooms and they need an architect and an engineer to make sure they aren't going to ruin the roof line, slab or overload a wall. Down the road it will have different rooms built in the fashion of three decades over a span of 80 years. Further down the road a room will get sealed off when the roof caves in and the owner doesn't have the funds to fix it.
Code quality problems are, as expressed in the OP, an unknown unknown. It's rare that anyone has a handle on just how bad things are, until it's too late. Also, there are sociological issues with legacy rescue projects that businesspeople, who'd rather think of programmers as fairly interchangeable, don't like to contend with: maintenance is only effective if done by high-quality programmers, but it's expensive and often hard as hell to motivate good people to do it. A pat on the back and a 10% bonus won't do it, because it doesn't come close to offsetting the career costs of doing undesirable work.
There is an apocryphal story about a trader buying chocolate santa futures and forgetting to sell them on. Eventually a truckload turned up at the Wall Street headquarters.
This has little to do with the subject itself, but this has actually happened, sort-of. Futures contracts usually don't include Wall Street as a delivery location, so it's not like the trader ends up taking physical delivery at the workplace. The contract will specify where one must make or take delivery, and most delivery locations are in the Midwest or Plains.
When a trader fucks up and has to make or take delivery, there are services that handle it, but it's expensive. If you get stuck taking delivery of, say, $500,000 worth of butter, you'll probably have to pay the agency 50 cents on the dollar, or $250,000, to handle the task of getting your surplus butter to a wholesaler. It hurts pretty bad.
This pretty much describes what I've been enduring for the past year, especially since we only get (adjusted for profits + performance) 10% bonuses.
I also think that the call option is a bad analogy for similar reasons. In a call option you could be obliged to pay out. You never have to pay out with code. You can decide to get your developers to write code or not write code. No bailiffs will come knocking on the door.
Technical drag coefficient may be a better analogy.
For a car, the drag force increases as you speed increases.
For code, the more features you want to add, bug you want to fix, etc. (i.e. the bigger your velocity) the more the TDC will kick in to slow you down. It may put a speed limit entirely on what you can do until you put work in to reduce it.
Also you can write-off the car and invest in a new one. I have seen this many times of course - the 'new product' that is written that won't have all of the problems of the last one.
To really stretch the analogy, those same people often then end up in (technical) debt again a few years later when they repeat the same mistakes.
In some ways it is worse than real debt - the compound interest rate is atrocious. Like a loan shark.
But in some ways it is not as bad - you never are required by another party to pay back the debt. If you stop making money from the product, the debt is cancelled to $0.
Real debt has to be paid down, unless there is collateral, but even then most lenders want to know when and how they are getting their money back.
Moreover, in a startup context, both metaphors break down: you can walk away from your technical debt by shutting down. You can't walk away from your financial debts (or call option shorts) so easily.