Technical Debt
martinfowler.com
martinfowler.com
Making poor architectural decisions or writing poor quality code isn’t tech debt. They are just institutionalised consequences. The way the world now works.
Legacy code isn’t tech debt either: it was right for the time.
What you have instead is knowledge debt as engineers turn over, and the once intuitive and elegant parts of the codebase are utterly alien to the new hires who learned their craft in a totally different way. This isn’t tech debt because engineers can’t intuitively anticipate the business’s future hiring decisions, particularly when those are made independently of the team.
If you want to establish tech debt you have to define the schedule to pay it off and there should be some record of that work somewhere, with accountability attached. It is not merely the accumulation of things you don’t like the look of or find hard to work with.
Similarly, I think MVP is a conscious choice to incur product debt because it is far too risky and irresponsible to go all-in with your perfect vision right off the bat. You want your initial buy in and a chance to bail out or pivot before all is lost.
Edit: as an example that comes to mind, a bank running on COBOL doesn’t have tech debt unless their COBOL engineers are aware of their missed opportunities. If they think COBOL is no longer fit for purpose, it’s not debt, it’s time to reinvest.
You make a choice to accumulate knowledge debt by not writing docs, not cross training team mates, etc.
This, and I will accept being wrong, is why I think we see code we didn’t write and immediately see how we would do it better. What we’re doing is sending ourselves back in time and replacing their shoes with our own. They didn’t know what you know and you have no idea what they knew.
A potentially effective approach is to introduce an immutable decision log. We have these people in many other fields - they record the decisions made and they become part of the overall reasoning.
I reckon anyone in a position of power should have an independent decision log. Tech architects, CTOs...
After the MVP, you may wish to add more features or support more users, it is at that point that you can assess what is debt and what is not. It's not a matter of "cruftiness" but a matter of what's holding you back from adding additional features or supporting a larger number of users.
If we frame all work as doing something that will deliver some sort of value (be it customer value or unlock the potential to add more features to add customer value), then it becomes a more constructive conversation about work that needs to be done, debt or otherwise.
I'm not a fan of just looking at whether code is crufty or not to determine if it's "technical debt". If you look at code long enough, you'll find something you don't like and want to rewrite, but it has to be in the service of some other goal or else you'll just become an architecture astronaut building wonderful things that don't provide value.
Edit: as an example that comes to mind, a bank running on COBOL doesn’t have tech debt unless their COBOL engineers are aware of their missed opportunities. If they think COBOL is no longer fit for purpose, it’s not debt, it’s time to reinvest.
Should there be an analogy for depreciation? Some of the "best practices" in my current C++ shop are in direct contradiction to the Law of Demeter and related philosophical stances in Object Oriented programming (particularly ones espoused by Martin Fowler) despite the fact that my current shop advocates for OO. Neither the old position or the my current shop's position is so bad that it's not viable, though some hotheads in the field act as though this could be the case.
A lot of the churn seems to be arbitrary and driven more by fashion than solid first principles. (Which would include cost-benefit analysis, for which there is often insufficient data, and for which no one wants to pay the cost to gather the data.)
sure it is, in the same way that taking one of those payday loans to cover a coke habit is still real debt, it's just real bad debt.
Besides which, it trivialises the seriousness and the tragedy of addiction. If you think your tech debt is similar to that you are well equipped to quit your job and work elsewhere. An addict does not have such an easy choice, they can’t leave their addiction to someone else and move on.
You choose to take the payday loan or hit up a shark, addiction or no.
The point where I concur is when you gamble the short term against the future despite all evidence that you will benefit from spending time to save time.
Agreed. The addict might recover. Organizations with poor culture are doomed.
That's not what "technical dept" is. Choosing a quick solution instead of a proper one that would take more effort is. I've made that conscious more than once myself: to hit a deadline, or because I didn't know if the MVP would even survive long enough to worry about code maintenance.
Doesn’t this assume that code as written was of the highest quality? Under tight deadlines or misaligned priorities code quality suffers. Even if it works or “is right” doesn’t mean it wasn’t done to a standard that maximizes productivity.
If they consciously chose to sacrifice on it, that’s the debt.
I think there's a lot of value to be found in separating the known debt from the unknown debt though, and giving it a name. I might have to start doing it.
So thanks for making me think about it.
This assumes all things can be usefully viewed through the prism and language of economics (though my gut feel is few things can or should be).
Making progress in the world is fighting against entropy. Just because it is hard to fight the disorganizing efforts of the universe doesn’t mean that your output is a liability.
> liability: a thing for which someone is responsible, especially a debt or financial obligation.
According to the balance sheet equation,
> Assets = Liabilities + Owners Equity
Certainly all code is a liability - it must be at least understood and maintained from time to time (80% of lifetime cost is in the maintenance).
And while an accountant might also mark it an asset - not all code is an asset - and we recognize that fact when we delete for badness or disuse - or rewrite it to make extending and maintaining it easier.
I'd accept that debts are a subset of liabilities, but not that 'liabilities are debts'.
A debt implies an externality, while a liability implies some generic exposure -- it's that which I focus on when considering (all) code as a risk to an organisation.
I suspect, however, that we agree on the fundamentals. There's some cost and risk to developing and trying to maintain any code base, and accepting that can usefully inform development and operations work better than thinking of code as a (near-)risk-free asset.
I used to program COBOL on IBM 370 DOS/VSE systems they later went to Visual BASIC on Window 3.1 PCs. The conversion took a long time and most code had to be rewritten because COBOL is not BASIC. Management didn't care about that, they just wanted a working project. We all did our best going overtime without pay and working weekends to meet deadlines.
COBOL was a liability when they wanted to convert to Visual BASIC. There was liability to convert the code, convert from DB2 to SQL Server on the database end.
If that were the case, the company would have a negative value.
Code is asset that requires maintenance.
But yes, it's definitely worth reminding ourselves of the ongoing cost of maintaining code.
I reconcile the concept that code is a debt with your statement in the following way:
The feature produced by a piece of code can be an asset, but the code itself is a debt.
Thus, for a new feature to be valuable, you want the value added of the feature (or asset) to out-weight the cost of the code (or debt).
As a general rule, the complexity that comes along with a piece of code grows as the code-base and feature set grows, thus, for a young codebase, the value of a feature more easily out-weights the cost of the code.
'Technical debt' is just not debt, it's just an interesting way to think about it, and a neat use of lingo.
There are any number of other business practices for which one could try to apply the same analogy, but it wouldn't work because very quickly the term wouldn't seem right.
For example 'training employees' for a situation, knowing they might need to be trained differently in the future, is not going to be regarded as debt.
Same thing for processes, investment in equipment, parts, materials etc..
Although, you could say that a lot of profitable companies have negative value codebases where the cost to rebuild them is less than the cost to maintain.
This probably explains why some companies get replaced by small nimble startups.
Technical debt is configuration and plugins. You are able to deliver something now with the benefit of changing behaviour later but at the cost of having an enormously increased test surface and invasion into your type system with possibly hostile parameters.
You're betting that the maintenance foregone in the present is paid back with interest in the future.
This "pay back" could be the success of the company which allows you to pay for more developers and tools.
Is it worth it? Time will tell.
The cobol example is great - it is code still solving the problem but is no longer efficient or productive to find and pay cobol programmers.
“Legacy code” is usually a label applied exactly to identify that code which has been determined to represent technical debt.
I found it be a pretty accurate summation that gives some perspective from the other side.
The two categories overlap, but not all code generating revenue fits in the traditional scope of “Legacy Code” and not all “Legacy Code” is generating revenue.
> I found it be a pretty accurate summation that gives some perspective from the other side.
What “other side” are you referring to?
Technical debt that does not generate revenue in any way is often far easier to replace/refactor.
I look at legacy code like this:
- Could we remove this from the product? - Could we replace this part of the product with a totally new system? - Are we OK with potentially introducing subtle bugs in this code as we work through refactoring it.
If the answer to the above questions are “No”, it’s likely revenue generating code (or part of some compliance process).
In this case the “other side” is non-technical stakeholders who don’t inherently understand the importance of resolving technical debt.
You know, if there only were a way to pass knowledge from one person to another aside from directly talking to that person. Perhaps if we had means of preserving knowledge, and even making it a part of the job description... But alas.
Joking aside, not writing technical documentation and product design docs is very much technical debt.
A lot of information that is lost with turnover is information that could have been written down and passed on.
Documentation is not jut code comments. It is:
-A solid bug tracking system, with clean and reproducible bug descriptions
-A version control history, where every change has a why in addition to a brief how, and which links with the bug tracker
-Well-written tests, which not only verify how a system works, but showcase how to use the APIs.
-Documentation for tests explaining why this input was chosen, what kind of output is expected, and why it was written that way
-High level comments in code (especially libraries) explaining how the API should be used
-High level documentation explaining the structure of the code that is checked in, and lives with the code. The documentation can be in Markdown or a plaintext README, but should be there
-A "code map" telling you what the code in various directories does, and where to look for things.
-And, most importantly, a lot of design docs - for every feature and algorithm. "We are solving this problem. Here are the alternatives considered. Here is what we are going to do and why. Here is how we'll implement this. We used ideas from this, this, and this paper".
Yes, all of that is needed. The price you pay is engineering time. But if you take on the debt, your new engineers will have Hours Of Fun[1] trying to figure out, say, whether the imaginary nanometers[2] model parameter increasing as a result of their change is a good or a bad thing.
What I am saying is documentation debt is technical debt. Documentation is just as important as the code. And having it does help[3].
You might think that your code doesn't need all of that. And that's fine; but that means you plan on never paying that debt. Maybe you plan for your startup to fail/sell out in a few years. Maybe you plan to leave that job. But anything long term means you have to write things down.
And if you haven't been doing so - now is as good time as ever to start.
[1]http://wiki.c2.com/?HoursOfFun
[2]True story. There was a good reason for why the distance parameter had complex-valued nanometers as units; it was an elegant way to do improve a model while staying backwards-compatible. It was written down nowhere, and have fun googling "complex nanometers". They only exist in that code, AFAIK.
There is information that must be experienced, drilled, and committed to habit and muscle memory: tacit knowledge. Drill, practice, experiment, coaching. Often 1:1 or at least 1:few.
It's highly arguable that individual knowledge does not become cultural knowledge until a means of transmitting it from one generation to the next becomes both established on institutionalised. It need not be a large institution, but it must have persistence.
Within any knowledge domain, the explicit knowledge transmits more readily than the tacit. With time and complexity, I suspect a situation much analogous to Amdahl's law is reached, in which generational knowledge transfer comes to be dominated by the tacit component -- it is harder, slower, and more expensive to transmit, and tends to irreducibility.
Technical cultures which become or incorporate a tradition of shared stories and lore, as Unix and Linux have done, seem to achieve this better than others.
And that neglects issues of semantic and cultural drift which affect written texts.
Though I'd come to this conclusion some years ago, I'm finding reading Mortimer J. Adler's How to Read a Book that this is a significant theme of that work.
I think it is. Technical debt is all about trading current productivity for future productivity. Poor architectural decisions can be caused by several factors. Inexperienced devs, not taking enough time to understand problem, not spending enough time brainstorming the problem. These are all decisions that trade current productivity for future productivity, which is technical debt.
Also most technical debt is not accrued consciously. Very few people are doing cost benefit analysis on technical debt. It's usually made intuitively by project and product managers who don't directly interact with the costs of technical debt but directly experience the benefits.
Requirements changing is also a big one. 4 years into one project that I had created and maintained a "one to many" relationship changed to a "many to many" in some of the tables central to the system. Something that I had clarified wouldn't when originally designing the system.
This is a very succinct and accurate description, and one I haven't come across. Thank you for this phrasing :)
I compare technical debt to debt.
You may go in debt because you looked at the housing market and decided that now is likely a good time to buy a house with low interest rates. You take on a large load of debt, but it is done wisely based on currently information.
You may go in debt as a business operating on credit to fund an expansion that should pay off quite nicely in a few years time. Maybe it will fail, but it isn't a bad risk based on current knowledge.
You may go in debt for an education. Depending upon the degree and extent of debt this could be good or bad.
You may go in debt by spending beyond your means on a credit card. This is generally unwise.
You may go in debt because of a medical emergency that is entirely beyond your control (especially in some countries, but even some countries with universal healthcare it is possible to need treatment that isn't covered).
All of these have something similar when dealing with technical debt. Sometimes it is a wise investment. Sometimes it is a risk that may or may not pay off. Sometimes it is beyond one's control. Sometimes it is poor planning that will be regretted later.
For the cases like legacy code and turnover, perhaps "tech depreciation" is the more appropriate economic term. You didn't really borrow against the future to have more today, rather something naturally decreases in value over time.
If you just say that some code is bad you will get unproductive reactions like people getting defensive, trying to assign blame etc. Calling it technical debt allow you to call attention to an issue while avoiding blaming anybody.
What you have instead is knowledge debt as engineers turn over, and the once intuitive and elegant parts of the codebase are utterly alien to the new hires who learned their craft in a totally different way. This isn’t tech debt because engineers can’t intuitively anticipate the business’s future.
That is it! Thank you for sharing your thoughts.
The time it was written often isn't when the technical debt was incurred, when maintenance to keep it right, technologically, for the changing time was deferred, that's when a decision was made to incur a higher price later to save now: that's when the (technical) debt was incurred.
> as an example that comes to mind, a bank running on COBOL doesn’t have tech debt unless their COBOL engineers are aware of their missed opportunities
The existence of technical debt is unrelated to awareness of the deferred costs. “technical debt” isn't a statement about the existing technical staff’s perception of the system.
When I joined Netflix I was assigned to add DASH support to the Silverlight web player. I spent weeks working on the new architecture, refactoring the streaming code, etc. One day my manager stopped by my desk and said that "I needed to wrap it up. It was taking too long." I was still weeks away from being done. I explained to that I was cleaning up a ton of technical debt and that's why it was taking so long.
He explained to me that they had different values on his team.
1. Most code will be rewritten every 2 to 3 years. 2. For code that doesn't get rewritten often, preserving battle tested code trumps paying down technical debt
Just remember that technical debt doesn't apply to all software.
a.k.a. defaulting on the technical debt
Sometimes you can say "I know this will need to be rewritten later" and be pretty sure you're seeing technical debt be accrued.
But other times you have no idea. It might be a tool that was intended to be used for a few weeks, but turned out to be incredibly useful and lasted years. Had you known, you might have written it better if the first place.
Technical debt isn't like financial debt in that there's nobody with an accounts sheet telling you that you owe them anything from the start. You don't necessarily make a deal with anyone. So you can't judge its existence in the same way.
"The extra effort that it takes to add new features is the interest paid on the debt."
If adding a new feature starts with, "just throw out all the code and start from scratch", that's a much larger cost over the long term than building code that's easier to maintain and refresh.
(Mind you I'm not fully sold myself that 'technical debt' is a useful analogy at all)
Research code is much the same. Your code doesn't have to be great (or even OK, to be honest) as long as it runs well enough to provide the results you need for your paper.
Likewise, rewriting a front-end in the framework of the week doesn't have to be because of technical debt; Angular 1.x was fine and it's still fine, however you'll see a lot of upgrade or rewrite projects happening right now not because of tech debt, but because people simply don't want to work in Angular 1.x anymore, or an employer can't find competent people that want to work with it anymore.
Likewise, COBOL applications aren't bad because of technical debt per sè, it's just become very hard to find people that can maintain it.
> The only point here that jumps at me is that the manager came a few weeks to late to explain the development values.
Well...(and I'm not having a go at rsweeney21). They were asked to implement a feature, not to spend ages cleaning up technical debt, and being "was still weeks away from being done" would be an alarm bell for me as a senior dev/manager. If the feature couldn't ever possibly be implemented due to technical debt then GP should have raised this with their manager far earlier rather than just going ahead and fixing this silently. If the feature can be developed and released, despite technical dept, then go ahead and point out to your manager (as soon as possible) this is going to be horrible for the dev(s) working on the next iteration, but the chances are your manager will likely insist you get on with the feature. Communication is at fault here.
We all work in projects that have technical debt and we'd all love to go back and fix these things. But you also have that obvious tension which is the fact that you also need to release features which have been identified by the business as another positive cash-income thing because you're competing against other services - and sometimes these features are very time critical.
Sounds like GP poster seems reasonably experienced (working at MS, Netflix) and should probably know this (unless there are other mitigating factors not included in your comment)
If you never touch code again, but that code was rushed and is ugly, there's no good reason to pay off the debt.
Similarly, if the code's getting thrown out (and if you design code to get thrown out), there's less reason to prioritize cleanup.
A recent example: React introduced hooks, which effectively means that there's a new de facto standard way to write components and you shouldn't be writing components as JS classes anymore.
The kneejerk reaction is then "We need to rewrite all our class-based components to use hooks!", which can take quite a long time (and it's tedious, boring busywork).
Luckily the React documentation on hooks itself [0] basically tells you to not bother unless you're actually going to work in that component anyway, and even then you should make sure everyone else in your team understands them; classes are fine, it'll keep working for a long time, resist your kneejerk reaction.
[0] https://reactjs.org/docs/hooks-faq.html#should-i-use-hooks-c...
It's sometimes controversial to say "We should not prioritize tech debt" but I feel pretty strongly that it's true. We aren't paid to write beautiful code, we're paid to write code that gets the job done. If code is getting the job done, even if it's fragile, our job is done. Only if that code is impeding new work should we prioritize it.
That's kinda scary.
Sometimes it's rewritten because it's so unstable because debt wasn't paid off but total cost of writing it twice is much more than paying off as you go along. That's scary too.
Even well written code might not scale or be flexible to changing requirements. The feature could even be removed. The effort it took paying off tech debt prematurely would have been a waste.
Although then you butt up against the "Second System Effect", so you're damned if you do, damned if you don't.
I used to measure code lifetime in half-lives at Google: the half-life of most of the code there was about a year, meaning that after a year, 50% of your code will have been deleted. It's pretty accurate: by the time I left after 5 years, about 97% of the code I'd written had been deleted. Ironically, I'm told that my one contribution (after 10 years) that still exists is an attribute-renaming CL that I wrote over break at a team summit; basically the whole team agreed it was a good idea, we would never have a chance to fix the problem later, so I just went and did it before the framework got too entrenched to change. Meanwhile stuff I slaved over for months, sometimes even stuff that was directly sponsored by a VP (who is no longer there) or got commendations from the CEO (who is also no longer there) was gone within 1-2 years.
Expecting code to be replaced every 2 or 3 years is nervous-making for a couple of reasons.
First, new code is always less reliable than old code, so -- all other things being equal -- devs should lean toward keeping the old code rather than replacing it.
Second, code should be architected so that any changes required to adapt to changing conditions should be limited in scope. The bulk of code for most projects should only very rarely need changing to adapt to changing conditions.
Reliability issues at large consumer tech companies will usually be caught by the QA/canary/SRE process. If they're not, you'll hear about them soon enough with 100M users banging on the product, and then just do a rollback and fix it on a more leisurely pace. Consumer expectations in non-tech industries have gotten so low that you can burn down whole towns, poison people, and leak millions of records of personal data with few consequences to the company; not being able to access your favorite video for 15 minutes is comparatively minor.
Also, the type of changes in market conditions consumer companies face are usually not those you can architect around. They include things like "We are no longer shipping DVDs to customers; we are streaming content online", "We no longer write web software, we're a mobile-first company", and "our business model is no longer selling personal data, it's payments". There is no architectural fix for "your product is canceled".
I have a great deal of experience with both environments.
I'd say it's an outright incorrect statement, fads come and go but the revolutions (like the move to web apps) are rare and take and take decades to complete. If everything around you changes every 2-3 years then you're hanging around fashionistas not engineers.
The example at hand was netflix... The bulk of the platform seems pretty stable to me. If they are rewriting their best code every 2-3 years that'd be a red-flag to me.
We did things like incrementally rewrite a million-line binary from C++ into Java while it was running, or completely change the indexing system from batch processing to continuous updating, or grow the number of documents indexed from about 80B to 1T. There's a huge iceberg of development that you don't see, much of which has to do with scalability (you generally have to rewrite every time a key metric grows by a factor of 10) and much of which is experimental, trying out new features to see what resonates with the userbase.
There were some pragmatic reasons though. It's very difficult to multithread C++ correctly, while Java at least has a proper memory model and thread support (note that this was before C++11; at the time C++ had no standardized memory model at all). Debugging core dumps in production sucked. Most of the newer parts of the company (GMail, Docs, Google+, etc.) were written in Java, and the rewrite let us share code with them. Compile times sucked, and Java let us pluginize the architecture, load code at runtime, and build & push each component team's codebase independently (as well as shut them off independently if they started crashing).
As the article mentions, it's about figuring out what makes sense to fix now, or wait until later.
But yes technical debt applies to all projects. The way we interact with it just varies.
An argument that "eh, most code is rewritten anyway, who cares" is what I'm reacting to, and have seen countless times myself.
But I don't want to say there is never a time someone will say, "this code is going away, don't spend too much time fixing it now" and it's really really good advise.
The ones that didn't were effectively stalled projects. This makes me think that technical debt really can be thought about in a completely different way for frontend code in many cases.
In theory, this is just marketing, but in practice if often turns into much more.
I don’t think they quite got to the point where they had to rewrite it.
The original developers have left after a few years, newcomers don't understand the system and may not even try to. It will ultimately get rewritten in newer technologies and frameworks, that happen to be nice for their resume, whether it was obsolete or not.
I'm not following. You still need code that could be rewritten and deleted every few years. Any code that couldn't just be deleted as needed wouldn't fit that plan.
So whatever forced code to outlive its welcome is technical debt in that scenario. Examples could be code that used outdated middleware, maintained arcane ETLs to keep data accurate, consumed said data in unconventional ways, code that wasn't repackaged and deployed regularly, etc.
It should have been communicated earlier though, so you wouldn't have wasted weeks on soon-to-be dead tech.
Seems weird to do that without talking to your boss about it.
The "handy rule" from the book Team Geek -- the newer edition is called Debugging Teams[1]. It's from chapter "Offensive versus Defensive work":
[...] After this bad experience, Ben began to categorize all work as either “offensive” or “defensive.” Offensive work is typically effort toward new user-visible features—shiny things that are easy to show outsiders and get them excited about, or things that noticeably advance the sexiness of a product (e.g., improved UI, speed, or interoperability). Defensive work is effort aimed at the long-term health of a product (e.g., code refactoring, feature rewrites, schema changes, data migra- tion, or improved emergency monitoring). Defensive activities make the product more maintainable, stable, and reliable. And yet, despite the fact that they’re absolutely critical, you get no political credit for doing them. If you spend all your time on them, people perceive your product as holding still. And to make wordplay on an old maxim: “Perception is nine-tenths of the law.”
We now have a handy rule we live by: a team should never spend more than one-third to one-half of its time and energy on defensive work, no matter how much technical debt there is. Any more time spent is a recipe for political suicide.
I've always found this hilarious... we spend thousands on hiring developers and one of the primary goals is to find a well rounded person who is okay doing maintenance tasks and ensuring proper code quality and knows how to write effective tests. A short time later, I come on to a project just for a status update and find the developers cutting tests and maintenance tasks at the orders of the PO who then complains that tickets are getting larger and larger.
Reclassification.
Refactoring should maintain identical functionality not counting performance unless it's a requirement.
Of course, many refactorings will make it easier to apply performance optimizations later.
Generally refactoring worries only about the functionality that you test in unit tests and integration tests, not counting performance.
This why I think it might be useful to allow the stakeholders and pms to decide the % of time spent on defensive work instead of what defensive work to do. And then you have a specific metric on technical debt(we budgeted 30% for defensive work and we've only spent 10% that's a 1000hrs of technical debt) Something to specifically point to when a stakeholder asks why is this taking so long or why is the application buggy. And ideally this caused a company wide increase in the allotted time for defensive work.
Bonus:It will also give developers who like working on clean projects a number to ask when being hired.
That would solve a problem I saw at a previous job: new features would be described as technical debt, so they could take up tech debt time at the expense of _real_ debt.
This is precisely what bothers me about the vast majority of software out there. It's hard to find software that doesn't suffer from what I call, for lack of a better name, "forced evolution": the devs keep fiddling with the UI and adding new features of dubious usefulness. Meanwhile, the quality slowly, but surely deteriorates: the software gets bloated, runs slower, becomes harder to use, etc.
This was how I said it, buried off in the documentation of an open source project:
I'm not a fan of the way most software is developed, where features accrete on features in endless succession like barnacles attaching to the hull of a ship until there's more barnacles than ship. I think most developers don't spend enough time on quality, on refactoring, or on cleanliness, and I think they don't weigh the costs of every new feature (and every piece of old compatibility cruft) as heavily as they should. I'm also not a fan of the kind of personal rigidity where one never questions one's own past decisions and thoughts. I regularly make stupid design mistakes, and I don't want to have to live with them forever. Software projects should at least try to value simplicity and purposeful design, and try to avoid the Second System Effect.
-- it's likely revenue-positive
-- it's visible
-- it involves understanding customer needs
I wonder if that gives room to take on extreme debt at the last minute before shipping a game. An example might be when Steve Ellis added multiplayer to Goldeneye on N64 a couple months before launch. Since they were going to ship a finished product, he was probably able to build it much faster without having to worry about code quality.
https://www.engadget.com/2012/08/14/goldeneye-007s-multiplay...
Maybe an appropriate way to look at this is that the developer working on the feature could only need the company to take $100 in debt, while another developer needs the company to take on $100,000 in debt for the same work. Typically this difference, unfortunately, does not reflect in the individual developers' salaries.
So, in the context of oldschool console video games that ship and then end development permanently, it's absolutely a reasonable decision to add some technical debt if it means fitting one more nice-to-have feature in before launch. Because you don't pay interest or pay down that debt ever. The code is shipped and then it retires, presumably.
Technical Debt is about taking shortcuts to have something working now, in exchange for more work maintaining and evolving it in the future. If the software doesn't work, it's not debt, it's a bug.
If you are not changing the software, it doesn't matter that a list is hardcoded instead of parameterizable, that the modules are tightly coupled and can't be reused, or that the build must be done following a particular set of incantations in a specific machine. If you are, these are all examples of debt that will come back charging interest.
Properly planned and managed, technical debt is a tool much like actual debt (a loan) that can accelerate your growth and your time to market. Thing is, most teams don't know that they are taking on a debt and don't realize how much interest they are paying each day.
Given what you knew at the time you created a definition, and per that definition you could have written 100% maintainable 100% bug-free 100% complete code. It's in uncovering your lack of/ incomplete understanding of your problem that you may need to go back in to that code. Even though at the time it was considered 100% bug-free, complete, and maintainable.
This is the type of choice you need to make upfront often when designing software. If you get it right your system is more maintainable. If you get it wrong your system is less maintainable. Sometimes you don't know in advance which is right.
(I'm guessing the biggest factor is the massive investment required to make a game nowadays.)
In the classical model of ship-once gamedev, certain big-ticket software features tend to be thrown out when the designers are in control, because they're so costly to get right and don't surface as much immediate impact:
* Large-scale persistence(just store a minimal representation and instance that)
* Online features(too much synchronization logic - it hinders almost every other feature!)
* Complex AI and procedural simulation(too much chaotic behavior, limited ability to control outcomes)
The transition we've made has, in essence, been to construct microtransaction service products that do support these more complex features: the game is a platform for the sale of assets, and it is massively online with total persistence and sometimes even robust AI. And when you start doing that, your ability to revise the design diminishes substantially. For the most part, you can't take away assets and features that are already there without a substantial player backlash, which means that games of today tend towards accumulating their way into an incoherent maximalist experience with no particular focus or intended playstyle, all of it following templates intended to encourage monetization:
Make sure the players can customize their character cosmetically, and give them a huge array of skills and powers, but nerf all the unique aspects of these powers so that the game will play roughly the same regardless of what you choose. Present them with opportunities to draw in their friends, and add hooks and incentives to come back repeatedly and to check the in-game store for deals. Provide play opportunities for all segments and demographics of players(following the lines of the Bartle player types).
When this model works, it does extremely well in the mainstream(Fortnite basically ticks all the boxes and is outstanding in its ability to progress as a live service) - but it has the downside of being a compromise for everyone. It makes the game too Byzantine in its complexity, too restrictive in its customization, too carefree in its changes to competitive dynamics, too willing to let annoying minor bugs persist - to say nothing of the cost and complexity needed to pull it off.
So I think there's definitely room for more of a spectrum of products, from the mega-blockbuster down to tiny boutique indie games. But the marketplace is still catching up with the idea of providing full service, and a lot of games are still released with the focus being on developing a franchise or on addressing a specific genre. In many cases the capital requirements needed to do services crush that idea out of the gate.
For example, one of my favorite games is Celeste, a fully offline, single player platformer that's about 18 months old. They've had a lot of releases[1], and at least one of them sounded like, to me, like a decision that was made purely because it was easier to implement technically. (I won't get into it here because it's pretty off-topic, but if anyone is interested, I can elaborate. It's "The Prologue Dash cutscene is skippable") There's also one more major content update coming that appears to have new mechanics, and it's already late. That said the game is amazing and I don't fault them for any of that at all...
https://www.reddit.com/r/EnterTheGungeon/comments/9ykpgc/onc...
In 1982: all games were written from scratch. You might copy the good parts from a previous game, but there generally wasn't much: poor abstractions meant the game logic and UI was mixed together (in their defense, on the limited CPUs of the day you couldn't have made proper code separation work, and of course the limited CPUs also meant you couldn't create such complex systems where it mattered.)
As games entered the mid 90s games started to get releases, and game engines came out. However each release was treated like an all new project even if some code was reused. Games would customize the engine in ways that wouldn't apply to the next game. Bugs in the engine would be worked around by ensuring that part of the game could not be reached (thus resulting in games that you could never get the 100% complete stat as the engine knew you didn't go through some locked door without a key). Some of the core engine was reusable, but for the most part the engine and game were tested as one.
Sports games tended to be the first to figure this out: every year was a new release of essentially the same features, but different player stats. As such it is becomes obvious quickly the game engine maintenance is a cost, there were few new features (until the next console offered better graphics). Most other games were latter because part of the new game was engine changes to support it, so it was hard to find a stable core to the engine.
Gradually games have seen more and more value from separating the engine from the game. As this has happened engine changes are no longer for one game, but for the possible use by all future games. Part of this is game designers have figured out the right abstractions, things that previously would have looked unrelated and thus change different areas of the engine are now seen as variations on a theme if you squint just right - and we know how to squint
Of course there have always been, and probably always will be one-off games written like 1982. When your game is small this is okay.
Different game companies have reached this at different times. Infocom had mostly figured out their reusable engine in 1982.
Technical debt doesn't care about your release status and can kick in long before you reach the finish line, which I suspect happened with Perfect Dark and many subsequent rare games. How many game delays and cancellations have technical debt as the root cause?
In enterprise projects I've seen enough over engineered crap added at the very start of the project that hampered development for the entire lifecycle.
Now, if you're working on open source and will be in this codebase for a decade, or this is your company or your software, it makes the tradeoffs much clearer, and that works because your incentives are right. So this metaphor works better for helping the typical developer make choices well, than for helping the typical manager/director/CEO make choices well.
Which does happen, of course, but not always, and not almost always either.
Also, the metaphor starts to fall apart if you consider good debt vs bad debt in terms of financial investments. No examples of good technical debt come to mind, other than the benefit of shipping quickly.
I think the article somewhat addresses this where it talks about not needing to pay off debt in areas where it doesn't need to be changed, so perhaps it's a valid argument that if it's "working now" to leave it alone. As for the hard challenge of when to pay off (or accrue) technical debt, it really comes down to individual judgements from an experienced technical leader who has a high level and long term frame of reference. The focus should ultimately be on optimizing delivering overall value to the company or mission. One thing to avoid is to have non technical managers making decisions about technical debt, since it will likely be an uninformed decision.
Consider a financial instrument that is interest only on a perpetual timeline, and you are not required to pay down the principle. If the cost of paying the interest forever is still less than paying down the principle, why wouldn't you just keep paying interest? Like you said, it takes an experienced, competent leader to make a good judgement about what the true costs are either way.
A pretty good talk from Railsconf a few years back. The debt metaphor can be thought of as two axis: planned vs unplanned, and low vs high interest.
If you explain to the stakeholder that by not paying down technical debt, they're only paying off the interest, but they have to keep paying it forever, perhaps that will sink in. The opportunity cost of those interest payments is profit-generating work. By failing to address technical debt they are constraining the business.
Technical Debt is something for which you have a repayment plan and a timeline on which it will be repayed and you are making the repayments.
Technical Tax is everything else. And your tax rate will keep climbing higher and higher unless you make a change at the policy level.
I think that's where the metaphor works best. Good technical debt lets you ship sooner than you otherwise would have, similar to how a mortgage lets you afford a home sooner than you otherwise would have. Paying interest isn't ideal, but it may work out a lot better than renting. Likewise, code cruft isn't ideal, but shipping 6 months sooner might pay off a lot more than having a perfect code base can.
Over the right time scale, technical debt can be a huge win. That's "good debt".
Consider the race against time - either time to market (get a product out the door asap, for valid business reasons), or because it's currently in proof of concept stage and will either be replaced or fixed later, once it becomes a real project destined for production. In either case, technical debt is a good deal. Even if it costs you more in the long run, the time saved in the short run can be the difference between success and failure.
These are the justifications that lead to Technical Debt.
Unless you write a self destruct system that prevents the software from starting after a certain date... there is no such thing as 'good debt'.
Projects come from POCs, they get moved into Production and are never replaced, just patched.
Unfortunately every project I have been on has very little of this. The real problem I see are not intentional decisions, but "If only I had known". The team that choose a framework and 3/4ths the way though the framework company dropped the product (I know a product shipping on WinCE 6 - MS dropped support for that on them 1 week before code freeze). The team that started on some new fad that seemed good only to be the first to find the issues when used in their situation (the fad is overall good, but the limits are not understood).
Don't burn the village in order to save it.
Technical debt is good if what you can create quickly and dirty actually improves your productivity. That is what tools can do.
In the real world if I incur in debt and my debt pays for a tool like a car-truck that lets me move myself and my things faster, I am improving my productivity. Sometimes tens of times.
Think of the difference between a sewing machine and hand sewing. Hundreds of times slower.
I can pay back my debt faster with good debt.
In software, if I can make a tool that in a quickly and dirty way is able to do things in seconds that take me hours and fatigue me, then it makes sense.
Sometimes you can even use your quick and dirty tool in order to clean the dirtiness of your own tool way faster. That is what bootstrapping is all about.
That is what compiler writers have always done. They started in assembler, then created the core of the language, and then they used their own language in order to improve the compiler.
Other writers used languages like Lisp with super slow compilation, most of the time interpreted, in order to create the core of the language in days instead of years.
You don't see the benefit of shipping quickly. For me it is one of the most important things in the world. People's salaries cost more in direct proportion of the time it takes to do something.
I think you need to answer why shipping quickly would benefit the company. Shipping quickly is not an end but a means to an end. The point is the state of the market and the impact you plan to have on it, with an added constraint of some deadline.
Perhaps it’s an AR app a company wants to ship in time for a sporting event or art show, and the business has only enough runway to get to a potential windfall of revenue from it?
That being said I think for tech debt to be “good”, the real challenge must be well understood, as well as the alternatives, it must be documented somewhere and all stakeholders must be aware of it, because I’m going to force them to consider it in every prioritization meeting until it’s done or not needed any more.
Personally, if I have a client that’s constantly questioning my expert analysis, the thing for which they entered into a contract with me, I would question whether that relationship should continue. Seeking info is fine and expected; shutting it down is not.
For example after a few months of no refactoring, you start adding 10% to all estimations based on technical debt. It grows more the more refactoring is kept at bay. Similar to how we add a certain percentage to each task for communication, bug tracking, qa, scope changes or anything else. You can keep an arbitrary count, and perhaps even have a "technical debt" reduction amount for certain backlog tasks.
I haven't ever tried this though. But it's an interesting way to communicate technical debt (and is very close to the metaphor).
https://medium.com/@brlewis/fighting-technical-debt-in-an-ag...
It's the same as buying insurance, it reduces risk and exposure in exchange for money. Ask management if they want insurance or risk and cash. You pay money to reduce risk. It's a decision they are capable of making when put into non technical terms. They assume the risk of catastrophic failure if they dont buy insurance. (Obviously major refactoring has its own risks.)
After much grumbling from the devs the CEO prepared a presentation where he told us technical debt was our friend. We had a choice to pay down the debt, or add new features. New feature would get us more customers, and more income, which would mean we could hire more developers to eventually pay down the debt. I felt it was a very wishfull thinking type of argument. We were having increasing numbers of outages due to the lack of a proper testing process in our CI pipeline and our DB was massively overloaded. Adding new feature was literally adding to our technical debt and increasing our risk of outages, something that wouldn't help get new customers.
My issue with the above argument is that the problem with debt isn't the amount or type, but the person holding the debt. A 30-something in a safe job (e.g. doctor) earning €100k a year can safely hold €300k of debt, so long as they are responsible and always strive to reduce the amount owed.
A 20 year old working in a fast food joint on minimum wage, but holding €50k of debt is in serious danger. They aren't in a stable job and they aren't being responsible.
Some companies are like the doctor. They have debt, but are responsible and actively work to reduce it. Some are like the 20 year old who thinks that they can deal with it later, oblivious to the rising interest and danger they are putting themselves in.
I prefer to see engineering as a series of trade-offs. These trade-offs come about often as not having enough information about future scenarios and yes, sometimes they come about because you are in a rush to do something. Either way, whether something should be redone is dependent upon a lot of factors and calling something "technical debt" is a sort of way to politicize work as though it is "the right thing to do" and msut be done. This feels too simplistic to me.
Often, shortcuts we take will live for a long time in our code and often it doesn't matter that it's "wrong". If we call something technical debt, we should only call it that at the time that we determine work definitely needs to be done, not as we're going along and thinking "this is not the ideal design" as this will just lead to YAGNI designs.
Debt only becomes a problem when you take out too much of it. If you owe 90% of your monthly income in interest payments, it'll take a long time to a buy a new car. If you have nothing but hacks/shortcuts/etc in your code, it'll take a long time to add new features.
Debt is only debt when you have to service it. Shortcuts that aren't affecting your ability to develop new features are not debt.
You don't have to pay other debts either, you just have to be prepare to deal with consequences of that (bad credit scores, bankruptcy). Real debt too you often have choices in what you pay off today, versus what can wait for next month or next year.
There's also nothing inherently wrong with taking on debt. Most people will have all sorts of debt in their lifetime, and some economists believe that debt is much more interesting economically than wealth ever is. Debt is not a moral proposition and calling something "technical debt" isn't saying that it is bad/wrong, it's saying that we're writing IOUs we may or may not have to cash at some point. It isn't necessary to avoid debt at all costs because it isn't wrong to have some debt on the books (beyond very conservative religious readings that feel that all debt is sinful, of course).
Tech debt mainly becomes an issue when making changes to the code. If no changes are needed in the code and it is working correctly, it is best to leave the debt in there.
If you miss a deadline because adding a feature causes to many issues: you may have defaulted.
This becomes a compounding problem, where the technical debt gets so great that it becomes more appealing to just keep pushing out features and dealing with the fallout. Everywhere I've worked, the code has ended up exactly this yucky. And yes I've contributed to it too. I just don't think that the idea of paying for someone to go back and make the code nicer with no tangible changes to the application's external behavior doesn't seem to resonate with my CTOs. Maybe I just need to find better places to work.
This is where static typing can help a _lot_ in my experience. Your types (and your function signatures) are contracts you still need to adhere to after you're done refactoring. Keep making changes until the compiler tells you it's all good and then usually it is.
If you think of a new requirement, it's easier to be sure that it doesn't conflict with six requirements than with sixty, or six hundred.
Additionally, requirements are hard to remove/change once their implementations are deployed and depended on.
Basically we are already saddled with debt at the requirement level, before we even consider the implementation quality.
Thus, the first tool in the fight against technical debt is to very carefully manage the acceptance of new requirements. If a requirement doesn't exist, then its code doesn't exist, and that's as clean and maintainable as code can possibly be.
Secondly, try to gather all of the requirements up-front before implementing anything, and then resist futher requirement creep. If most of the requirements in a system were gradually introduced after implementation began, that will tend to degrade the quality.
There are cases where you need to consider technical debt, but most often it should be considered as something you create rather than something you inherit- because the creation of the technical debt is where you're making the judgement:
* Is delivering now important
* Is this code going to need to adapt and develop in the future
* Is the code this is building upon going to stick around or will this code likely be superseded.
* (Selfish developer: Am I going to be around to deal with this)
A lot of people are shoving all technical debt in the Deliberate / Prudent category. However I've seen a lot of code bases written by inexperienced developers and it would have be done completely differently if they had any experience (both of the inadvertent categories). This means actually updating the code base can be a complete pain. Sometimes you know if you're editing the code you risk creating huge regression issues and suddenly something that should have taken a short time can spiral out control if you want to add more functionality.
[0] https://martinfowler.com/bliki/TechnicalDebtQuadrant.html
When you take a debt it's usually to finance something. Focusing the wording on debt highlights the negative side of the trade off. It's like we're suddenly all turned fiscal austerity hawks, where any bit of debt is evil.
I feel like developers are each placed somewhere on a continuum from purist to entrepreneurs. Purists can spend weeks to tweak an SQL query, while entrepreneurs would be happy with a no-code Zapier MVP that actually lands customers
It's not to say that one is always better than the other. A balanced team probably needs both. In our company, I'm more on the entrepreneur side (which I guess is good for launching a startup) but the recent addition of a "purist" to the team has improved our codebase significantly.
Obviously with experience you can make decisions that can lower the volume of technical debt once you reach that breakpoint, but in my view the right play is always to get to that moment as efficiently as possible and that never involves re-architecting or rewriting until then.
Technical debt is the same way, and I often talk about it that way at work. But it's a phrase that means something that "business investment" doesn't. When I tell my boss that we're incurring technical debt (and what exactly that debt is), he is already in the "business investment" mindset on the matter. I'm the one trying to reign that in and focus on making the future easier by avoiding some of that debt now. But I fully understand both sides of the matter.
The reality of most code, especially as it gets closer and closer to the end user, is that over time there will be requirements added and removed. It is really hard to perfectly design such a system from the start.
Thus some level of technical debt is inevitable and it isn't really debt. For instance, an overriding goal of zero technical debt is not always the right decision. You may be expecting additional requirements and have not come up with a better conceptual model.
This focus on the technical makes you lose sight of underlying issues which are often organizational rather than technical. Is someone neglecting the code intentionally to get a promotion for a rewrite? Is there a ship now or feature mentality? Are your engineers empowered to change the code?
Other industries and systems have the same issues. What do they call this?
This can happen because the product roadmap changes (new PM, business changes, etc...)
Try writing UI for several different designers over the course of several years for several different iterations of features when there is no common design language.
Lets say I had a warehouse that stored food, all stored neatly in its own section, each with the least frequently moved at the top where I need a forklift to access, and the most frequently used at the bottom where they are easy to access.
Someone orders too many boxes of rice, and there's no room in the rice section, but there's room on the lower shelves of the sauces so it gets stored there, and some are left on the floor, partially blocking forklift access.
The immediate problem of rice storage is solved, but every time I need sauces I have to grab the forklift, and it may take longer to get there and back because the route is blocked. This will persist until I refactor the warehouse so that there is more space to store the rice appropriately.
Now the manager decides we should be able to store refrigerated goods, and I have no refrigerators yet. The work to refactor the shelves to make room for the refrigerators is a conceptual change in how I use the warehouse, and not a direct consequence of how I've previously been storing goods. It's not tech debt.
From experience, a lot of people/companies have difficulties marketing what/how/why/when technical debt should be tackled for both internal and external stakeholders. This leads to a less than optimal structure where "new" or "shiny" work is preferable and rewarded, as opposed to a more balanced approach.
Tech debt are all the factors that erode productivity. This includes many technical factors from inefficient systems design to unnecessary testing. It also includes slowness from weak developers and developer training.
I know it puts me in an extreme minority, but I am a big fan of extreme simplicity, which is not what it might suggest. Extreme simplicity removes everything in a system present for the easiness of the developer so that all that is left is only that which is necessary to accomplish a business requirement as directly as possible. If there is less to read (including dependencies) there is less to maintain and less to test. Simplicity is not easy.
The confusion is that a system must achieve a certain level of easiness or it will be rejected by some developers outright or require training at great expense. My basic premise for making any challenging decision is to never compromise on integrity, which includes being honest about the challenge at hand. You can embrace that challenge directly and solve for it or you can hide from it behind layers of easiness.
E.g. changing some business logic can be 10 minutes of work to implement a hack (which will pass tests, mind you), or 2 weeks of strenuous yak shaving for a "principled" solution. Which would you rather pick? And more importantly, are you prepared to select for developers who favor 10 minute hacks to principled solutions?
How do you define a principled solution versus a hack? All that matters is whether the result achieves the desired business goals without regression, and that is what tests are for. That being said I would gladly pick the 10 minute solution that passes all the tests.
So much of development is bullshit posturing for developers to justify their existence with unnecessary tasks to make things easier for themselves.
>> So much of development is bullshit posturing for developers to justify their existence
That applies to any profession. Developers aren't unique in this regard.
Isn't it obvious how to do things in programming? Many developers, in my corporate experience, occupy themselves with how to do things in code. That is exceedingly unfortunate. How to write code should be an obvious disqualifier before obtaining employment. Nobody would hire a lawyer, for instance, who didn't know how to a legal argument.
Writing code should be as obvious as writing an essay, but I suspect many developers that are hiring probably have trouble writing essays as well.
Anyways, if you had the choice between a 10 minute pizza and a 3 day pizza with ingredients more evenly spaced apart which would you choose?
And you will hardly hire a lawyer to work in unknown to him legal domain in foreign country in foreigh language.
That is why the software gods invented test automation.
> paying interest on it is
That is why simplicity is important, because less is more. All code is ultimately debt as it demands some amount of maintenance. When there is less to maintain there is ultimately less debt.
> And you will hardly hire a lawyer to work in unknown to him legal domain in foreign country in foreigh language.
Has that ever happened to you as a software developer? The one time I was told to learn a new language I was allowed the time and space to learn it.
Not in my field, no. But I might be an oddity. I get hired to do the kinds of things others have tried and failed to do. So while they may be "productive" by whatever arbitrary metric (solving small problems real quick), they fail at gnarly, "deep" shit which can't be done in a day, or, sometimes, in a week, but which does need to be done.
One typical example in real world is React Hook.
By moving boring stuff into a folder called "hooks", we centralize concerns into its own file, so that we could manage them effectively, or we don't need to think about it anymore.
I've had to restrain developers from "turd polishing" systems that were poorly thought out to begin with. Code gardening in a subsystem that was simply the algorithmically wrong way of doing it (that we might have had to have shipped quickly in order to stay afloat) isn't a first class activity. The problem is with the metaphor of technical 'debt' is that implies a balance sheet, but many (most?) of these bills will never come due.
But there are definitely times that you feel certain it's going to end up being technical debt. When you implement a system that doesn't scale, but you know it has to eventually scale massively, that's going to be technical debt.
You could write it right now to handle that load, but it'd take a lot of time and money. Or you could write it the quick and dirty way, and get money coming in immediately and put off the scaling until it's actually necessary. You know it's coming. But you decided to incur that debt in order to make money sooner.
IME, technical debt is usually about time or money, and getting something done cheaply now and worrying about the full cost later when it's more applicable.
It's a tool, not a problem, so long as you're managing it and not just blindly choosing to do things the quick and dirty way all the time.
If you know a piece of crufty code is going to be irrelevant in the near term after some major architectural change or new feature, it isn't really worth it to pay down the principal with minor improvements or shims.
As a workaround for the time sensitive issue we literally copied the old code and overloaded our class so that the customer runs the old code while we develop a robust fix.
This should be managed carefully to prevent a lot of independent gradual improvements that don't follow a clear architectural pattern and end up conflicting with each other.
Furthermore, all code changes to existing modules require validation of the affected functionalities, which are often hard to spot. Hence, I'd say that a wide scale refactoring effort needs a clear testing plan before it's put into practice.
Frequently, efficient methods or parallelization are not used and the least creative method is implemented to solve a scientific question. Functions are not written in a modular way but merely for a “quick and dirty” analysis.
The end result is that you spend more time getting things to run or waiting for them to run than if you had gone back and reworked the code to begin with. It’s a serious drag on productivity.
Any exception to that?
You can either take the debt and try to increase your business revenue more than the interest of the debt.
The same is true for the technical debt,or decision not to modernize / write shitty solutions to fix something quickly.
In real world any business man would ask before taking the debt: is there a chance paying it back.
But no one asks about paying technical debt back,or if ignoring it will yield a high enough return ..
"Managing Technical Debt: Reducing Friction in Software Development (Sei Series in Software Engineering)"
https://www.amazon.com/Managing-Technical-Debt-Development-E...
I remembered using PhpWiki and being able to auto-link pages just by using camel case. I don’t believe MediaWiki has that capability. And it’s such a standard I had forgotten it even existed.
Something very soothing about the idea that links could be handled so gracefully.
A feature implemented through "indebted" code tends to be buggier and less performant. There are exceptions, of course, but by-and-large this has been my experience.
So the "interest" is not repaid by just the programmers, but also by the users, over and over again as they use the feature.
Already posted on HN here : https://news.ycombinator.com/item?id=19353352
I don't think it is that simple, because the attempt to remove cruft might introduce new one and can make it even harder to reason about the code.
In reality a lot of technical debt can never be paid back without restarting from scratch, and a lot of it is accumulated unknowingly only to be discovered way later in the project.
I’m having trouble thinking of an example of financial debt that can be taken on unknowingly. Maybe identity theft? But that's less like an engineering tradeoff... more like NSA backdoor type stuff.
Whatever it is, I think most projects deal with "technical debt" accumulated unknowingly. There's no theory around how to create a "good design" and thus it's often shooting in the dark or using preexisting patterns. Most software projects have design flaws that only become evident later. In my career, I have dealt with this type of problem more-so then debt accumulated knowingly. Additionally my current company is calling this type of debt, technical debt. I have rarely rarely seen technical debt taken on knowingly and documented.
I don't know what kind of software you work on, but I disagree. It's incredibly common, from what I've worked on, that we have an existing subsystem X, and project management wants it to interact with this new subsystem Y but it was never designed in a way to do so, so either we can hack it in real quick, build a moderately pleasing integration with only a little technical debt, or completely rebuild X from the ground up in a way that makes sense and leaves no technical debt. From my experience, bad managers will want the quick hack while engineers and good managers will take the middle option. Nobody opts for the complete rebuild.
A good way to test if your subsystem is modular is to ask yourself, can you pull your subsystem out of the current system and use it in a completely different app in a completely different context? If the answer is no, then likely the module is either IO or too tightly coupled with other systems. Too much Mocking in your tests is another sign of a design flaw. It's a signal that your code is dependent on other systems rather than a module that can be pulled out and reinserted somewhere else.
Another design flaw is, is system Y itself made up of modules or interconnected dependencies? If subsystem Y needs to reconfigured to do something slightly different can I pull out 10% of the system and replace it with new modules? Or does pulling out 10% of the system involve pulling out another 80% of the system as a dependency? You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.
Most programmers choose dependencies because modules are harder. Also the dependency style of programming is heavily promoted. Additionally, most programmers are Unaware of what it means to make a modular component, they organize code following nomenclature rather than a systematic theory, and they think they are doing modular design but they are not.
Likely your subsystem Y was designed in such a way that it is tightly coupled to another subsystem and can't be used outside of a certain context. Possibly it is tightly coupled with IO or some specific database schema. Likely system Y itself is made up of modules that are interdependent. Also likely it was programmed by someone who did all of this unknowingly.
So it could be a matter of perspective. What you see as debt taken on deliberately I see as debt from the past "discovered" and interest accrued because the debt can't be paid back without a rebuild.