All code is technical debt
tokyodev.com
tokyodev.com
I think "technical debt" as a term is overused and sometimes misapplied, but the core idea resonates with many developers, at least the way it is most often used - which isn't exactly the original proposal as another commentpoints out. The key part of it is the analogy with an interest rate. I can decide do put 5k on the credit card for a last minute holiday, but for as long as I don't pay that off, I'm paying 20-something percent, every month, forever. Similarly for example a rush to add features to hit a demo deadline can introduce brittleness and abstraction failures into your codebase that you pay for every time you make changes, add features, do nearly anything. The analogy is that some of your development is going to be wasted "servicing the debt" going forward until you fix that (pay off the debt). That doesn't mean its the wrong move, for the same reasons that taking on debt can be the right move for a person or business.
Then the equation is simpler. More code = more to maintain = more liability. It says nothing about the quality of the code, just the quantity.
Tech debt is a trade-off between near term and long term ambitions - it is literally impossible to build a successful project without accruing this kind of debt, simply because you cannot perfectly anticipate the future. In that sense, it's not so much debt as it is a maintenance overhead.
The ideal software implementing a given functionality has as low of a carrying cost as possible. More code, in quantity and complexity, typically means more carrying costs.
One of the appeals of SAAS is that this carrying cost is easy to quantify and predict. Going further, for high-quality OSS projects like the Linux Kernel, the carrying cost may be 0 or close to 0 for most users. But lower quality OSS or buggy SAAS has a carrying cost in the form of bugs (possibly affecting downstream users), patches, need for support + back and forth.
In this analogy tech debt is a tradeoff for some short term benefit (like simplicity, time to market) at the expense of higher carrying costs down the line. But some carrying costs simply can’t be reduced - you can’t mooch off something like the Linux kernel, get your own carrying costs all the way to 0, or find a SAAS provider/vendor willing to give you the software for free. So in this model not all code is technical debt.
The under-appreciated corollary to that: tests are code, hence tests have non-zero carrying cost as well.
> The ideal software implementing a given functionality has as low of a carrying cost as possible. More code, in quantity and complexity, typically means more carrying costs.
You understress the complexity point. Higher quantity of code with high clarity and low complexity would have lower carrying cost. This is where DRY bites people: ill-suited or excessive abstractions can be worse than a little bit of copy-paste that are immediately visible to the eye.
> In this analogy tech debt is a tradeoff for some short term benefit (like simplicity, time to market) at the expense of higher carrying costs down the line.
In a general sense, doesn’t this hold for everything we do? If you really dig into it, is there an activity that is not a trade-off of short-term benefit for higher costs down the line (be it waste, pollution, something unknown or generally entropy)?
Also: nothing prevents us from deleting tests and do as if they never existed, except the nagging voice of sunken cost.
I've worked in so many places where a test suite suddenly takes a day to run with no parallelism, and requires 50 other machines to get it down to 20 minutes. And so many of the tests that waste so much time are down to them being pointless, or being stateful for no reason.
I constantly advise people to stop writing obvious bullshit tests (e.g. the kind of test in Rails where you just assert that you wrote `belongs_to :some_model` in your model), or ones that are too heavily mocked. Focus on your business logic and the inputs and intended outputs, not how you used a framework or dependency.
-- Dijkstra in 1998 (https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E...)
For similar sentiments consider "Science and the Compulsive Programmer" from 1976.
https://www.sac.edu/AcademicProgs/Business/ComputerScience/P...
"Programming systems can, of course, be built without plan and without knowledge, let alone understanding, of the deep structural issues involved, just as houses, cities, systems of dams, and national economic policies can be similarly hacked together. As a system so constructed begins to get large, however, it also becomes increasingly unstable. When one of its subsystems fails in an unanticipated way, it may be patched until the manifest trouble disappears. But since there is no general theory of the whole system, the system itself can be only a more or less chaotic aggregate of subsystems whose influence on one another’s behavior is discoverable only piecemeal and by experiment. The hacker spends part of his time at the console piling new subsystems onto the structure he has already built—he calls them “new features”—and the rest of his time in attempts to account for the way in which substructures already in place misbehave. That is what he and the computer converse about."
Even though I don't use any of his techniques, Dijkstra's "A Discipline of Programming" did affect the way I see things considerably.
A car or a house also have ongoing maintenance costs, but they would normally be classified as assets (because they generally produce value in excess of the maintenance, or because you can sell them in a secondary market).
I think an accurate, useful, but not very catchy rephrasing is just 'working code requires ongoing maintenance'.
I don't love this either because working code needs no maintenance whatsoever. After all it's already working. There's also no notion of "repairing" code because it isn't something that can just break suddenly.
So really what we're talking about is that necessary changes to the codebase can arise from not just the stakeholders as we typically define them but also because the environment around the code changes.
Someone who invested in shutters that can close instead of being purely decorative will be able to adapt more quickly and cheaply to a wind storm than someone who has to go out and buy wood to board them up.
And working code, no matter how well it works, imposes a cost to implementing new features. A new feature might be a cinch to implement in a new app. But it can easily take much longer to implement the same feature in an existing app.
Not all code. You have platform dependent code and then code that doesn't depend on anything outside of your own code. The code with no outside dependencies can easily stay around unchanged for decades and is not a liability.
"All dependencies are liabilities" I could agree with. Dependencies are technical debt that will continue to demand interest from you, they are often worth it but they are the main source of maintenance burden. Some are unavoidable if you need to interface with an API, but many are not.
That said, I agree that it's helpful to think of dependencies in terms of their potential liability as well as their benefit.
If you have a viable solution to a problem that doesn't require code, then you still get the value from that solution without the burden of maintenance.
Classic case in point: your in-house infra platform on K8S is an incredible liability no matter how much value it adds, because it's a means to an end for scaling and serving your product. Or a cost-centre, in other terms. You could just as well host on a collection of SaaS and offload a large portion of the maintenance cost onto the provider.
Rather than writing code that is easy to maintain (read: easy to keep), it's optimal to write code that is easy to discard. The latter is a lot more challenging than piling on architecture and in some sense it's akin to having illiquid code (stuff you can't get rid of very easily), and liquid code (stuff you can get rid of easily).
Here you're saying code is a liability because it has costs. But a car or a house have costs, yet they're almost always classified as assets, because the benefits outweigh the costs.
So you're left with a few choices:
1. Accept that code isn't always a liability, at least according to the dictionary definition of the word
2. Argue your way somehow to code nearly always being a net cost (seems unlikely, otherwise why write code?).
3. Redefine the meaning liability for this particular application
I think 1) is the most reasonable choice here.
Different domains have different jargon. Code doesn’t map very well to finance because the loan is also the productive asset. It’s easy for a deprecated function to exist which literally nothing calls and therefore it provides you zero benefits, but such cruft is still an ongoing cost. Meanwhile that exact same function was critical last week.
An illustrative example where more code is a lesser liability: I work on a product with an activity feed feature built in 2018-19. There was a political battle on whether to build it internally or use a third party "feed as a service" product. The build it internally folks won. The "FaaS" under consideration went under during the pandemic, so we dodged that bullet. We also have customized it in ways not possible with an off the shelf product, with no big rewrite necessary.
On the flipside, we've never had time to implement things that are trivial with commercial FaaS like iOS and Android push notifications. Trade-offs are hard!
No - runnable programs are an asset.
The code to produce those runnable programs is a liability.
It's a necessary liability for a sustainable software business, as without the code fixing bugs is nigh-impossible (though not theoretically impossible - long-ago programmers wrote and read raw machine code).
But, the code itself is not what produces value. That's the code's output.
(Yes, you could be clever and say "but what about interpreted languages?"
I'd respond by pointing out that it's pretty rare to distribute a pile of executable code files to end users. Front-end JS only producing UX if delivered as part of a website, and similar principles hold for other interpreted languages - they're effectively object files once deployed, as in any deployment they're being run by the interpreter, not read by a human.)
> The code to produce those runnable programs is a liability.
That is total nonsense. So you throw away all your code after you built the program binary to avoid the liability? Don't think so, that code is a massive asset and throwing it away would make continuing the project unfeasible most of the time.
> It's a necessary liability for a sustainable software business, as without the code fixing bugs is nigh-impossible
Ok, so you agree with me, the code is a massive asset since it is the code that lets you change the program. Code is there to help programmers do their work, that is valuable as you say.
> But, the code itself is not what produces value. That's the code's output.
Both produces value. The code is valuable since it makes it easier to inspect and change the program. That is value. It isn't uncommon to sell source code access for products, why would anyone pay for that if seeing the source code is a liability? No, of course it is an asset, people arguing otherwise have just gotten too hooked on the "code is bad" mantra and try to argue all code is bad even though code obviously is very valuable.
Edit: And to be even more clear, throwing away the code and trying to work directly with a program binary creates technical debt, that is way worse debt than keeping the source code around. Code isn't inherently technical debt.
You're certainly right that when a business's customers include programmers who want to extend or modify the programs, the source code can be an asset. I was mistaken to say categorically that code's never an asset.
On average, though, I think it's still more often a liability on the balance sheet.
It's why we lionize programmers who cut out thousands of lines without removing any features - they reduced the liabilities while retaining all the assets.
Coding & story-writing is a good analogy. If you are inserting a new scene into a large anthology, you need to make sure the new scene fits with everything else. OTOH, writing a brand new story does not have this constraint / cost.
Which is all to say, software is an asset, code is a cost. The functionality of software is the valuable thing, all the code to do that is overhead. Thus, every line of code has a cost, and an ongoing cost that is forever then payed (which is exactly the interest; simply taking the time to scan/read over a line, to scan between 5 files vs 20, those are all ongoing costs)
Rushing out features with known bugs, poor architecture, no test and no documentation is akin to buying a house with a 20+% loan. You'll get your house sooner but you'll spend most of your resources servicing that debt until you can pay it down.
The idea that "all code is technical debt" is saying that all code has a non-zero interest rate. In popular parlance though, we don't actually call code technical debt unless we think the interest rate is unnecessarily high.
To try and be more practical, logging is a good example. For some, cleaning up logs once every month or so is just a cost of doing business, it is not avoidable. Similarly, just adding a "grep -v DEBUG" is also just a cost of doing business and watching the logs. Someone else might look at this and call it technical debt, the lack of logrotate & overly verbose logging make all of the above unnecessary.
There are just lots of dimensions to it. To one person, having those DEBUG logs might have been the difference in finding a problem vs not, yet to another those DEBUG logs may never be useful and all they do is create noise. The interest rate we assign to code is incredibly subjective, depending on both task and person.
Another response gives the construction analogy (I try to avoid construction analogies for software building, because it's a bad analogy.. but I would digress - story writing & editing I think is a much stronger analogy for software development).
Let's say there exists two townhomes side-by-side (attached building). Let's say you need to replace one. The cost to replace one is a function of its attachments to adjacent buildings and overall build quality. If that other building were not there, all costs associated with compability would be zero. We can draw this analogy much further though, let's say to inspect the electrical lines - you need to remove half of the dry-wall of the existing building that you do not want to even touch. Had the electrical lines been put in correctly, that re-do and re-painting of the interior, the temporary eviction of tenants would all be unnecessary. This is the example of the overhead that comes with code. Anything that is 'attached' to something else is going to be more expensive to do anything with compared to it not being 'attached' to something else. The cost of considering and maintaining that attachment is not related to the value provided by that attachment, but instead to the quality of the attachment.
The 'debt financed' metaphor comes in when instead of doing something right, it's done badly which means anything that is attached to the 'badly done thing', needs to incur a high overhead cost to even just understand the badly done thing (every time, and whether we are even trying to modify it or not - this is high interest code). In comparison, if the system is decoupled and the APIs clean, that other thing might need only a moment of consideration instead (low interest code)
At the end of the day, it is to say - if you are replacing the 11th floor of a building, you have to consider every floor whether you want to or not. That is more expensive than if you did not have to consider those other floors. Considering the impact to 10 other floors is always more expensive than if those floors did not exist at all. The cost of maintaining those floors though is not a fixed cost, not all floors have the same maintenance cost. The maintenance cost of equivalent floors is not even the same, it's a function of the quality of those floors. Even high quality floors can be low-quality if we are trying to do something the designers did not expect. It's very context dependent, but all of which is to say that the value from features is significantly independent of the cost of the thing that delivers those features. The thing that delivers those feature might be very high cost, or it might be low - but it will have a non-zero maintenance cost. Simply reading a list of 10 items is more expensive than reading a list of 5 items. If anything, those extra "5" modules have at least that overhead cost, it's not zero. The debt financing analogy comes in to describe how high that interest rate is. Saying "code" is a cost is essentially just saying that all code has a non-zero maintenance cost (maintenance being the activity of reading, understanding, debugging, modifying, or deleting code).
Sorry for being "that guy", but I think you meant "per se".
Technical Wealth pays dividends; Tech Debt charges interest on which the payments will eventually become unwieldy.
That doesn't mean you shouldn't take on Technical Debt, like many startups do with real money you can and should go into debt in order to get your business off the ground. But just like irresponsible financial management can kill your company, so can irresponsible technical management.
The idea that "all code is technical debt" because all requirements could hypothetically change someday, makes as much sense as saying "all currencies have no value" because 1,000 years from now we might not be using any of them.
Technical debt is not defined in terms of possible future requirements, it is defined in terms of current requirements. Code that elegantly and concisely expresses current requirements is debt-free. Code that is a mix of elegantly expressing old requirements, with new requirements expressed in a hacky fast way, has debt.
But trying to judge code in reference to all possible future requirements is utterly nonsensical. You can't measure that. It's literally meaningless.
So no, all code is not technical debt. Technical debt is technical debt, and clean code is clean code. Trying to redefine words to mean their opposite helps no one. That way lies madness.
It isn't a profound thought to take a term like "technical debt" and just say well that's just everything.
Technical debt is absolutely that, adding something that incurs a level of interest, until you take the time to pay it off. In life, we often always have a level of interest to be paid.
It's a good analogy
Most of his code was actually technical debt. The code that wasn’t was external code he integrated. All of the code he wrote would eventually break down and until it did world force you to work around it, so it wouldn’t fail from interfacing with it.
In one of the standup’s i used the term technical debt to argue for fixing one of these issues that broke later but which he discouraged everyone to fix.
After that he integrated the term into his daily use to the point that it became meaningless. The people that believed this snake oil salesman thought that he had already carefully weighed his own decisions in advance.
Needless to say the issue was not fixed before he left the company and cost millions and plenty of customer headaches.
Only a couple days ago I decided to change some data structure. It was needlessly hard because I had neglected to create an accessor function and was instead accessing it directly everywhere. Had to change every bit of code that accessed the structure's fields.
That's technical debt: something the programmer should have done but for whatever reason didn't. I paid back the debt by doing it: I created the function. Now if I want to change the data structure I only have to update the function. If I hadn't paid it back, it would start accruing interest: it would be even harder in the future as more and more stuff accessed the fields.
Here, heed Ward words directly, poor programming is not debt, and debt doesn't cover "whatever reason programmer may have" but has a specific well defined scope: https://m.youtube.com/watch?v=pqeJFYwnkjE&t=198
Too often code is a proxy for complexity being added unnecessarily, consciously or unconsciously.
Premature optimization or over-engineering or burning your innovation points on shiny object syndrome can create technical debt in a different way than just taking shortcuts.
So yes, the core idea of technical debt resonates with developers, but not with business stakeholders. So if you really think the business is going to be okay with "we're gonna slow down our time-to-market and say no to all your requests for a while because we need to pay back our technical debt" (which is something I see happening over and over again), you're just keeping them out of the loop and taking over their responsibilities without actually having their accountability. Developers shouldn't decide whether time and money should be invested in time-to-market and quality or feature delivery, but that's effectively what's happening when business is not involved in the tech debt discussion. This only leads to friction and suboptimal results.
That's why I think we really need to be careful when using the words "technical debt" and not just label everything we don't really understand or think can be improved as "technical debt". Challenge yourself: what really is the problem with this piece of code, other than that we don't like it?
I think that if this is the case, you are presenting it wrong. Business types are very familiar with the tradeoffs of debt financing, you have present it as being paid in effort vs. $ (which after all, have a close relationship) and they will immediately understand that a large technical debt means you can get less desired stuff done for the same input of effort. And then you collectively decide.
Agree you have to be careful about what you present as technical debt. If you need buy in from the business side, you don't want them thinking "that's just what they say when they want more time, it doesn't mean anything". It has to be pretty concrete.
If that were literally true, then every new development would rationally necessitate a total rewrite of the codebase.
You might have meant that parts of the existing codebase would need to be changed to adapt to the new development, but as long as you've made a rational decision to keep using parts of the existing codebase, it must have been a net positive otherwise you'd just throw it all away.
This is the same for bog standard development though. Any code you add, regardless of whether it is termed technical debt or not, will make your future development slower.
Like you say, that doesn’t mean it’s the wrong move, you want to build a product after all, but I think I can see where the author comes from.
Imagine I'm trying to do a bunch of graphics stuff. The code I write to integrate a 3rd party graphics library will make my future development faster. I am effectively creating a lever I will later use to increase my productivity.
Say I need to do a bunch of linear algebra. Code I write to calculate dot products, determinants etc (or better yet to use a library to do those things) will greatly accellerate progress later and enable me to think in terms of the linear transformations I'm trying to accomplish rather than the underlying primitive operations.
No! Ward's blog post linked in this very paragraph [0] describes the original definition quite clearly, and it's not this.
The term was originally intended to describe the delta between a programmer's current understanding acquired through the process of developing the software, and the accumulated implementation which is based upon a combination of past understanding. Because writing a program is often a process of discovery and learning about the problem and solution space, it's not usually possible to write the ideal implementation up front.
It's quite clear In Ward's 2nd paragraph where he describes the correction of this debt:
> it was important to me that we accumulate the learnings we did about the application over time by modifying the program to look as if we had known what we were doing all along and to look as if it had been easy to do in Smalltalk.
What the author is describing is the modern definition that it has acquired independently, where by programmers knowingly take shortcuts... this is an entirely different phenomenon, intentionally writing code in a way that is less than ideal even based on current understanding. I find the original definition far more interesting and insightful, but the modern one is perhaps more reflective of the more common realities of the pressure developing software in a business today.
[0] https://wiki.c2.com/?WardExplainsDebtMetaphor
[edit]
Despite this error, I do agree with the sentiment of the article, that too much code is a liability, that features can cost more than they are worth.
Ward's definition of technical debt affects the cohesion and correctness of the code, the idea that if you could tear it all up every day and re-write it from scratch that it would always reflect the most up to date understanding, and for moving targets the most up to date problem, that it be absent of the weight of past misconceptions or solutions to past problems... aiming for minimal code makes achieving that state far more viable - these are complementary insights, so it's a shame the nuance of the original definition was lost.
The effect is the roughly the same though, there is an impedance mismatch between what you want to do to/with the system today, and what the system actual is today, and that has a cost. The work of reducing this impedance is real work, and can be thought of as "paying off debt".
Oh I suspect Ward's context was even more idealistic. That there was a problem to be solved, and there were few to no moving targets, the only differences over time were in the programmer's heads as they learned about the problem domain.
This is not the only factor in software development today, which is mostly a moving target (although I kinda wish it was, it's my favourite form of development, a clear goal and end state to achieve - aiming for maintenance mode on 1st release, or pretending you are getting a bunch of discs stamped - and I really like trying to break real world problems into these types of mini projects, but that's another story). Perhaps this is why it was inevitable for the phrase to take on a new meaning, and honestly maybe "debt" wasn't the best analogy anyway, your "impedance mismatch" is probably already a lot better as the sibling commenter points out.
> [...] but doesn't allow for a lot of other real-world reasons for this mismatch between the current and ideal state. The effect is the roughly the same though, there is an impedance mismatch between what you want to do to/with the system today, and what the system actual is today, and that has a cost. The work of reducing this impedance is real work, and can be thought of as "paying off debt".
The effect may be similar, but I think it's useful to distinguish the forces at play. In particular I believe conceptual mismatches as per the original definition of technical debt are catalysts of the "other" debts... The original form of technical debt is the most common attribute I notice in the vast majority of source: code is accreted, existing code, existing decisions are rarely questioned with intention beyond what is necessary to make the next requirement on the todo list function, and this tends to result in a lot of unnecessary artificial work and code to work around the exiting code base - which of course only makes it even harder to change.
I suspect it is a mistake to think that software development today is fundamentally different from how it was practised when he first wrote that. Possible application of recency bias..
This is a better conceptualization compared to the idea that tech-debt is the result of shortcuts or rushed code. Tech debt arises for a lot of reasons, including shortcuts but not exclusively because of shortcuts. I've seen tech debt arise because well-meaning engineers were trying to do things "the right way".
I think it is just a kind of entropy thing that we have to fight against. A lot of confounding factors lead to code that is hard to maintain and extend. I think it is an unfair characterization to suggest it is solely because of laziness or negligence.
I certainly don't agree with the idea that all technical debt is due to shoddy work, that's obviously silly. I suspect that people fall into it because it is easy to understand. The original framing is more nuanced and harder to think about but no less in effect.
Sometimes that's not making things flexible enough, or too flexible, or not having all the requirements captured, or making technical bets that didn't pan out.
> The explanation I gave to my boss, and this was financial software, was a financial analogy I called "the debt metaphor". And that said that if we failed to make our program align with what we then understood to be the proper way to think about our financial objects, then we were gonna continually stumble over that disagreement and that would slow us down which was like paying interest on a loan.
So indeed, his definition of technical debt is explicitly that by rushing software development now you will slow down future development.
The specific software engineering task that caused him to develop the analogy was about rewriting some code to incorporate new learnings, but the analogy itself is clearly applicable to any software engineering task that could be deferred to the future (with a cost).
Unfortunately, while managers are often happy with "it'll make page load faster" or "it'll increase uptime" as a justification for a project, and "it'll reduce bug reports" and even sometimes "it'll automate manual tasks", "it'll make development easier" can be a really hard sell. It sounds pretty abstract and I think some managers start to think that it's just an excuse. It's certainly hard to quantify the impact, especially if you're expected to do that in $/yr.
Of course if there was something wrong with the code at the time then it’s more of a “proficiency debt” (or burden) where the shortcut taken was not in the coding but in not reading the docs and practicing the craft.
While I don't think we're there yet or maybe not even close, I kind of feel like this kind of technical debt would be something that an LLM should be able to help with. Why can't I point to a piece of code and say "Look at this new code I wrote, now go and refactor the rest of the code base to align with it." Or can I? I assume if I tried to implement something that did this I would hit the context size limit and that would be it.
No error! That seems to be included in this summary, which Cunningham reviewed:
https://www.agilealliance.org/wp-content/uploads/2016/05/Int...
Quoth the PDF:
> When taking short cuts and delivering code that is not quite right for the programming task of the moment, a development team incurs Technical Debt.
Corner cutting == tech debt
Ask any ops person and none of them will tell you they're content with the state of their infrastructure.
I would assume if he misinterpreted Cunningham's original idea, would it then follow that the rest of the article is suspect considering the original premise is so far off from what Cunningham originally intended?
"Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure. It is a disciplined way to clean up code that minimizes the chances of introducing bugs. In essence when you refactor you are improving the design of the code after it has been written."
The book then goes on to describe the methodology and runs through common refactorings step-by-step. Nowadays the term is used almost generically for code improvement. I'll see commit messages like "refactored the API", which violates the edict to not alter external behavior.
Languages evolve and so on, but it's remarkable to me how quickly precisely defined terms (or even new words) change their meaning in tech. My cynical take is it's easier to get management to buy into "assuming technical debt" and addressing it with a "refactor" later than it is to "cut corners" and "rewrite" the code later. As long as everyone understands the new definition I guess there isn't even anything wrong with it, but it's a shame we've then lost a term for the original meaning.
That's the meaning of refactoring that I typically hear, tbh.
I guess it just depends on what one means by "external behavior". Changing a function signature changes the external behavior of that function for callers of that function, but not for other modules. Changing the interface of a module changes its external behavior for other modules depending on it, but not for clients of the whole server. Changing an API changes the external behavior for the client, but not for the end user (to whom API changes are invisible and irrelevant).
Traditionally, one might say "if the tests still pass, it was a refactor." Though nowadays that's probably not true either, as business code tends to contain a lot more internal testing than it used to (which is good! but also bad).
Fowler goes on to elaborate on the definition in chapter two, where he presents two context-dependent definitions (one a noun, the other a verb). The noun form is a specific type of change and the verb is the application of one or more of the nouns. E.g., "Move Method" and "Move Field" are types of refactorings. These are defined as changes made "without changing its observable behavior." To further qualify, he states "[o]nly changes to make the software easier to understand are refactorings." By way of example, he compares to performance optimization where he says: "A good contrast is performance optimization. Like refactoring, performance optimization does not usually change the behavior of a component (other than its speed); it only alters the internal structure."
Maybe it's just the domain or languages I work with. Nowadays I never see anyone apply refactorings as described in the book. I'm sure devs implicitly apply several of them, perhaps without the rigor presented, and that's understandable. But, I often see "refactoring" just as a catch-all for any code change.
While certainly there are some blurred lines in terms of what constitutes a refactoring or not, I think Fowler handles the product/API split fairly well. He adds:
"The second thing I want to highlight is that refactoring does not change the observable behavior of the software. The software still carries out the same function that it did before. Any user, whether an end user or another programmer, cannot tell things have changed."
By stating a programmer should not be able to tell things have changed, I think we can at least anchor public APIs as something that should not be adjusted in a refactoring. Such a change could improve the maintenance of the software or the quality of the product, but I don't think it fits the narrow definition of a refactoring. Admittedly, the part about the observable behavior of the software not changing can get murky. But, I think Fowler tries to scope this more at the code level than at the product level.
A white elephant needs a living are, food, elephant experts, vets, etc. That's much closer to the type of asset most code is.
It's not a big thing and there's no need for us to fall out over this .. but gold bars are famous for requiring additional ongoing security costs.
Even just burying them in the desert isn't as secure as it used to be.
You do realise that if I were to win then the five gold bars would be gone in the wind and you'd be ponying up for another five to pay out on the wager?
Whenever we build, we use intelligence to rearrange things in more complex ways that don't naturally occur. Ideally, this creates value by unlocking things we want that we previously couldn't have.
But this work always creates a burden on the future (debt) to maintain the complexity we built. This is especially true if the complexity doesn't gracefully degrade back to a more primitive state and the choices of people coming after you are "maintain" or "collapse and regress back further than your predecessor started, because you've forgotten the old ways".
If you don't want to place this burden on the future, then the only surefire solution is to not build anything. Just be happy with maintaining what you have and hope that the environment doesn't change around you so much that what you have isn't good enough.
But I don't think this is what we want. We need to build a LOT more for some of the existential goals we have as a civilization.
Just know that every act of creation that sticks around intrinsically creates a burden on the future to maintain it.
Acknowledge and respect that, but keep building.
A concrete example of this conflict that I've been wrestling with recently. I've been considering going all in on home automation. Zigbee devices and sensors everywhere managed with custom rules on Home Assistant. I want to get to the point where I don't ever have to touch a light switch.
However, I also recognize stories of others who have done this, and then died leaving their less technical spouse with an unmaintainable mess that gradually degrades and falls to Entropy. They eventually have to hire someone to rip it all out and revert to a dumb house.
By going all in on home automation, I'm creating debt for future me and my partner to maintain it all. At the moment, I'm not convinced that this act of creation will actually create enough value to justified the debt incurred.
The reality is that some things we use are joys to use, and other things are problems that we have to keep patching. Some cars are fantastic, good-performing, reliable tools that get the jobs done. Others are constantly breaking down and just might make it, sputtering, over the next hill.
I wonder how hard it would be to get major religions to endorse the concept that engineers who build useful, reliable, loved things are rewarded in the afterlife, while those who create things we all hate will be punished in the afterlife. Most religions already endorse the concept that we should serve our fellow man, and not cause needless suffering. Causing millions of hours of suffering for a paycheck is generally considered evil.
All code and features are overhead. It requires maintenance when it breaks, or requirements change. It increases complexity when working on other parts of the system, as other new features have to ask "how does this interact with existing features?", even if the answer to that is "it doesn't". At Amazon one of the VPs had a rule of thumb that if a dev or team worked full-time on a new feature, they would be at 50% of their capacity after the feature was "done" for 3-12 months due to ongoing maintenance.
So yes, writing less code rather than more code for the same feature is good, generally. But all software requires some code, so the challenge is figuring out which code is necessary for which the maintenance and overhead costs are worth it.
Technical debt is a bit different; the code is there, but something about it makes it challenging to maintain. There's lots of reasons this can happen. Sometimes the business requirements change and the domain concepts or abstractions no longer make sense. Sometimes the technical requirements change and decisions made at the time are no longer the right ones, if they were ever right at all.
But the most common reason, in my experience, is that the people have changed. New people have come in, looked at existing systems and code, and either don't understand it or don't agree with how it was written from a personal stylistic point of view. Like a functional programming looking at a system that was designed with textbook OOP and calling that technical debt.
As you move into more senior technical positions or engineering management one of the bigger challenges is determining what's "real" technical debt that's worth tackling, versus what's perceived technical debt that's based on personal opinions.
A line of code is something you own. It makes you obligated to either maintain it or let it go. The only truly free development is greenfield development, where you can write whatever you want. And now you've got code you own, and you're in its debt.
So yeah, we can all live wonderfully free lives by writing no code, but that does tend to up our other debts, i.e. student loans, mortgages, etc.
Good code is not more code or less code but code that does what the customer actually wants it to do. The better we understand the customer's needs, the more we can write just the correct code. We can't always do that; often the customer themselves doesn't know what they need.
Regardless, the way out of this conundrum is to stop counting code, technical debt, or other ways of trying to make the human problem into a computer problem. The real problems are always out there, not in our repositories.
Best code is no code.
All other code requires updating dependencies.
Close enough.
Some of the managers decided that it would be a good idea to track the progress of each individual engineer in terms of the amount of code that they wrote from week to week. They devised a form that each engineer was required to submit every Friday . . . .
Bill Atkinson, the author of Quickdraw and the main user interface designer, . . . was just putting the finishing touches on the optimization when it was time to fill out the management form for the first time. When he got to the lines of code part, he thought about it for a second, and then wrote in the number: -2000. . . . they stopped asking Bill to fill out the form, and he gladly complied.
It seems that the author's focus is on how things can become needlessly complicated, even when trying to implement a seemingly simple feature. However, citing only a few examples is not sufficient evidence. Could we not also consider instances where writing additional code could actually help reducing long-term technical costs?
Aside from this, I found the following points interesting:
> A technique for reducing technical debt when adding a new feature is to work within the constraints of existing assumptions, rather than adding new ones.
> Even when it becomes apparent after the fact that the feature isn’t performing as well as we anticipated, the typical approach is to do nothing. Part of this is the sunk cost fallacy. Often there is the hope that even if the feature isn’t used today, it will be in the future. But there’s also a legitimate reason for doing nothing. Removing a feature also has a cost, both in the development effort needed to cleanly remove it, and in potentially upsetting customers. So once a feature is added to a product, it is almost always there to stay.
If we look at math / category theory we have functorial semantics, which means semantics (code) is auto-generated from syntax (design): https://www.algebraicjulia.org/assets/slides/nist-workshop-2...
If we look in another direction we see LLMs which (will one day) generate code from design.
If I look back those years in my career, I was always paid to write the same code all over again (lists, trees, tables, menus, user profiles, shopping carts, dashboards, heroes, apis, databases...), each time in a different context (given by the company) ... Such a waste of my time, and my employers money.
Today at least in the MVP / startup scene coding is debt / obsolete. Startups all should go no-code.
Later, after the startup phase, companies should focus _only_ on provably-correct, likely-correct understanding of their problem domain, on a design specific to their audience, and rapid iteration. The rest, coding, should be not no-code, but fully automatic yet provably correct (in the sense of total and partial correctness)
I hope, and I wish coding will be eliminated. It's a big fun writing machine code in Assembly, algorithms in C/C++, full-stack web apps in Ruby on Rails, and user interfaces in React.
But after a while it's not sustainable neither emotionally nor economically.
Ironically, startups hire inexperienced people who often over-peform but sometimes underperform. A less risky approach would be to spend more (cash or equity) to hire people who are more of a sure thing. This could be called "execution debt" but it is accepted as visionary and associated with believing in people and being able to sense talent.
Tech (architecture, code, devops, etc.) is no different. A highly skilled team will be able to make the right tradeoffs and cut the right corners.
Tech debt is usually what people call something that appears in hindsight to have been the wrong corner to cut, however one would need a time machine to understand why the decision were made and what was competing for time/attention/resources and why.
Engineering is the discipline of managing tradeoffs to create something.
When working with a codebase that appears to be technical debt, the important thing to ask is what the options are today to address it. It may be beautifully written, clean code that uses the wrong abstractions, or it may be a solid architecture with some ugly code and style that needs to be addressed. Lots of people have biases due to their own preferences, but the thing to rememeber is that chances are the code was not made by someone with infinite time and perfect knowledge.
E.g. Buying a sandwich so you can eat it to be full is choosing not to spend that $10 on anything else.
In durable and intellectual domains, choices can incur opportunity costs, but also lead to switching costs in the future. I've always understood technical debt to be the combination of opportunity costs and switching costs resultant from a specific choice that resulted in a durable physical or intellectual good.
In this way, technical debt is not a particularly complicated topic. The issue is that switching costs can be very high, and to eliminate the accumulated opportunity cost and switching costs of a previous decision, it is necessary to incur a new set of costs going forward.
There is no "debt", there is only cost. The idea that you can "owe" on a series of choices implies that you can "pay it back". This is false. You can make choices - some may be more costly than others both immediately and in the future - but all choices have costs always. To quote Rush, "if you choose not to decide, you still have made a choice".
For me, "technical debt" would be shipping a product that immediately starts consuming technical resources not associated with value production. Like a rickety distributed database that needs manual labor to fix regular corruptions.
That would seem analogous to debt. Not fully paying for what you have, resulting in continuous expense until the full payment is made.
Writing code without care for future work is a different problem. It is the "bad foundation" problem shared by design, engineering, construction, book stacking, ...
It has 1000 names: "Building on Sand", "Ticking Timebombs", "Bandaid Solutions", "Cutting Corners", "Short Term Fixes", "Playing Jenga", "Putting Lipstick on a Pig", "Haste Makes Waste", "Disaster Waiting to Happen", etc.
Living code is a foundation. Shoddy foundations are dangerous to build on & costly to fix. Don't build shoddy foundations.
No new terminology. You don't need a special strategy to explain this.
Tell your CEO they can get their enormous features yesterday, they just need to sign off on a "shoddy foundation" exception. Informally by email is fine. You are ready to unleashed the troops from responsibility for future mishaps! All you need is that little written record! Then presto. The coders are standing by, breathless, ready to party! (And preparing their resumes.)
___
I love the essay's point that each feature adds assumptions, and assumptions add complexity. The rub of remodeling.
Because based on what I read, that seems to be his problem. He started implementing a system without having a good understanding of the domain's inherent problems and their common solutions. Not everything needs to be re-invented.
Even though I have never worked in this domain (event organizing) but still I could from the top of my head pull out one such example where a team went into great detail how to design and implement an event organizing system which answers most of not all the problems that the author faced: https://learn.microsoft.com/en-us/previous-versions/msp-n-p/...
I see these phases for many small software product:
1. Development Phase (Features are added continuously)
2. Maintenance Phase (No features are added, compatibility is ensured and bugs get fixed)
3. End of Support Phase (Basically no development happens for the product any more.)
The second phase "Maintenance Phase" is usually much longer than the development phase for a successful product. The cost during this phase heavily depends on the amount of features, that were added in the first phase. So be careful what you develop, you need to maintain it for a long period of time and at one point in time it will be obsolete anyways.
If a module is fully replaced later, it doesn't mean it was technical debt. Technical debt is characterized by its need to be refactored/modified in a way which is not backwards compatible. If a module can be substituted in its entirety with minimal changes which directly reflect requirement changes, then it wasn't technical debt.
Good code is about setting up components in a hierarchy which allows you to adapt to requirement changes by substituting modules as opposed to refactoring those modules.
Note that if you're adding features to an existing module, this does not count as a refactoring. Not if the module is backwards compatible with respect to its dependents. But of course when adding new features you want to stick to the minimalist philosophy so you need each module to have clear responsibilities.
Your job as an engineer is, amongst other things, to identify the (sometimes rare!) instances in which code creates leverage beyond the business value it creates. Good abstractions are valuable assets!
If we are trying to deliver fast by expediting software development and taking shortcuts (technical debt), another department can also do something similar (quick, our HR process is slow and cumbersome, implement that HR system) - which would solve the particular issues for the business at that particular point in time, but may not the needs of the business later on.
For software, the issue is that we believe we can introduce technical debts, that is, it is permissible to introduce and talk about technical debts. We wouldn't necessarily talk about implementing a new HR system often.
We think writing a line of code is fast (and define what "fast" means) but it isn't.
"Contributing" 100K LOC by implementing a bespoke http.c and mysql_client.c in 2020 is almost never something to be proud of from a SWE and product perspective. Great for ego/NIH and job security though.
“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” ― Antoine de Saint-Exupéry
Of course, at the time noone thought copilot could actually do any of that, but it was a few weeks ago and things are moving quickly.
It's a cultural issue where we think major faults "just happen" and are not directly tied to prior product decisions, and therefore there isn't much accountability.
But if you propose a major refactor, there is no question about (rightly) where the blame lies.
> But if you propose a major refactor, there is no question about (rightly) where the blame lies.
LOL Yep.
Worse is when the blame lies on the semi-technical cofounder of your company who, granted, achieved quite a lot on their own, wrote a barely maintainable mess. Do you really want to be the one to indirectly point out that his code needs to be replaced? Haha I think a lot of developers would rather wait for the Dilbert Principle to kick in so that the cofounder gets out of the way enough for significant changes to finally be made.
It’s not a perfect analogy, but the difference between a low interest rate mortgage and a high interest payday loan have parallels to the balance made in delivering software while limiting complexity. A developer that is a bit of a tornado, rapidly churning out new and exciting capabilities at the expense of reliability, brittleness, and complexity, will surely catch up with an organization sooner or later.
Not a fan of this habit of taking a perfectly good technical term and redefining for another purpose; especially if there is already good, existing measures, such as code complexity.
Rather, than calling all new code "technical debt", how about every planned feature, as part of it's total cost calculation, include the incremental costs of any increased code and/or architectural complexity it introduces.
Conceptually, I think every developer eventually realizes that every line of code ends up adding maintenance cost, and potentially constraints on the future. "Technical Debt" is one way to think about it - if you don't have the resources to maintain. Otherwise, it's just a cost as others have pointed out.
Maintenance is harder than most people think: documentation, translations, tests, all are part of this cost.
Other examples that come to mind are "a complete rewrite is always a mistake" (no, it just has different risks/rewards), "if you are reading this part of the docs, you are trying to be too clever" (your API is too much for my puny brain is it?) and "all functions are APIs" (no, my two-line string quoting func is not an API, actually).
That seems to describe most of the programming clickbait here. I could swear there used to be more discussion of startups . . .
This makes a lot of sense to me, and explains why you always think "I can do better than that monster", and it does always feel better at first, then then inevitably grows to feel like a monster where it's hard to make any changes.
Especially when we are thinking about tools for developers, where developers are the users, libraries or web frameworks and such. Or even languages/ecosystems. A bunch of existing libraries/frameworks/platforms/ecosystems are a mess... surely we can do better! let's make a new thing!
This explains why we are always hopping to the new thing, and always convinced the new thing is better...
And the new thing is better and so much easier to work with, even though it doesn't have all the features the old thing did, but okay, we just need to keep building it out to encompass those features, using our much better sense of architecture and usability that we've proven we have because look how clean and simple and usable our first draft is!
Now, there is such a thing as better or worse designed, as the OP admits, poorly implemented or beautifully implemented. But... comparing the new thing that only has the most basic features and seems really elegant to the old thing that covers all the edge cases... isn't actually a good comparison to tel you which is better implemented. That new thing is going to get cruftier when you cover the edges, _always_.
And it shows that if you want to keep your thing feeling smooth and elegant and easy to work with.. you have to keep it as small as possible and resist all feature requests!
> For a feature to add value to your product, it needs to be useful to users. Features can have a negative value when the technical debt they add to your product outweighs the value they add to it.
Thinking of tools for developers where developers are the users... oh yeah it's true.
> A technique for reducing technical debt when adding a new feature is to work within the constraints of existing assumptions, rather than adding new ones
I would describe that as limiting the number of "concepts" inside your code... and definitely applies to libraries/frameworks/tools for developers.
Technical debt is a thorny issue already. Arguments like this don’t help.
I think there's some truth to this, but code isn't always written by software engineers and isn't always executed by computers. You might end up with crazy Excel macros, and/or arcane people-processes with single points of failure.
To "not write code" at all is probably to never move or do anything. Sure, you never lose if you never play, but you also never win.
- Unnecessary (too many or too heavy) dependencies that can be replaced by DIY code, or at least, smaller dependencies.
- Implementations that stray from the standard set by the rest of the code in the project or actual guidelines written by a project leader (e.g., we don't use jsdoc anywhere but a developer randomly crams it in during a copypasta).
- Over-complicated implementations that serve as an ego-pleasing flex for the author, but are impossible for even an experienced developer to reason through.
- Implementations that were done to service a single customer in pursuit of short-term revenue, or, that were rushed through due to manager/stakeholder panic and inevitably collide with one of the above.
---
Keep those three in check and you'll have limited issues.
Edit: a good talk by Oliver Reichenstein (of iA Writer fame) that helped me years ago https://vimeo.com/102343006 -- it's from a design perspective but has parallels to development. One quote of his that constantly pings in my head: "slow the fuck down."
This is one of the reasons I oppose the “automate all the things” mantra.
I like to dive into a codebase the same way people do accounting for the finances of a company, the parallels always make sense.
"What do you mean, we have to pay health insurance for all of them?? Let's get the cheapest option available!"
- Similar to the generality "all code is technical debt" I'll make one of my own: "All technical debt is abstraction assumption failure". You've got an abstraction; you assume it behaves some way; it doesn't. You've got an abstraction; you need to add new functionality behind that abstraction; you assumed this is a good place or a good way to add this functionality; it isn't. The vast majority of technical debt is just this. There may be a couple more categories like "we can't hire people to work in this language" or even something like system performance, but the vast majority of technical debt is a failure of understanding of some abstraction.
- That previous statement does not assert that this is an invalid way of viewing technical debt. "Understanding" the code is literally the most important job of software engineers; less "engineer" and more "museum keeper" sometimes. I've never liked the term "bad abstraction" because it shifts the responsibility of being understood to the abstraction, and not the engineer. It encourages "code bad, rewrite" mentality. Sometimes that is the best course, but I think its always a bad idea to tackle that without understanding the abstraction, and if you understand the abstraction then you probably wouldn't be calling for a rewrite because its "bad"; you'll be calling it for more specific reasons.
- Unilaterally classifying all code as technical debt isn't just wrong; its asinine. "Technical debt" is a useful way of thinking about some code; if you remove that from your lexicon, you may start believing that all code either is or isn't technical debt. If all code is technical debt, your ability to think about the quality of code holistically, with its nuance, is reduced durastically. If you think no code is technical debt, you'll inevitably walk down the path of hyper-engineering perfect giga-systems that, even if they work flawlessly and satisfy all business needs, took way too long to build, cost way too much to run, and inevitably still confuse your replacement in six years.
So what should our goals be?
Less code means fewer features, which can make the UX path longer. More code means more features, which can make the UX path more confusing and strenuous. There is some kind of balance to be struck. The goal is for every UX path to be flexible and easy. Is that even possible? Can we have our cake and eat it, too?
> Adding new assumptions increases debt
There's the word I have been thinking about all week: "assumption". It's a tricky little bastard, and it really deserves more focus than it has any right to.
You see, that word is the heart of our problem. It's not the amount of code you write, the features you add, the way you design your UI/UX, or even how flexible your user configuration is. It's the assumptions themselves.
Assumptions are walls. Once an assumption has been added, it’s there to stay. It's easy to move an assumption around. Too easy! So easy that you move it without even knowing; but it is still there, lurking. No matter how hard you try to get rid of it, or to get around it, the assumption is guaranteed to stand in your path. Your only hope is that the assumption is there to guide you; its only other option is to guard you.
So what can we do about it? We can not. No need to get rid of something you never made in the first place, right?
But how do you make software that is unassuming? How can you make a tool without catering that tool to its intended work? How can you make a path without a destination?
There's something we have gotten notoriously used to as software engineers: a finished product. After all, what else would an engineer strive to make?
Finished products are things. Programs aren't. Programs are systems! Programs are stories. What kind of system is ever finished? I can only think of one: a dead one. At the end of its life, a system becomes cold, hard, sturdy, and complete; and the story meets its end. How did we ever convince ourselves that that is a worthy goal for software?
I'll tell you how: we assumed. That assumption has been lurking at the heart of software design ever since, and all we can do now is move it. It's time to start over.
---
This is where I answer my very first question: who? Who designs the software? Don't assume it's the engineer's job: it's the user's turn now.
The next question is, when? When does the user design their software? When they use it. Any time before that would make them an engineer.
What should our goals be? That's the best part! The user already knows her goal. That makes her precisely the right man for the job.
---
If you followed any of my rambling so far, you are probably asking yourself the most important question, "What could this possibly look like?" That's a good question. I have something really close to a coherent idea. If you would like to hear about it, feel free to ask.
we seem to believe we are craftsmen sculpting some work of art
code is just shit you write, means to an end, be it closing a ticket or going home. soon deleted ("deprecated") and replaced with new trendier shit.
> "we need to ensure we’re working on valuable features"
yeah valuable features are what your boss's boss needs to get their promotion and/or bonus.