Most Technical Debt Is Just Bullshit
louwrentius.com
louwrentius.com
Calling code "a mess" does not really communicate why it should be improved. Who cares if it is a mess if it works? What developers actually mean with "a mess" is that the code gets increasingly harder to maintain which incurs cost down the road. Technical debt is a good metaphor to explain this.
I think the article is misunderstanding that we makes the whole article moot.
One of the big things I’ve learned speaks to your point though: it’s not a natural way of thinking for most engineers. There are a few who are “natural complainers” who can tell the difference between “work that needs to be done” and “work I have to do because someone took a shortcut in the past”.
But actually many of the “natural complainers”, while they have strong “this is insane” detectors, they often complain about anything that’s not perfect. But that’s not exactly what we want. Many things are not perfect but they’re pretty easy to work around. That’s not an interest payment it’s just an annoyance.
And on the other side of the spectrum there are a lot of coders who have a “whatever it takes” mindset, and they can’t really perceive interest paid at all. To them, every necessary thing is reasonable. If it has to be done it has to be done. They don’t really think about how it could be done more easily in a different world... because we’re not in that world! Why think about a hypothetical!
So, I have some more thinking to do about how to teach folks to think critically about their time and past decisions.
And the other big learning I got is the recording of interest payments is only useful if it’s somewhat regular. So I think it has to be an official, required part of the production process. If people have to remember to report on interest paid, they just won’t.
The project is on pause right now, but I’d like to give it another shot with those things in mind. I think I’m going to wait until a time when it feels like it fits in with the needs of the managers though. Otherwise that second piece of “part of the process” is gonna be hard.
But if I can at least put an “interest paid” number on some tasks, my hope is that will provide a good way to identify which projects will “pay off” if they are “paid off”. Basically I want a vaguely rigorous way to put refractors and infrastructure work up against new features so they can be compared 1:1 during planning.
Also feel free to email me if anyone is interested in something like this at their company.
Debt is a great thing (financial debt). You can get free things now and pay for them later. Or never pay because you're changing company anyway. (That's what MBA managers understand)
Also, debt in software is inevitable. Every developer that comes in say everything is debt, and the next one will invariably says the same thing about his predecessor. It's as if everything ever done is debt, might as well ignore these guys. (That's what managers interacting with developers quickly come to understand)
Technical debt is misleading. A better term is a time bomb (fixing consequences might be exponentially harder than the original issue).
If we look at the wikipedia page about technical debt, there is a long list of possible causes of technical debt.
To site some examples:
Insufficient up-front definition
Lack of clear requirements before the start of development
Lack of documentation
Lack of a test suite
Lack of collaboration / knowledge sharing
Lack of knowledge/skills resulting in bad or suboptimal code
Poor technical leadership
Last minute specification changes
Notice that these issues are called 'technical debt' because they can have a similar outcome as technical debt. They can create a bottleneck.
But why the hell would we call these issues technical debt?
These issues are self-explanatory. Calling them technical debt not only seems inappropriate, it just obfuscates the cause of these problems and it doesn't provide any new insight.
The question I always ask is what happens if we do nothing? Debt at 0% interest is obviously not a problem.
Often the original paradigm was chosen for a reason and it becomes clear that the refactor was a step backwards.
Any ideas on how to prevent this?
If it's innocuous and they have a bee in their bonnet, consider letting them refactor on their own branch, and subject it to normal code review
Recount the Parable of Chesterton's Fence
Ask them if they are just bikeshedding
Let them make low cost mistakes to learn cheap lessons, saving expensive lessons later
My experience is junior and senior developers fall for this.
> Often the original paradigm was chosen for a reason
True, but the reason is often: - we didn't understand the problem domain as well when we wrote it - it was written this way because it needed to integrate with a system that existed at the time but no longer does - there was a guy who was really into X and he insisted on writing it this way. He no longer works here and no one wants to touch that code - It was written by an intern - It was written by a bad software developer (you're not morally bad for being a bad software developer, but they are real, and they write code) - It was written as a prototype and was supposed to be thrown out - It was written by me, and the fact that you want to rewrite it makes me feel bad
> Any ideas on how to prevent this?
Exploratory refactoring is actually a really good way to understand code deeply. Sometimes the refactor is bad and your throw it out, that's ok, you learned something. This applies to developers refactoring code, they learn the reasons for things as they rewrite it.
Now, new hires need to be given the new features so they’re happy and can rush out and create new technical debt from day one.
If you accuse something of being a mess that should be rewritten/refactored, some people may feel hurt which is counter productive.
In start ups that became big people often complain about existing code being a mess, but if it wasn't created as mess the company wouldn't ever had become that big.
I take a broad view of tech debt and include all the things: corners cut to ship; (2) missing tooling (tests, linters, CI/CD etc.); deferred maintenance (platform updates, etc.); and just plain "messes". What all of these have in common is that they are work that needs to be done in the future if the software is to remain usable and maintainable. They also interact—for example, it can be difficult to update your platform if you don't have a good test suite to catch regressions.
Anecdotally, I've been working in a codebase for the past couple of years that had a ton of tech debt. It was a tarball of Python 2 code with no tests or other tooling. My team has been delivering new features but also addressing the debt. Now the codebase is Python 3, with build-time typechecking and linting. Lots of copy-paste code has been consolidated into libraries with good test coverage.
This refactoring cost us time, but it is now paying off. Our development velocity is so much better and we're delivering features faster, with fewer bugs. It feels like our time has more "principal" and less "interest" to it. I know part of it is that the refactoring work necessarily made us more familiar with the codebase, but I also know that there is just less debt in the system.
In my view they both require continued ongoing maintenance to prevent negative consequences.
And they both can be intimidating and/or not feasible to “pay off” at the current time.
Just like financial debt, technical debt means accepting an ongoing burden and higher overall cost for a near term gain and avoidance of immediate lump sum payment of money/time.
Debt is perfectly good metaphor for the collection of issues described. Just like one can get into financial debt via student loans or reckless credit card spending, tech debt can be accrued many ways.
Additionally, talking about the origins of tech debt also misses one big thing - many of us primarily work on other people’s code. We inherit the debt, so it doesn’t matter how it gets there.
You still have to dig deeper to really understand the problem anyway.
I need to explain to non-technical people that this will eventually need more work to be fixed. What metaphor should I use to explain taking on temporary business risk now to save time, then spending more time in total to fix it in the future?
If you work on a team, that just isn’t going to fly. I can’t imagine the toxicity of an environment where someone constantly was calling out other people’s code as being “bad”.
When issues do start to come in, the next blocker arises: the dreaded Cost Benefit Analysis. If a data fix can be applied in, say, 30 minutes and the business is seeing one of these every week, it's easy to live with the cost if the alternative is to spend 30 man-hours on pushing a fix through "the process".
(I'm talking about shops which haven't been able to move to CI yet. Shops which rely on a dedicated System Test team and a dedicated Release Management team. Shops which require downtime to ship fixes of any sort and which require a release slot booked at least 24 hours in advance.)
Furthermore, that 30 minutes of hassle making the data fix isn't felt by the manager whereas shepherding a fix through the release process most certainly will be.
A codebase that has been shaped over the years by this combination of mindset and process is probably too ossified to refactor by the time it's recognised as a problem.
Cue the "chains of habit..." quote.
Given that there is a continuous change of priorities, knowledge and staff with different skill levels, it's just a fact of life that we accumulate some crappy design and lack of documentation. I think it's useful to have a label for this because it needs a different process for judging it's merit than customer or performance-oriented changes.
A debt is something that gave you value in the past. If nobody ever got value out of it (in comparison to an equally feasible debt-less solution), then it was just a mistake, not a debt.
A debt is something you pay an ongoing interest on. If your proposed solution to the technical debt is not provably reducing the ongoing costs, then you're not actually addressing a technical debt.
"Technical surplus" is your ability to learn and implement that learning as code.
The "bullshit" part is complaining about debt while not being able to run a surplus.
I wasn't really aware there was a much broader use of the term than that. Although I guess you you can overload the "deferred maintenance" category quite a bit if you get creative.
If the non-legacy system provided value, you could convince users to switch to it instead of a competitor, and your old system could be retired instead of legacy.
However I see the term applied to any system as young as 5-7 years old. I suppose in cutting edge areas of computing this may be the case, but not at all for things like ERP software that underpins the business. Major versions of systems like that often have a minimum maintenance horizon of 10 years on release, a clear roadmap, and a relatively smooth update path when new major releases come along. Where I work, we recently upgraded from Version 8.x of such a system, which the vendor had maintained for 15 years, to version 9.x. We'd upgraded into 8.x only a few years earlier from a truly legacy system had run for most of 30 years. It still worked fine, and was faster than what we have now.
We didn't update because the system wasn't doing what it was supposed to do: we updated because we wanted to do some new things, and because the vendor after decades and multiple extensions, stopped providing annual compliance updates for state & federal regulations, which also would have been extremely costly to produce ourselves (not to mention being a major legal risk if we got something wrong, and the vendor had previously indemnified us if they made an error of that sort). Otherwise we might very happily have turned to various "best-of-bread" stand-alone options for the new functionality we needed and then hired a couple of extra database jockeys to manage the complex ETL & data warehouse needed to make it work nicely.
> Technical Debt is the result of a weak definition of done