Tech debt gets worse before it gets better
jeremymikkola.com
jeremymikkola.com
As a concept it lacks all nuance. I argue it is one of those vague, amorphous blobs that means anything to anyone, to the point that no one agrees on anything when discussing it, except that it exists. Everyone thinks they know what it is though, which makes it all the worse.
I advocate we discuss real issues with code and architecture instead of using the term 'tech debt'. Articles like this are like going to a self-help speaker. You walk away feeling great but also totally unequipped to change anything because the advice was contrived and only works in non-existent general situations.
Aka if it’s 5 years old and nobody had needed to edit it then yes it’s ugly but the interest rate is probably low. Meanwhile stuff that just a little off but gets touched every week can be much more meaningful.
Better to start with real observations about the workings of a particular piece of software or architecture, then reason about why it may have certain 'bad' properties. This more often leads to action and mutual understanding, but requires actually putting in some mental effort.
In your example, calling some code 'ugly' is another example of what I'm saying not to do, it is starting at a conclusion and reasoning backwards.
For an example of what I'm advocating, say you started a project and defined certain architectural principles after a series of discussions with stakeholders. Let's say one of those principles is efficiency/speed of execution because the stakeholders use the software synchronously. So, that is how you define code quality in that project. You benchmark and eliminate hotspots until you reach acceptable performance. Independent of context, fast code may appear ugly to some people, but in this context it is not, perhaps it is even beautiful as it so embodies and elevates the architectural principle it was designed around.
Developers now have a frame to surface issues and everyone can agree on what an issue is, because you defined what an issue looks like and why it is important, up front, as one of your architectural principles. This is basically a context-bound value system. There is no need to even use the words 'technical debt'.
In a real project you have multiple competing architectural principles but these are prioritized so you know how to evaluate whether or not something is 'good' or 'bad'.
As a project leader I believe it is your job to define these architectural principles, to enable developers to frame 'good and bad' as well as to prioritize their work.
I used this approach when leading a project and it was very effective.
There are however a meaningful class of issues that have zero direct impact outside the team, but are still issues for your team. The best example of that is how long it takes to setup your build environment, frankly your users have zero reason to care but it can still slow your teams progress over time. Really the cost and payoff of fixing it is simply time / a better work environment.
Different teams will of course use technical debt to describe stuff outside of that context, but if it directly benefits users then it’s really something else.
Ugly code doesn't cause issues. Refactoring a reliable, functional, easily maintained codebase because it isn't elegant (or 'beautiful') is a waste of time.
At this point we can’t for example fix historical calendars, so all highly accurate date and time related code is going to be complicated and full of edge cases. Such code is in effect an interest payment on society’s existing technical debt. So, the temptation to refactor such code is generally attacking the wrong side of a problem. The best approach may be to quarantine such code/systems and try and minimize the impact of such issues.
> The best approach may be to quarantine such code/systems and try and minimize the impact of such issues.
That’s the old Google’s “v2 is wip v1 is obsolete”. Works ok internally, externally not so much
I challenge you to come up with a situation where technical debt is a better way to describe activities than indirect impact as you are doing here.
For one it’s more specific. Saying the existing code has indirect impact doesn’t even define if it’s a good or bad impact. If you want to say, the existing code has long description that just means technical debt then sure they mean the same thing but the term technical debt is more concise.
You can still add all the low level details, but redundancy in communication still aids clarity.
A few years ago, a director at my organization said at an all-hands meeting that he viewed "tech debt is a debt we never have to pay and have no interest on". In one swoop, he told the entire organization that technology concerns are not valid.
People have been complaining for years using specific issues. We could point at times that important services went down. We could point at full time operations people who have to babysit applications. We could point at failed initiatives. The bi-annual company survey always has "we're fighting the same fires" in the top issues.
I suppose the issue with talking about tech debt to non-technical people is that they don't care. It doesn't affect them. At least not directly, like it does for developers.
In re-reading my comments, I think it's easier to interpret what I wrote in an only negative way. I think that dealing with technical debt is would be a topic that the company does poorly.
On the flip side, they build new products quickly and use new tech often. I consider the organization to be similar to other big-company feature factories. There's a degree of dysfunction in that, but also has a purpose.
Engineer's lose motivation and put less effort in. The company has to spend more budget on operations. Emergencies come up. Projects fail to deliver on-time.
These things are the tech debt and the interest being paid.
He had a point in some cases. There are products that aren't used or aren't under active development.
He was lumping all of the tech debt together. He was saying to everyone in the organization that he viewed all tech debt this way.
Everytime I see someone mentioning technical debt I take as a signal that there is a vague idea of opportunities for improvement, but no one has taken the effort to quantify the costs and benefit in a specific way. If you see yourself using the words "technical debt" as an engineer, it means you need to do your homework and be more specific and ask yourself what you're actually saying.
Instead it should be pitched to the business as “we should change how we architect $service because the time spent on cards which interface with it are mostly spent wrangling rather than actually implementing the desired features. With $new_model here’s the code for what those same cards would have been and we expect that features of this nature will take 4 hours instead of 40.”
But then it's also used for subjective things like "too monolithic" or "spaghetti code" where the value of fixing might not be clear. Or even that the issue is really an issue.
I recently explained tech debt to my 17 year old son, who is very interested in programming. It was difficult to come up with a crisp definition that captured what I mean by tech debt, but it was a good conversation.
a team of elite coders and a senior architect was put in charge of the project, and they decided to implement IoC & OSGi throughout the code base and split the repos up into cleaner modules
a year and a half later, they were done. glowing company wide emails were sent, it was a new era
things got significantly worse: the new system was very difficult to understand, testing cross-module issues was impossible to coordinate, OSGi added almost no value at a heavy cost across the code base, etc. developer productivity, already low, fell even further
the senior architect left shortly after the project was completed
careful with that refactor, eugene
The goal was never to “level up”, it’s just that these large deliverables take a lot out of you.
This is an attrition issue. The company lost an employee, fair game.
A small team orchestrating a large scale refactor in isolation for over a year seems appealing to project managers because it allows that team to focus without interrupting the pace of feature development, but it makes it exceedingly likely that the output of that effort will fail for these reasons:
1. Evolving requirements. A refactor is a design that is focused on solving challenges _at that time_. Things happen over a year. A product grows, its supported feature set requires rethinking the nature of the domain, the pressures of scale force new considerations, teams get more rigorous about the kind of automated testing they practice, etc. A team that doesn't have a feedback loop into is likely to build something that may have worked for the system of a year and half ago, but no longer makes sense.
2. Lack of buy-in and knowledge sharing. In isolation, most software engineers tend to solve their own problems. Senior engineers often make assumptions about the knowledge of the rest of the team and about the coherency of their abstractions when they don't have to teach it to anyone or battle test it in any meaningful way. This is why collaborative processes like code review and pairing are often a huge win, but sharing these efforts needs to go farther and a team should be pushed to try incremental adoption and feedback until there's more confidence about a refactor. Large scale refactors should be chunked and interspersed w/ a workload that attempts to validate the success of the design (i.e. working alongside teams doing feature development to make use of the refactored design). And perhaps most importantly: the team working on this needs to have clarity on what it means to be successful. How do they know that their proposed design is meeting the needs of the team at large? Infrastructure, dev ex, or any team that is orchestrating something like this needs to think about their task like a product team, and "what does success look like" needs to be a question from the onset.
this is sort of like w/ the agile folks: when an agile implementation fails, "oh, you didn't do agile right"
obviously, better code is always desirable, but the temptation to rewrite is very strong in developers (I feel it all the time) and experience suggests it often leads first to a worse situation, and then a worser one
A rewrite is usually a quagmire if you don't do these things, but even the need for a rewrite is usually avoidable. If a team is making frequent observations about pain points and places value on fixing them quickly, they can often be avoided. But there are discontinuities: product concepts and features that dramatically change the constraints of a system late in the game, unanticipated growth requiring you to reconsider architectural decisions, etc. There's a long tail of events that you could be proactive about, but would likely create over-engineered solutions that place more limiting constraints on the design if you tried to be. They still occur though. It's very important to sufficiently interrogate the question of if a rewrite is necessary to solve these challenges, but you also need to interrogate the cost of band-aid solutions w/ limited applicability.
I don't think there's a silver bullet here, but most of the failed attempts at doing this that I've seen have been a result of teams that didn't sufficiently consider what the outcomes they were trying to provide were and how to test their candidate solution more rapidly.
Giving that group a big budget and autonomy to fix the issue is like paying tobacco company to cure your lung cancer.
Repenning & Sterman, Nobody Ever Gets Credit for Fixing Problems that Never Happened [PDF] https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.... (pretty sure this is the one I want)
Being proactive and doing the right thing (maybe, potential higher costs earlier than they can be supported for instance) is a low-visibility activity. Even if it's your job to do the cleanup work, the one who wrote it first is the one most likely to get the credit no matter the quality. Because they got something out there for customers, not you.
Not only have you guaranteed total loss of knowledge, someone who had never maintained anything has no chance of writing code that survives contact with production.
You'll get all sorts of shiny frameworks and nice architectural dreams, but no logging, no monitoring, no validation of the actual user requirements, no nothing. You spend the next 5 years rewriting from scratch if you're lucky.
I've worked in a couple of large and messy codebases where we undertook a big refactor, it took longer than expected and then the resources were pulled to other things before it was completed. Net result: an even larger, messier and harder to understand codebase!
"We decided not to FOO" is already called tech debt.
Say "we MUST BAR but never planned for it" is an Oops.
And "SQUISH lacks any sense of consistency or intentionality" is a SNAFU.
And a "we don't all even have common language to talk about what we need to do" is a Whoopsie.
Then maybe we can also introduce sparkline graphs to help us visualize the effort needed to overcome it all: the Whoopsie-Doodle.
If you fucked up, that’s not technical debt. If business requirements change, you don’t have technical debt
Calling every problem or challenge technical debt is so unhelpful and frankly a bit dishonest too.
There's a lot of technical debt that's going to accrue before the process debt is sorted if you're trying to scale quickly.
But it is my impression that this is the rare exception. In most cases, people are just bullshitting to cover up their ineptitude.
"We maintain a lot of BAZ we think no one needs anymore but we don't know for sure" is called a Yikes.
"You just have to know which alerts matter and which don't" is called an Uh-oh.
And of course "securing the QUUX wasn't an explicit requirement for launch so we haven't done it yet" is already called a vulnerability.
>"SQUISH lacks any sense of consistency or intentionality"
And once you acknowledge the problem and decide not to address it, it's tech debt again.
The point stands: Unforeseen problems aren't 'tech debt' in themselves. They become tech debt once they are recognized and not dealt with.
The garbage crap dumpster fire of code at your company is not tech debt; it’s incompetence on display.
1) My house has structural problems and is about to collapse.
2) Each time I turn on the lights in my house, I start a fire.
3) Each time I need water in the kitchen, I need to go get it from the second floor.
4) I don't like the house color.
You can refer to these 4 situations saying something vague like "My house has a problem". The term "tech debt" is similar in terms of vagueness.
Not everything described as "tech debt" is objective or actionable. And the term "tech debt" facilitates neglecting things that need to be done by mixing them with things that do not need to be done.
"There is an antipattern that prevents developers from testing code" is a problem that needs fixing. "I don't like tabs" is not.
Yes, some people use 'tech debt' improperly and aren't actually describing something that acts like a dead. Poorly structured code means the coders need to spend more time implementing new features than they would if it were just correct, for example, so that's actually 'technical debt'. Not taking care of the business's needs isn't technical debt (faster onboarding, etc), it's just a problem.
1) I owe $0.25 to my 5yro niece because I lost a bet about whether grandma's pancakes would be overcooked. She will probably forget about this as soon as Harry Potter comes on the TV.
2) I owe $50 to my housemate for groceries but actually I paid the electric bill and he didn't so maybe he owes me instead?
3) I have an $8000 balance on my credit card but can pay my minimum balance plus $500 extra every month with no sweat.
4) I have $500000 left on my mortgage and didn't make my last 5 payments...
These are all still "debt" and have varying degrees of actionability. A word that implies a dimension does not need to specify the magnitude or severity on that dimension.
I’ve also noticed numerous long term migrations that don’t tie back to a strong long term value proposition. And when the claim is that it does, it’s often just overly optimistic gazing into the crystal ball. Just as dubious as multi-year product roadmaps, for most businesses a ton of unpredictable things will occur.
-Steve Jobs.