Self-Admitted Technical Debt
neverworkintheory.org
neverworkintheory.org
I call myself a “Loan Shark” because if you aren’t given an option by leadership to take the time to do something right, “admitting” technical debt is less about “taking blame” than it is about covering your ass. When that debt comes to bite the company later, bring your receipts and use them as leverage to do some proper technical investment driven by the engineers on the ground floor.
"Loan Shark" nicely anchors the business consequence of piling debt on top of debt. The interest rate ain't gonna be your safe little bond interest rate, it's gonna be whatever the situation says it is.
Please elaborate.
- Open to connecting about tech debt?
- What's the going rate on a tech debt loan these days?
- Tech Debt Loan Blue Whale Opportunity
I really enjoy the challenge of plotting a course out of a hole like that. However in my experience without management buy-in it just won’t happen. Even if you as an engineer do the planning work to make it as easy as possible to pick off low hanging fruit, you’ll typically be seen as “off task”. You’ll get lip service about how code quality matters, because without that they’ll scare away the other devs. But they’ll be hoping you give up and just focus on delivering what the management team is asking for each quarter.
It takes a ton of patience and management hand-holding, on the scale of 2+ years to shift the culture. And you have to be willing to forgo any chance at personal advancement for that time. Because you’re “off task”.
I think you could take that approach and eventually get an advancement bump around the 3-4 year timeframe but it’s not the fastest way to advance by any means.
And the total failure rate for an endeavor like that is pretty high. Maybe more than half.
Still, it is fascinating work.
Sure, it's possible to live debt-free and manage finances or architecture frugally - and I consider that a noble thing - but you will have a hard time keeping up with those who keep covering up their debt with more and larger debt.
All debt packages risk. However, the kind of debt/risk perception we are personally conditioned around in daily life does not translate to software. Managing software debt requires investment banker-like thinking, rather than the mindset of a credit card or mortgage holder.
We like to blame bizdev and sales for putting pressure on us, but that feels like a cop-out to me. Maybe I'm just a bad programmer but even if you took out the deadlines and "requirements" I'll probably still ship code with some warts here and there. Complaining about people in other positions is no way to improve.
Sorry, maybe a bit of an extreme take. Am still interested in stories of especially maintainable codebases, as I'm probably over-correcting here.
Programmers who aren't you, and those damn MBAs ruining your flowstate
Doesn’t mean you should ship untested code because that would decrease both the value to the user and your development speed. There are projects out there that go into the other extreme, entering the realm of premature abstraction and accidental complexity. In my experience that’s even worse because in that case it’s hard to convince stakeholders that there even is a problem.
It‘s a tough pill to swallow for devs with perfectionist inclinations like me, but at the end what matters is providing value to the user. In fact, I think technical debt will always be present as long as requirements change, and as the saying goes, change is the only constant in software engineering. It‘s not something to be completely avoided as that’s impossible. It just needs to be managed in a pragmatic way to keep up velocity.
Startups you want to see with fairly high tech debt. Tech debt there is a very often a sign of shipped code that is meeting some customer need. You do need to watch out for insane tech debt that may be hard to undo later on (running everything in an ancient proprietary language for example).
Startups with low tech debt often are devs navel gazing without a lot of revenue coming in. This is surprisingly true almost all the time.
Very large companies will tend to have a mashup of all kinds of technology that is never fully untangled. A very large Corp with little tech debt is exceptionally rare. If it ain’t broke don’t fix it is the rule.
Mid sized corps and late stage startups seem to be the sweet spot. Fairly low tech debt systems that is providing high value is still possible. This is where you have enough maturity to know what “good” looks like, and enough developers, momentum and budget to implement the vision.
It's possible the people working at the time were not conscious of the trade-offs they were making. But it still happened and that's how it came to be.
> if I inherit the code base of a bad programmer on my job
It sucks, but it's almost guaranteed to be the case, to some degree. Think from the company perspective. You're thinking too much from your personal perspective, which is kind of irrelevant to the company. You're not using the term just for you.
I do, rest assured, part of this is my job description. I'm just bumping over the assertion that a deliberate trade-off is a prerequisite for technical debt. It's too specific a reason, and like saying your expenses are higher than your earnings, that's why you are in debt. But in reality, there are many more ways to end up in debt, spending more than you earn is just one possible cause.
If you identify technical debt, either hypothetical or existing, the term is an option to explain this to your superiors in a way they might understand. They'll know that they need to do regular small investments to pay it off, or chose an alternative option to avoid it. But that's all there is to it.
What are some others that don’t trivially collapse into exactly that?
Likewise, for technical debt: a dependency you relied upon is not maintained anymore. There is an exploit found, or regulation changes, and you realize you used the forbidden pattern everywhere.
Basically: earnings minus expenses are the operating result. Concious decisions to accrue technical dept would be the equivalent. But there are costs directly affecting equity as well, which can still result in debt.
More often it's just a sign of a new company not knowing what they are doing and making unforced errors as a result, rather than a conscious debt/investment transaction.
Technical debt isn't inherently bad. Otherwise it would be called "techinical loss".
The problem though is that in the financial world there are rules and often laws about how much debt is acceptable and a bank will generally refuse to let you get into dangerous levels of debt whereas there are no such rules concerning technical debt so a company can keep piling it on until it eventually causes a collapse.
The implication here is that if only you had enough time you wouldn't have any tech debt. I don't think that's true.
Some of the tech debt is due to not having enough knowledge about the future of your system; what it needs to do, how it'll be used, etc. You have to make a decision about the tech now and sometimes that decision turns out to be wrong, or at least incomplete, which leaves you with some debt that will need paid off some time.
I also believe some tech debt is unavoidable due to churn from outside of the system. Best practises for a given part of an app aren't fixed (eg React class components shifting to functional components), so sometimes something that was once good can turn into debt over time with no input from you.
The term "tech debt" means deliberate choices to compromise the system in a controlled way, to achieve a purpose. E.g. speed to market.
If it's not that, then it's not tech debt.
That's true, which is why things can become debt if you fail to maintain them, upgrade them, change them when the customer's requirements change etc. By choosing not to maintain code and do other things instead you're allowing parts of the system to decay. That is a source of tech debt (arguably the most common IMO).
When I am in 'bug fix' mode, I will try to locate and fix a few of them. I try to allocate a day now and then to focus on cleaning them up.
When you are in startup mode, trying to find the right product-market fit you cannot afford to make the code perfect. You have to get it to the point where potential customers find real value in what it can currently do. Shortcuts are a byproduct of that process.
Even if you can't get to it right away, a quick note of the issue in the code itself can make it easier to address it later when the problem is not fresh in your mind (or if someone else is tasked with doing it).
i'm one of two feature devs on a team of four. over three years the team wrote a few hundred tickets for debt. when the client was acquired and we finally got space to address debt, non-technical stakeholders asked us to nuke the list and write a much shorter one that we can prioritize by complexity and customer impact. the debt is now being addressed in the same way as new build, because that's how we treated it; much like new build, it won't be completed to our satisfaction
in my experience the only thing that works, short of a major incident that devs forecasted and offered to prevent, is buying time--either to just write better code up front, or to use on _undirected_ remediation
I always tell POs that if they want X done until Y day, that is going to result in tech debt.
When they are asking later why Z doesn't work properly or why it takes time, I just remind them of that technical debt they've agreed on.
edit: use better words
https://en.wikipedia.org/wiki/Gemba
Gemba means "the actual place" (where value is created).
The core idea behind the Gemba walk is a follows:
> Gemba (aka Walkie Lookie) walks denote the action of going to see the actual process, understand the work, ask questions, and learn. It is also known as one fundamental part of Lean management philosophy.
> Taiichi Ohno, an executive at Toyota, led the development of the concept of the Gemba Walk. The Gemba Walk is an opportunity for staff to stand back from their day-to-day tasks to walk the floor of their workplace to identify wasteful activities. The objective of Gemba Walk is to understand the value stream and its problems rather than review results or make superficial comments.
E.g. returning a NotaimplementedError may have a a FIXME / TODO next to it to help me find such places in code quickly.
True technical debt is stuff like 'lets just use a csv file for now and pretens its a proper solution and totally not give any thought on whether we have an appropriate infrastructure to accommodate a proper database that we will inevitably need down the line'.
For good reason; the more TODOs and FIXMEs a codebase contains, the less impactfull they are. We had so many TODOs no-one every batted an eye when seeing them, and they never got fixed. It just made you feel bad.
Also a lot of TODO's expressed opinions about the "ideal" scenario, but in practice most of those TODOs were for code that was "fine" or "good enough". In those scenarios it's better to omit the TODO/FIXME keyword and just describe the design decision that was made.
If you have mostly "fine" and "good enough" code I fully understand why people want to put TODOs and FIXMEs everywhere; as the code is likely a technical debt catastrophe all over the place. (Because that's the usually meaning behind wording like "good enough".)
If nothing of this got ever fixed this means the devs never got time to do that. Just the next red flag…
If a manager would tell me I shouldn't put TODOs / FIXMEs into the code I would be on my way out pretty quickly. As this means they constantly produce tech dept but actually want to hide this fact.
Often however those ideas are also wrong perfectionalism and can lead to more bug potential or complexity to understand, like trying to create the perfect abstraction while you only have n=1 use cases, especially the younger devs are prone of. (Or one dev believing this part should be written like that and that, while 5 other devs would disagree, that's the best.)
TODOs should be really for stuff that is still missing and needed, because of a requirement or some real ugly workaround taken and tech debt created, and not just because a dev has feelings about some code. Similar FIXME also only for really broken stuff or corner cases one for whatever reason cannot fix right now..
> If a manager would tell me I shouldn't put TODOs / FIXMEs into the code I would be on my way out pretty quickly. As this means they constantly produce tech dept but actually want to hide this fact.
Noone said that (just not too many), it is not black and white. Often what one sees as tech debt is only wrong perfectionalism, that in the end doesn't help. If you need to place TODOs and FIXMEs often, then why not do them right now? If they are really necessary and you need to put them often like once a day, then actually I would question the place.
Placing TODOs and FIXMEs is right where necessary, but please not once a day for something that maybe could just be done better.
I btw like the "No TODO without a ticket" rules, they help with this problem automatically :)
If the TODO is not really "to do" before release, I change the name to: PERF (performance), MAYDO (nice to have) or FUTURE. If it doesn't break the user's story then it's not a real "to do".
I also leave comments for obvious good ideas that don't work, in case someone has the same idea later.
Debt is a part of life as tradeoffs are made every day. But sleep-walking into it is the real risk.
Knowing you have it, or are going to incur (more) of it is half the battle.
Getting space and buy-in to pay it down, the other half.