Of Debt and Decisions
monkeysnatchbanana.com
monkeysnatchbanana.com
The bigger problem with technical debt is that it limits or slows your ability to do things. With real debt you have to make monthly payments and that reduces the total amount of cash you have to spend, which reduces your flexibility. With technical debt you "make a payment" whenever the speed you can make changes is reduced due to bad technical decisions of the past.
For example, if you don't have tests then you can still change the code, but you have to spend a great deal longer doing manual testing. When the inevitable bug slips though, you have to spend the time to fix it. Since it's often not catastrophic, non-technical management will be willing to pay a known ongoing cost instead of an large, unknown one-time cost to remove the problem altogether.
At my last job we were so mired in technical debt that progress had slowed to a near standstill. It was a 20-year-old codebase running on an IBM AS/400, mostly written in RPG (which is sort of like assembly language but for databases.) Field and table names were limited to 11 characters. When they migrated the product to the web, they kept the RPG code but added in a bunch of perl. When DB2 couldn't keep up, they started adding new data to MySQL.
In all, there were 500 thousand lines of code in the system. Adding a field to the most important table was a 20 hour process, including a late night or two to issue some commands while the system was doing its nightly maintenance. There were five programmers while I was there, and the majority of our time was spend keeping the system going. There was never time allocated for cleaning up the existing system, so instead we'd just try and add features without making any schema changes (or adding fields in MySQL instead, as I mentioned before).
Whenever a practice becomes established as a "best practice", such as automated testing, then the next question that arises is "Why isn't everyone doing this to the max?" And folks that don't follow best practice generally feel guilty and keep quiet. But those who follow fashions in tech tend to underestimate the opportunity costs of doing so.
This is the moral of the above story, and it is a good one:
"Hopefully you are horrified by all things we didn't do that are now considered best practices. No reproducible builds, lack of automated testing, creating and then repeatedly sidestepping technical debt. For a long time, that is how I told these stories: as horror stories that came complete with hand gesticulations to emphasize the "can you believe I had to do that?" nature of it all. Here's the thing though: we were making money. We were growing and supporting ourselves."
I spent this winter working for a startup that had the opposite problem of most startups. The CTO was a serious Pythonista, and he wanted to ensure that all the code was written was of high quality. He wanted to avoid the build up of technical debt. There were 9 programmers on the team, using Python on the backend and AngularJS on the frontend. And he felt they could invent a new way forward, in terms of combining Python and Angular. And so the team was reinventing the whole Python stack: they had their own serialization libraries, their own database libraries, their own web server, their own authentication system. Adding each new feature lead to very long meetings in which people were encouraged to be thoughtful -- everyone was suppose to think about the long term consequences of whatever design was chosen. In some ways, it was a careful style, but it had a cost: the 9 developers spent more than 50% of their time on infrastructure questions that had already been solved elsewhere. In terms of meeting the real needs of this small startup, the startup might has well have had a team of 4 developers -- the money paid to 5 of the developers was, in a sense, an act of charity being offered to the Python community.
We've all ready endless stories about startups that cut corners, ran up technical debt, and eventually succeeded. We read much less often about startups that doing everything perfectly and then become successful. And that suggests that, for startups, cutting corners is the more adaptive path.
I worked one place that wrote their own build tool (instead of using Make or Visual Studio). They wrote their own templating engine, instead of using PHP or ASP. They had their own version of MFC and String library, with everything implemented slightly weird, so I had look at the source whenever I needed to know how it actually worked. It was nuts. They spent a year writing their own tools for everything, which was a year they didn't spend working on the product.
To continue the "debt" metaphor, when your product dies, it debt dies with it (or if you switch technologies, you might get a 'debt relief').
So for the thousands of startups and products that disappear or get pivoted before they can mature - it might be worth it to bet on not paying technical debt.
Technical mass might be better. The code keeps accreting, developer efforts orbit around it in ever-decreasing circles, and at some point it becomes impossible to escape from it.
But is this really a technical problem, or a documentation and specification problem? If there's no separate formal model for the project, the code is the documentation, and that's never a good place to be.
There's no free lunch here. When you do this you either pay off some of the debt by removing some value from the product, or you spend ages just reaching "value pairity" at which point you've likely accrued a whole host of shiny new technical debt.
Per the top commenter (jewel), I have to agree that this article misses the most important chunk of the metaphor - interest. Any time you have to work around limits imposed by architecture/technology/coding decisions, you're making an interest payment. Worse yet, side-stepping like the author describes is often like paying off one credit card with another - the premium compounds which dials up the interest payments. All of these sorts of things distract from delivering value and serve to further increase scheduling risk.
Point being, the author is correct that the case of having to make a large pay-off in order to move forward is quite rare. However, that doesn't mean that you should disregard good project planning and discipline. As you point out, there are cases when you should take technical debt deliberately as part of some tactical decision such as when you are just getting an MVP together, or when you need to deliver immediately in order to make payroll. However every project accrues technical debt which was not deliberate as well.
Regardless of the cause, you'll find that just like real loans, payments against the principle amortize best when they're made early. That said, it doesn't make sense to pay off a loan when doing so might cause you to miss out on some bigger opportunity, like a huge sale, or perhaps simply not starving.