Three Kinds of Good Tech Debt
engineering.squarespace.com
engineering.squarespace.com
What is good debt? It's taking money from someone, promising to pay them back more money in the future, and using the original money to make even more money than that, so both you and the lender profit.
Good tech debt is just the same. It's taking shortcuts now to save time and get something shipped, knowing that it will cost you more time than you're saving now to fix it later. But by shipping now, you ensure there is a later, with a product that's out there and being used, a bigger team and therefore adequate time and then some to fix the debt and keep improving the product. Everyone wins.
That's the theory anyway. I think mostly where it goes awry in the wild is people forgetting the paying it off part. Part of this deal is you have to pay back your lender - they have to win too. It's just easily forgotten because you're the lender, so the incentives get a bit conflicted, and you'll keep giving yourself more time. But if the lender ultimitely doesn't get paid back and loses, you lose, and it's a bad debt.
Which also fits the financial debt analogy perfectly.
Still if you really want to take the analogy further declaring technical bankruptcy could cause business bankruptcy (see Joel Spolsky). You probably want to refinance your debt and get onto a repayment plan (an incremental rewrite) instead.
Technical debt bankruptcy is probably closer to shutting down a service / product with no replacement - the effort has already been spent and you’re walking away from costs without caring for the creditors (customers and engineers alike).
The question of whether technical debt can be zeroed in a technical bankruptcy is who the creditors of technical debt really are - is it the business as a whole or the engineers that maintain this stuff?
To me, the state of technical debt bankruptcy is a business in a zombie state unable to grow to service the debts because of the technical debts keeping them uncompetitive rather than something more fundamentally wrong with the business like product market fit or bad reputation.
Financial debt has a clear quantitative measure.
Tech Debt is an incredibly fuzzy concept, even as a qualitative measure, to the point of being useless for anything other than covering one's ass. We can do better as engineers than resting on such fuzzy thinking.
I like the Go concept of aji [1] but that's too obscure, nobody will get it. Maybe something like "unfinished business" would work? A nebulous term for a nebulous concept.
You will find yourself squirming a little. It will be uncomfortable at first. But if you hold to your guns, you will find that you're forced to speak in much more clear terms about your process and decision-making.
And you will certainly find yourself saying humble things like "I have a sense that this isn't the best solution, but it's all I got right now."
But over time you'll find yourself improving. You'll find yourself saying "I have a sense that this isn't the best solution -- oh, you know what I just realized, this design choice is going make this other requirement over here nearly impossible to implement."
And with this kind of clarity you'll be in a much much better position to navigate the territory ahead and make clear, conscious decisions based on what you know and don't know.
"Tech Debt" is simply a way to dodge -- e.g., dodge the discomfort of not knowing stuff, of holding yourself to clear thinking, of admitting the need to improve...
Surely there are situations in which it's only useful to cover one's ass, but often it's a great way of explicitly understanding specific tradeoffs being made.
Now you have two choices:
1) You select and articulate your assumptions. You select and articulate your requirements. You also may try to capture and articulate your Black Swan list of unknowns and risks. The key thing is that you are clear. Clear what you know and clear about your concerns and risks. You then articulate your solution, holding it accountable to your assumptions, requirements. And then you move forward.
- or -
2) You come up with some solution sketch. You have a general vague notion that it isn't ideal. You're not sure why but it just doesn't feel right. You need a way to psychologically contend with this vagueness. So you give the vagueness a name. The first thought that comes to your head is: "Vague Notion That This Design is Not Very Good". This is not very flattering to your ego. So you think a little bit more and come up with: "Tech Debt". And you like this, because it sounds quite nice. It sounds like you're back in control, that you know what you're doing. You proceed to blame the design deficiencies over time to "Tech Debt". Everyone nods their heads.
What you’re describing with your product is the fruit of unskilled (amateur) labor.
When you're validating an idea, don't worry about making everything perfect. Worry about validating the idea. Once you're more confident that you're actually gonna do it long-term, then it starts being more beneficial to pay off the tech debt.
This happens with financial debt as well. It's the risk that drives interest rates. Financial debt is abandoned (or restructured) when people or companies file for bankruptcy, when homes are foreclosed on, etc. Sure, it might ding your personal credit history in some cases, but financial debt is constantly abandoned at little or no cost to the borrower.
It ignores the stronger debt analogy -- sure, there is a high chance that the idea fails. But if it succeeds and, all else being equal, you have taken on a less tech debt to get there, you will be able to iterate on and grow it more efficiently. So to the extent you're taking on debt, it should be an explicit trade-off to increase your chances of proving out the idea. Which is not the same as "don't worry about bad code at all."
From what I've seen, when it comes to talking about tech debt from an early prototype, it's often an excuse/euphamism to be considerate to the people that wrote it. The truth is often that it's just bad code due to them being inexperienced or lacking the skill or discipline to write better code. Which I don't mean as an insult, it seems like there's a strong inverse correlation between the kind of person who's passionate enough about product over tech, and is creative/naive enough to build and validate a brand new product, and the kind of person who understands enough to architect a great system and make the best tech debt trade-offs.
Would you accept a free rusty Ferrari tomorrow? If it would take $60k in repairs to get into usable form, and probably not that great even if you put in those repairs, which you have no inclination to do, probably not. Especially if you couldn't easily sell it. If you were a broke college student and it was in roadworthy condition, and you had no other way to make it to your job - probably you would.
This captures that it's not really that you're "$60k in the hole" if you accept the free rusty Ferrari. In a sense, yes, you are.
I propose rusty Ferrari as the new, more precise term for technical debt. "We have a lot of rusty Ferraris in our garage" isn't a debt, but I think might capture the meaning. "I don't want a rusty Ferrari in my garage" = I don't want this technical debt. Let's make it work with some rusty Ferraris = let's cobble together a hack we should probably throw away.
what do you guys think?
Whenever I'm moving from one framework to another I find loads of todos I can now delete. Or even if I have a well designed class, if I can find a nice third party library that does the exact same thing, I can now get rid of all that useless code and any debt around it.
Honestly I think calling it technical 'debt' makes programmers unreasonably averse to it, but unfortunately I haven't been able to come up with a better term.
If you have messy code that doesn't need updating, then it doesn't cost a thing to leave it in a bad state. The act of refactoring in these cases is a waste of effort and often does more harm than good by introducing bugs.
I take the YAGNI when it comes to refactoring. Just pay back the stuff if the code actually slows you down in the future instead of doing it in advance.
There is a lot of tech debt you can get away with for free without ever having to pay back.
> Not Fixing All the Edge Cases
We’ve all seen bugs that occur so rarely, and fixing requires destroying some part of the original design architecture. The ability to acknowledge these types of bugs as not worth fixing is liberating.
> Err on the side of building too little because you can always build more later. Build things to be easy to throw away and replace;
I find developers tend to really dislike throw away code or temporary solutions knowing they might break in the future. If the quick hack is well documented, written in a very encapsulated and removable way, who cares about removing it later.
IMHE, it's usually management that feels this way.
This is good way to level up: Learn to love throwing your code away!
Building things to throw away means componentizing your code, that is extra work overall.
No it doesn't. You can really tell who has and hasn't worked at a start-up by the responses to OP.
The word “component” barely means anything at all. Lots of room for interpretation there.
We found a better way to do this - boom, you're gone!
I do dislike coding solutions which I know to be temporary, sometimes it's necessary but it feels wrong if we already know something's not going to be adequate.
That said it's also very bad to future proof (by which a lot of people mean abstract) the code to be able to do everything you can possibly think of that might be needed in future, or even (and I've seen this attempted) things we might possibly want that we don't even know about and can't think of!
I have gone through that dance often enough. It ends up being more time consuming than doing it right in the first place each and every time.
I worked at a company where there was an API and an explicit place for such hack code, for consultants to customize the product. The way it was done, was to ensure such code was a pure leaf.
From the example in the article, if it would take them 2 months to build a robust email delivery system or 2 weeks to put a temporary hack in place, it could be worthwhile to the business to do the hack if it means they can start user-testing 6 weeks earlier.
Also, if they're still in the product-fit stage of their business development, they might find that the product isn't as useful/worthwhile as they thought, and scrap it after 2 weeks testing. This would allow 6 weeks of overall developer effort to be spent on another initiative.
Just because you know that developing throwaway code is waste, doesn't mean it's necessarily a bad business decision.
- be able to switch to other, more important tasks more quickly;
- allow the business to better discover what the actual requirements are while they have an MVP solution in place.
Resources are finite: you only have a limited number of engineers, and the business needs to be moving forward. This approach allows you to do so, while mitigating a lot of risks at the same time.
The costs are not lost: they bring back business value in terms of fast progress, and mitigates risks.
If this has happened, your encapsulation was insufficient or you waited too long to replace the stop gap solution or maybe your documentation wasn't clear about what assumptions made by the throwaway component are safe to make in the main codebase.
> The only time when it can actually be allowed is when it it concerns a feature that is a pure leaf in the system with nothing else built on top of it.
I think the article's description of when stopgap solutions should be used is far better. Yes, a leaf feature is likely to have fewer encapsulation issues, but that is only part of it.
I do. I care very deeply about removing it later.
The problem is that encapsulation is often mistaken as a strong justification for existence, and code that is written tends to justify its existence just by existing.
That's to say that once code is in production, it's scary to delete. It's much less scary to delete a couple of lines of inline code wrapped in a comment that says "this is a hacky solution," because code that isn't encapsulated probably probably isn't used elsewhere. Or if it is, it's copy-pasted.
The scary thing about removing encapsulated / abstracted things, is that oftentimes, other things refer to them, unbeknownst to the original author or their intent.
It's pretty common (especially on larger teams) - the first developer writes up a hacky solution, the second developer abstracts that hack into a module due to some sensibility (abstraction, aesthetics, organization, etc), so now the hack has legitimacy. A third (or often the first developer) sees that the hack is in a module, assumes that the person who put it in a module had more information than when the hack was written and that it's now a first class member of the system. And first class members of a system take thought and time to delete.
If the goal is to grow a piece of software over time (as in many startups), any architecture that relies on "better communication" is doomed to fail, simply because it limits the size of the team that can handle it (and thus growth).
Then your hacky solution goes on the inside, as inline code, and then it's no problem to delete it.
This can also be a part of tackling very complex tasks. To prevent getting overwhelmed when implementing something huge, I sometimes do what I call, "Implementing something stupid." Want to build an MMO? Build something that manages to send sync information to a multiple clients, but ignores auth, security, being cheat-proof, robustness, etc... Then once there's something running, start addressing those issues.
So instead of not fixing all the edge cases, it's not implementing all the edge cases all at once.
If the quick hack is well documented, written in a very encapsulated and removable way, who cares about removing it later.
When it starts causing "lava flow" then it's time to remove it.
That's fine in a system with a small number of quick hacks. In reality, the number of quick hacks grows over time and you end up with a brittle system that has a lot of unanticipated behavior that only a few people understand how to work with. You will eventually have quick hacks that exist to fix bugs in the other quick hacks.
One higher level consideration that's particularly relevant for early-stage startups, is that you often simply don't have time to build the ideal "tech-debt free" solution. Focusing on 80/20 solutions almost necessarily involves adding tech debt. However, if you're mindful about where you take on tech debt, and commit to tapering and then paying it down as your resources grow, then you're more likely to avoid the debt spiral where your development grinds to a halt, due to everything breaking all the time.
One other point I'd make is that a common form of "tech debt" for early-stage startups is simply doing things that don't scale; it's often beneficial to make your domain models flexible so that you can support one-off manual interactions, for example if a new customer appears that requires it. Then automate that thing and constrain the domain actions once you have collected enough use-cases to know what makes sense to forbid.
(Pushing the analogy to a breaking point: Is adding a linter then, doing an audit and depreciating assets for "tech taxes"? Come to think of it "tech taxes" would be a fun alternative term to add to the "tech debt" family. "Hold on, just paying my tech taxes; npm install, npm audit, and npm run lint.")
https://www.youtube.com/watch?v=XakfJ2spb3w
tl;dr: Technical debt can be categorized on 2 axes: Planned vs unplanned, and high interest vs. low interest. Interest is paid down in pain. If you are not experiencing any pain, it's not actually tech debt.
Low interest, planned tech debt (akin to a mortgage) is ok because presumably the cost of the interest is lower than the returns.
High interest, unplanned tech debt (akin to a loan shark, or pawn shop) is best to be avoided, because you're experiencing a lot of pain probably without a clear vision for paying it off. You're just continually paying the interest/experiencing pain.
But in both those cases, the pain is known up front. The discussion here seems to be no one knows what X is, nor Y, but it's expected that Y > X by some large amount that you may or may not have to pay off. Crazy!
I agree that money is much more comprehensible, but there isn't so as stark a difference in knownability as it might initially appear.
Rather than trying to use the tech debt metaphor to actually calculate the interest rate, I think it's more useful at a higher level of abstraction.
When faced with the product decision to do more work now and build something better, or less work now and cut corners, what are the consequences of doing less? Without some way of conveying the long-term costs, your stakeholders will always push you to do the least work possible to tick the box.
The debt analogy is a universally-understood way of conveying the fact that if you cut corners, you'll likely pay interest later in the form of slower development speed. And if you take on too much debt, then all your spend will be going to paying off your interest, and your progress will stall. Non-technical people understand this analogy, and so it's a useful way of conveying the shape of the problem without having to start from first-principles.
It’s also been funny seeing managers try to rebrand tech debt with other business speak like “tech health”. Debt shouldn’t be a bad thing but one that’s anticipated and respected.
Technical debt in your solution is fine. Running a Ponzi scheme in your solution isn't.
Loan consolidation is quite normal in the UK
Because otherwise, what would stop you taking on a loan for X, then paying it off after a while by taking on the same amount from another bank?
Nothing. That's why lenders do credit checks and due diligence.
https://www.moneycrashers.com/responsible-use-0-balance-tran...
A bank's primary business is managing risk. And borrowing from Peter to pay Paul (having previously borrowed from Paul) is a significantly different risk profile than borrowing for investment or even present consumption.
Erm, it very much isn't illegal. At least not in most places.
Taking on fresh debt to pay off other debt is the very definition of a consolidation loan which is perfectly legal in most, possibly all, jurisdictions.
There are limits of course, like not using a loan to pay the deposit on a mortgage, or simply not lending beyond defined means, but these are set by the bank to manage their own exposure to risk rather than limits set by law. Such limits are merely encouraged by law, via regulatory limits on standing capital compared to debt held and so forth.
There's also a distinction between "we made intentional decisions that we need to fix up now" and "We (or people before us) made unintentional decisions and it is in such a bad state that it endangers the health of the business."
Imagine you are building a prototype and you are clever enough to only focus on validating an idea. Having this focus you ignore all best practices and maintainability issues you might tend to out of habit.
What you are likely to end up with is a functioning application that resists change beyond its original goals.
And this is good. It lets you declare technical bankruptcy early enough to avoid painful rewrites on a software that was not meant to exist for long anyways.
More good examples:
1. Technology progress: Long time ago JQuery was the best for UI library. Today you would use React or friends.
2. Company stage. As a early startup you usually prioritize for speed and validating hyphothesis. As you got a significant user base quality matters more.
3. Size of development team. With 5 or less developers having a monolite is good enough. With 50 developers you likely take more advantage of microservice architecture so each team can be decoupled and move faster.
I think it's almost part of most companies' lifecycles that they start with a monolith, and anticipate rebuilding it as microservices if they get traction.
Microservices aren't magic. There still has to be a contract between the services. Microservices are best when they are something which need to be scaled independently of the rest of the system.
Bad architecture can cause urgent issues and it can also impact new feature development. The debt is a barrier that makes new features either take much longer than they should or make them just infeasible from a business standpoint. This can exist in both technical architecture or information architecture (i.e. you chose the wrong model and now it’s too engrained across everything to change).
So, just like this article's opening:
> Financial debt isn’t universally reviled in the same way. Your friend takes out a mortgage to buy a house and what do you say? Congratulations!
Except it is universally reviled, no one wants debt - they just accept debt in reasonable quantities as a way to efficiently get things done.
I think, in particular, the success of not-quite-agile-but-similar business practices has confirmed this, plan carefully and evaluate business needs, then get it done in a way that's fast and leaves you relatively free of debt - don't plan for everything. So don't get stuck in analysis paralysis trying to build the perfect things - and don't just code with no thought of the future... it's a difficult road to follow.
1. Unfinished/unpolished implementations 2. Implementations that are architectural flawed, such that fixing them requires throwing away the existing code.
Issues in (2) are a lot more problematic than issues in (1).
The problem is, the debt metaphor is not being applied correctly. Not doing future work now does not take on responsibility for the future work - so it's not like taking a loan. Doing work in a way that commits the team to extra work that didn't exist before - that's like taking on a loan. For instance, I don't want to do this 80 points of work now, so I'm going to ship the same requirement by doing 40 points of work on a solution that will take another 60 points to get to the original 80 point solution which is the way we know we have to go long term. I just borrowed 20 points on behalf of the team.
> We realized we could build something cheap that we didn’t mind throwing away later—scaffolding—to unblock getting user feedback sooner.
If not, can you say more about the difference? I'm interested.
Is it worth implementing a cache layer in a system? One on hand, you have a better system that can handle more load, on the other, you've spent time that could be spent doing something else. Your systems will almost always have limitations, it's important to understand these and communicate them well. Tech debt should be managed so it doesn't tip over and cause issues. Really good devs and dev managers develop the eye to see when that's about to happen and make sure they convince product that you need to pay that down soon.
This isn't a "Tech Debt" thing. This is a feature/scoping thing. Does my system need to handle 1M concurrent users? If so, support that. If not, don't support it.
But we don't need the absurd concept of "Tech Debt" for effective analysis. We can be specific. We can be clear. Does my system need to support low latency in the face of 1M concurrent users? Yes or no. Now proceed.
"Tech Debt" is such a ridiculous dodge to otherwise avoid real analysis, real work, real clarity.
The answer to "Does my system need to handle 1M concurrent users?" could be "Yes", "No", "Not for the foreseeable future" or even "I don't know".
Given that 'debt' is a way to, in effect, allocate (possible) future resources today, it's pretty clear why it's valuable, especially in the face of uncertainty.
That's a given.
You don't need the notion of "Tech Debt" to deal with that.
It is a complete given at any given time, during any given sprint, that all the features in the queue are ASSUMPTIONS.
That's just a tautology.
"Does my system need to handle 1M concurrent users?"
Maybe you answer Yes or maybe you answer No, but you have to answer something to make decisions about the work ahead of you.
The worst possible place is to be in the fuzzy middle saying fuzzy things like Tech Debt and rolling along fuzzily.
Many teams -- especially teams that love this term "Tech Debt" -- do exactly this.
But they are willfully navigating in the dark and "Tech Debt" is just a term that makes one feel psychologically okay with their ignorance.
But it's euphemism, that's all it is. It does not aid in rational navigation.
The code review articles are good too: https://engineering.squarespace.com/blog/2019/code-review-cu... https://engineering.squarespace.com/blog/2019/code-review-cu...
Financial debt is often repaid simply by bringing in more revenue because of the benefits gained from bringing on the initial debt. The only risk involved in the debt is that there will not be enough money to pay it back later. It is a very predictable and obvious issue.
Technical debt is closer to educated gambling. You are taking a risk that the shortcuts you are taking will not pop up and hurt your company before you fix them. You are essentially saying, "We know this is a problem, but we are estimating that it won't hurt us for the foreseeable future".
The better analogy to technical debt is investing. You are getting a certain percentage increase in productivity(a dividend or other capital gain) for the (hopefully low risk) gamble that your investment will not have a problem and wipe it out or simply cause significant losses.
Developers typically do not like this because when the inevitable few cases of these "investments" turn bad, they tend to be the ones that take on the majority of the blame.
Not being able to pay back money sounds like the same as not being able to fix bugs.
First, If the original code isn't deleted, you've added to the tech debt. Now every new engineer will need to learn two chunks of code, and every existing engineer will need to support both.
Second, unless you've had a major epiphany between the old code and the new code, you're more likely to run into the trap of the first example. Major epiphany's could be, 'wow, we really didn't need all of that extra stuff', or, 'we've duplicated so much code, we should re-organize this all'.
I've been at plenty of companies where you have to understand that the Perl code is really old and these components use that, the Mason rewrite was 80% done and is over there, the Next Gen code was in Ruby these components do that. But now, we're embarking on a new Node/React project to replace all of it, finally going to get rid of the tech debt! (Said with no sarcasm)
In my experience there have been plenty of significant refactorings that simply would not have been done if we didn't allow ourselves to get into this state where there is simultaneously an old and a new way of doing something.
Especially if it's an area that sees frequent addition - it's not helpful to have people constantly adding code in the old way while off to the side people are trying to create a new way and completely finish switching everything over to it before integrating.
Managing risk and balancing other active tasks in the pipeline also push towards integrating smaller pieces over time while multiple ways of doing the same thing exist.
But you need to have organizational continuity, which can be rare when everyone is trying to hop jobs for better salary every couple of years.
I'm with you all the way until the last paragraph which short-changed the conversation a bit. Quoting a few lines to respond below:
>> Build things to be easy to throw away and replace; it’ll make your code more modular.
One valid strategy to manage debt is to declare bankruptcy or amnesty on debt – that is throw away the code (because this code is no longer needed). In domains where this possible, definitely use it. But in most real-world touching software domains this is not likely possible.
>> Good tech debt has clear, well-known limitations. Document these in code comments, READMEs, FAQs, and conversations with the people who’d care.
Doing tech debt accounting in a more formal manner is required to intentionally take debt. In the locking example, it is a design debt (not just a code debt). If the debt isn't document well and then it can become a ticking time bomb to be tripped by a clueless developer later.
Technical debt is a close relative of externalization (I save time, you waste time) and of mistakes that result from incompetence rather than from haste and shortsightedness.
All three sections of the article, instead, describe healthy planned evolution from a decent provisional product to a better one, with solutions that might be rough but not necessarily bad. In other words, a clickbait title for sound (but maybe unrealistically mature) examples of properly pragmatic project and product planning.
@2: Sure, but this it not really technical debt. It is the wise policy of not immediately writing code for the most general case. If one writes several instances of this it would lead to code duplication that would need to be refactored immediately.
@3: Careful with this one. The thing is that the thing you call an edge case might have the more proper name of 'bug'. If you deliberately leave in bugs at the time of writing the code it can lead to very difficult to estimate 'stabilization time' at the point where the code is supposed to go into production but is not quite reliable enough. It becomes somewhat difficult to explain if that 'stabilization time' turns out to be months or even as much as a year.
When it comes to accelerate development in my opinion this is most (80/20) of it.
If we are going to allow the tech debt metaphor to continue, too much of the debt I see is the equivalent of a pay-day loan.
If you want me to stay on topic -- this is the worst wasted time debt Squarespace has to me as a customer :) code wise the product is really solid so their philosophy is obviously good!
There's an implicit assumption there that your friend can afford the purchase. If banks did no credit checks & your friends were impulsive & broke you might try to talk sense into them instead.
Maybe we need to store PM's technical debt for 10 years and perform a credit check when they say "I don't care, just ship it & we'll clean it up later"!
We do say Congratulations! - we say well done on finishing the project you were working on.
I do agree with the part on not putting too much worth on edge cases, however the biggest issue for me with Technical Debt is the increased time it often leads to in further development. So I wouldn't say it's something to completely ignore.
The very concept of Tech Debt, though prevalent, is pernicious, because its a concept that is far, far, far too fuzzy and therefore can -- and is -- always applied as a convenient narrative to explain away mistakes, process deficiencies, bugs, etc rather than a true conceptual problem-solving tool that aids in optimizing the software development process. In short, "Tech Debt" is a euphemism for incompetence.
We can see this play out in this post, which covers "3 kinds of good tech debt.
1) "Scaffolding" ... what the author describes here is good ol' fashion Prototyping, aka Learning. No invocation of a fuzzy concept like "Tech Debt" is needed here. You don't know something, so you go and learn what you need to learn. Learning takes many forms. In this case, it is product understanding -- and this author reaches for prototyping in particular and calls it Scaffolding. Why a new term? Why obfuscate, why fuzzify? Because that's what Tech Debt is good for: fuzzy thinking. If we move to more clarity we can be specific and we can take specific actions to optimize: here, maybe or maybe not prototyping isn't the most efficient way to learn what is needed: often, wireframes shown to customers will suffice. There are plenty of ways to skin this once the framing is clear.
2) "Hardcoding" Again, no fuzziness is needed here. Developers have a fetish for fully flexible and feature-rich subsystems and we tend to call anything else a pejorative like "hard-coding" but that's a psychological problem in us, not a technical or process problem. We already have a clear framework for talking about this and it's called feature Scoping. There are, again, plenty of ways to skin this cat, but the short of it is - if you can understand what features you are interested in with clarity then you can build a system whose flexibility is in accordance. This isn't a fuzzy "Tech Debt" thing. If you only need feature X now and going forward, then build that - and call it something pejorative like "hard coding" if you want: I call it being rational. But bringing in a fuzzy term like "Tech Debt" helps nothing here. Only confuses.
3) "Not Fixing Edge Cases" This one is the worst of all. By using a fuzzy term like "Tech Debt" we're able to completely alleviate our responsibility. And start down a path of extreme cost and pain. Here, he wants to avoid having to implement correctness because it would cost too much. The example is a good one: it reflects a pattern I've seen over and over again. And unfortunately I've seen the same choice this author makes -- and the terrible downstream cost it creates, as well. By avoiding implementing a correct solution (by purposefully shipping a race condition) he is not being clever, he is being irresponsible. If this sounds draconian, let me explain further. Perhaps he finds, as he mentions, that in production this is inconsequential. Okay, great. But first thing to point out is that most likely it will become an issue as the system grows and when are you going to notice that? Are you going to go build monitoring to make sure that this "effective correctness" holds over time? Of course not, that's more work. But even if you are happy with his choice here, there's a bigger issue. He has chosen to build a system with transactional needs on top of an eventually consistent/NoSQL solution. From my experience, it's rare that you have just one feature like his that needs transactional correctness and that is it. Tomorrow, next week, soon--he and his team are going to find out that they need another transactional capability and then they are going to be right back where they started: having to ask a question, do I slack off here, as well? Again -- we've created cost for ourselves by forcing us to ask, should I ship a bug in this case? One might say this is what "Tech Debt" lets you do! It lets kick the can down the road, save you some time now... But I do not buy it. I've seen this exact kind of choice made plenty of times and the can always comes back next week, next month and the tax you pay for not getting right is immense. "Tech Debt" here is simply letting us avoid responsibility, avoid having to understand the needs of our overall system. The cost of getting transactions right/correct is not even that high if you face it squarely and truly make sure you need NoSQL and in which cases. What's happening here is not clever process optimization but inexperience. And if not inexperience, laziness (not the good kind).
The concept of "Tech Debt" is awful. And always ends up being a euphemistic excuse. I've never seen the idea of "Tech Debt" help the actual process and technical quality of things. I've only ever seen it give developers an excuse for inexperience, laziness, etc. I don't mean to denigrate here -- I mean to challenge and ask for us to be more intentional, clear, and frank in our work. "Tech Debut" is fuzzy thinking and fuzzy thinking isn't much good for anything other than excuses.
A developer aiming for technical supremacy for its own sake isn't useful in a real-world business scenario. I've seen enough technically superior software products utterly fail in the market due to poor product-market fit iteration speed.
Tech debt isn't a choice an individual developer should make. Tech leaders at the top of any evolving software based product/service should deeply understand the pros and cons of the concept of tech debt and then decide to use it as a strategy lever.
They will have to make appropriate choices w.r.t people and process that is right for their company to leverage tech debt.
This will reflect in the technical architecture of the systems as well. There will be different layers of the stack where tech debt can be more vs less.
There are automated measure that can keep you informed if the tech debt is exceeding to the point where it is becoming a net liability (vs being a net asset as a strategic lever).
It may be true that many organizations use the term tech debt very fuzzily and don't really give it conscious and serious thought and put effort to structurally leverage it. In this, your rage against may be justified.
For those who are interested to read more on tech debt should visit these links:
https://martinfowler.com/bliki/DesignStaminaHypothesis.html
Please link to a real paper that explores Technical Debt. This is just the link to SEI's home page. I'm looking for clarity, not hand waves.
> A developer aiming for technical supremacy for its own sake isn't useful in a real-world business scenario.
You're straw-manning.
We're not looking for technical supremacy at all. (See my point #2 and my emphasis on brutal practicality.) We are looking for business optimization and all of the pragmatism that comes with that.
My objection is to using a dodgeball term like "Tech Debt", which in no way helps to carve a pragmatic path forward.
It is akin to hand-waving. If you move forward with any degree of efficiency, it will be in spite of your use of the fuzzy notion of "Tech Debt". You can do just as well without this (non-)concept.
I would call this "My Thesis" but that acts as if the burden on me is to justify my objection.
The burden is on whoever introduces this silly "Tech Debt" notion to 1) clarify what exactly it means and how to measure it; and 2) give a model in which the measurement can correlate with an improvement to the software process.
Otherwise you're just talking Santa Claus.
I have yet to see anything close to that kind of clarity when one discusses Tech Debt. There are a dime a dozen posts on "Tech Debt" and if you dig just a little you'll see that it is nearly always used to avoid clear analysis.
To be fair, clear analysis is hard, so I can understand why people give up and take the easy path. But let's call the spade a spade. "Tech Debt" is a dodge. It's not a constructive tool.
Your links are not helping your case either.
Fowler's writing explicitly makes a distinction between quadrants of Tech Debt and the only one he calls "Prudent" is the panic "We must ship now scenario."
If this is the only form of "good" Tech Debt, then my case rests completely.
Because all that is being said then is that "I literally have no idea what situation my code assets are in but I have to ship now, so let's go with whatever's on disk!"
Again, this is fair business. Anything is fair business.
But this is certainly a position of ignorance and not a position of strategic choice. So if can admit that "Tech Debt" is simply "Ignorance", then we are all in agreement.
Admitting ignorance, incidentally, is the only way an experienced practitioner can emerge and reduce the likelihood of being in such positions to begin with.
Matters are even worse. Besides "Tech Debt" being a useless concept. For the individual it creates no clarity, no admission of deficiency, and no incentive or need for growth.
I would highly recommend this book: https://www.oreilly.com/library/view/managing-technical-debt...
Also, read the papers presented at this conference series. https://2019.techdebtconf.org/#program
The book enumerates some examples of failed software projects and calls their failures "Tech Debt".
I have two issues with this.
The first is how fuzzy this still is. This term is thrown over all of these various kinds of failure. And the examples are also quite varied - most are phrased as mere symptoms of failure not root causes, which gives us little insight.
The bigger problem I have is that these examples are retrospective.
If Tech Debt is simply a term for "mistakes I have made", then by all means use it -- obviously in this form it has no strategic/constructive value.
But the way I've seen it typically used is for people to act as if it is akin to real debt, consciously acquired.
"Oh I didn't make this transactional because Tech Debt."
Etc.
And indeed if Tech Debt as a concept is to have any value it should be able to be used this way.
But my objection is that there is no way it can be used as such, because it is far too nebulous a concept to have any navigational benefit.
It gives no helpful path forward. It is purely a euphemism for mistakes. That's how I've seen it used universally, at least.
And if that was some day no longer true – if they realized later that they really did need "transactional correctness", then it still seems like they may have made the correct decision in incurring technical debt, especially given that the author mentioned that the limit was never exceeded in production.
If that system did in fact require a hard limit then they probably would have focused on implementing "transactional correctness" as that would have then been a core feature of the system.
I can certainly sympathize with your comments in this thread. But all of your examples rather seem to confirm the utility of the analogy/metaphor as it's certainly, clearly, true that some people incur too much technical debt and are never able to 'pay it off' later when it comes due. That's not a knock against the analogy but instead a mark in its favor – debt should be taken on mindfully and carefully.
Contrary to what seems to have been your experience, I've seen technical debt incurred to great benefit.
Sure, fine, the PM can give this as the requirement and all can go ahead. But you're gonna have a crap time trying to analyze and test in staging, making sense of load testing scenarios, validating this feature in anyway, etc.
And I think you know this, or you intuit that this is not very good but you're not quite sure why so you say hmm let's slap a label on this that effectively covers my spidey sense that it is crap even though I can't quite articulate why. What should we call that... hmm... oh, let's call it:
"Tech Debt."
This is precisely how "Tech Debt" is wielded. It's a fuzzy term to apply to a fuzzy understanding of our design choices.
And there you might say, Great, that's it, that's what Tech Debt is. But I can guarantee you -- from long experience -- that not understanding the consequences of your design choices means that you have NO IDEA the cost of that misunderstanding. It could be 10 units, it could be 1000 units, it could be 10000.
"Tech Debt" is akin to saying "I don't know." This is a big reason why the concept is useless. It's fuzzy, it's akin to saying "I don't know" and therefore there is no way to quantify the impact.
And not being able to quantify the impact means that the analogy to other forms fo debt (eg, financial) is completely disjoint and unhelpful.
Bad design choices cost you over and over again. Universally when someone has called Tech Debt on something, it almost inevitably reflects their suspicion that something is not quite right about the design but their inability to say what and quantify the cost. Otherwise, you simply would not need such a fuzzy term; you'd have a much clearer analysis and articulation of your situation.
> Contrary to what seems to have been your experience, I've seen technical debt incurred to great benefit.
Write a paper then and show me the money, so to speak. Quantify this. Because the only "great benefit" I've seen of the term "Tech Debt" is to the ego of the individual or team that invokes the fuzzy term.
Lots of terms – really, every term – is fuzzy at some resolution and thus all understanding is inevitably fuzzy too.
Empirically, you seem to be almost entirely wrong about people not understanding the consequences of their choices. It's absolutely true that sometimes people are wrong about the consequences, but that's true even with financial debt – maybe the interest payments on some debt that should have reasonable end up being the straw that broke the financial back of the organization.
> Bad design choices cost you over and over again.
In the sense in which you seem to mean this, this is a ridiculous statement because every design that's not perfect, and none is, must then be a "bad design choice". That's just not true in any reasonable interpretation.
It's also unclear why invoking 'tech debt' precludes a clear analysis or articulation of the relevant situation. And regardless of whether anything is quantified, it's also inevitably 'fuzzy' too. Why would quantification itself be good?
Take the first example from the article – implementing a 'bad' version of the component to actually send emails. They did this, consciously, because they wanted to test the editor component first, before implementing the more robust email-sender version. And they did supposedly quantify a lot of that 'technical debt' – they estimated how much time they'd need to implement the initial throwaway code and how much time doing that would save in terms of them being able to more quickly test the editor. And they seemed to have a very clear idea about what exactly was wrong with the initial version and thus an approximate idea of the costs of not eventually replacing it with a better one later.
> Write a paper then and show me the money, so to speak. Quantify this. Because the only "great benefit" I've seen of the term "Tech Debt" is to the ego of the individual or team that invokes the fuzzy term.
The blog post for this very thread is a good example of the great benefits of thinking in terms of technical debt! You should write a version of the same blog post, using all of the same examples, and clearly indicate why using 'technical debt' is pernicious and furthermore how exactly you'd have dealt with all of the same issues and considerations mentioned.
>In the sense in which you seem to mean this, this is a ridiculous statement
The point is to acknowledge the Cost. There is tremendous cost in design choices. No design choice will be perfect. Nobody is perfect.
The whole point is to optimize. To minimize bad design, to minimize Cost. That's the game.
And my whole point is that by using a fuzzy term like Tech Debt you are completely side-stepping the game. You are completely avoiding a best-effort at solving the optimization problem, so to speak, by waving your hand and saying Tech Debt something.
We already have clear terms for dealing with problems, solutions, axioms, risks, etc. There is already a clear vocabulary for problem-sovling that includes all the pragmatic real-world admission of imperfection.
But by avoiding these clear terms in favor of an umbrella kitchen sink, we are most definitely opting out of optimizing and into ouiji board mode. Which is fine for many people and the euphemism "Tech Debt" makes them feel good about this. But that's all it's doing; anyone leaning on "Tech Debt" as a concept is operating at sub-sub-optimal productivity.
> You should write a version of the same blog post...
In my original commment I broke it down point by point. 1) and 2) do not need a new term. There are already clear terms for these strategies, Prototyping and Scoping -- and these have much much clearer meanings than an umbrella term like Tech Debt.
So then we're left with 3). And I try to demonstrate the flawed reasoning behind this choice, as the author explains himself.
I'll take another stab using your statement:
> And, based on their subsequent observations, there was no practically realizable situation that would result in more than 10 items, let alone 100.
This is the famous last words that I've seen over and over again.
The simple fact of the matter is you will see 100 in some environment. Perhaps six months from now when you decide to load test your system in staging. Etc. And you are in no position to say how much that will cost you at that point.
But what I can tell you is that if you have a professional on your team, you can solve this OP's problem in very little time right out of the gate. You won't even need to have to try to do this impossible calculus.
Tech debt is such a general term it does more harm than good, and is often abused by people that have no clue what their actual tech debt may be..
It also implies that someone knew what the price tag was in the first place to accrue the debt, which they most likely didn't. No one knows what needs to be built or how to build it beforehand, except maybe NASA, this evolves along the way.
But the only reason their approach to the "11 item" edge case works is that they manage their own data. So they can monitor and might potentially cull manually if it became necessary (eg. What if the race condition caused more than 11 or a future codechange elsewhere stumbles on it).
In a non-cloud product where the cost is higher for a vendor to come verify and fix data, this wouldn't fly.
To me, technical debt is associated with a sense of regret. The example in the quote above doesn't have that, perhaps that is what makes it the good kind.
Time to market, for one. If you get the feature out faster to your customer, eating some debt can be worthwhile. This article is merely providing a structured way to think about the debt you're taking on.
So code that you can propose may be technical dept that make your Series A is good debt.
No canary? Tsk tsk.
Just fix it. Repeat until completion.