The Way We Look at Technical Debt Is Wrong
bigeng.io
bigeng.io
Instead of digging my heels in on not giving such PMs everything they want, I try to offer alternatives and put the choice on them: they can have this feature today and in 6 months we grind to a halt because we can't fit in anything new, or we slow down just half a beat and do it right the first time. "Up to you, buddy," I say.
It happens with jr. devs, too. They want to use a singleton to solve a problem, or they want to forgo using any sort of referential constraints in their schemas.
Of course, the PMs and jr. devs always choose the shortcuts. I help them both pick up the pieces afterwards, but I never work overtime for it. The point is to make it clear that nobody other than themselves is responsible for their own mistakes. They led us into the mess, they can lead us out.
After that, everyone seems to get along a lot better and everyone seems super keen on having discussions about the "right" way to do things and how to fit things into the current state of the project correctly.
Buy some daisies and chocolate for the PMs you work with... because here's how that would go over with the ones I've had:
PM: "Yes! Cut all corners and get it done today!" ...6 Months go by... PM: "I came up with a new idea for that! Today, please!" DEV: "But remember..." PM:"Today, thanks!"
So when you're told to cut corners today, then in 6 months you can't make magic happen to grant their every whim, you have to let it blow up. They want to be the leader. They are responsible for the failings of the team.
You have to play chicken with them. Because every time you capitulate, all they know is "I got what I wanted". They don't care how much you complained or how many hours off the clock you worked. And honestly, if you tell them it can't be done overnight, and then you do it overnight, you're a liar and they won't trust your judgement ever again. When you say, "can't do it", you have to not do it. Don't make magic happen.
Or just don't work at such places. I have thankfully been out of that bullshit for three years running now.
As you say, don't make magic happen. It doesn't matter how cool it would be if, or how you think you'll show them your ideas are better; when you say "it cannot be done" don't go and try and do it. Don't let the manager bully you and don't let them try and get another developer to tackle the impossible.
I've had that happen to me where I said it will take X amount of time to do it so let's not do it till next sprint. Then the manager went around my back when I was gone for a day and gave the task to another dev. The dev, who caused the mess, of course was able to fix it within an hour. Why did I say X amount of time? To cleanup and make sure the bug stays fixed.
When you work with other developers, actually work together. Stand up for them, don't make bad comments about them, and always stick together when you're asked to do the impossible.
While you didn't specify what your "X" was, it sounds like the manager (and the company) got a better outcome by going around you and it was far from being "asked to do the impossible".
On a long enough timeline, the better outcome turns into multiple bugs.
That's the everyday misconception. Nobody produces wealth just by typing at a keyboard. Products don't design and sell themselves and it's arrogant to pretend that product people and salespeople aren't a huge part of the value creation process in tech.
There are of course developers who are capable of identifying a market opportunity, conceptualizing a product to exploit the opportunity, building the actual product and getting it into market successfully.
But if this is common, why are so many talented developers sitting at desks in open office spaces working 10-12 hours a day for low six figure salaries?
Okay you've got a point, it is fairly arrogant; on the other hand, any sort of automation or small script or product is valuable enough that we don't need to be subservient.
> But if this is common, why are so many talented developers sitting at desks in open office spaces working 10-12 hours a day for low six figure salaries?
That's what I'm asking, why are talented developers allowing themselves to be subservient? That's why I'm wondering why technical managers all the way up to the CTO level are subservient to other departments when it's the product based on technology that determines the success of the company in the end?
And actually, your idea is optimistic; I wouldn't mind low six figures for 10-12 hours a day ;) maybe that's why we're so subservient, we look at our peers in other industries and they're making much less cash money and it's easy to see that we're lucky to be working in a nice air-conditioned office working with something we're typically passionate about.
Answer: trying to get a real product to market successfully requires a lot more than small scripts that automate processes.
It's not that automation and the like doesn't have value, but ask the average developer to get on the phone and sell something, or to have a meaningful conversation with a customer, and you might start to understand why other people in an organization (salespeople, product managers, etc.) have some sway too.
If you simply put a bunch of talented developers in a room, chances are you're not going to end up with a successful business.
Quite, yet this seems to be how many first-time startups operate.
There are a lot of terrible PMs out there, but there are also a lot of really good ones, and you learn to suss out which is which when you're interviewing for a new team. If you read Rands in Repose and start wishing your PM actually thought and cared about the things he talks about, maybe you need to find a new job.
>PM: "Yes! Cut all corners and get it done today!"
You: Can I get that in writing, please?
Generally I agree with you on the principle of using communication to help people learn from experience. I wrote about it here: https://medium.com/@brlewis/fighting-technical-debt-in-an-ag...
Even worse, if you're then not equally technical as a manager you will never know whether you've made a solid choice by aligning with or against the engineer's view, and even worse your engineers will quickly take on a "see, I told you so" stance should things work out not perfectly under an agile model.
Edit: I should add that I am an engineer myself.
This is a spot where I'm definitely in line with the Scrum position: If it's really causing problems, then the developers should have no problem making a reasonable business case for fixing them. If working on it's really causing that much pain, then the developers will figure out a way to quietly fix it in a way that doesn't (and needn't) vex the product owner. If neither of those is the case then you best let sleeping bears lie.
(I'm an engineer too.)
Contrarian anecdotes aside, the reason everyone knows the phrase "technical debt" is because SO MANY of us see projects wrecked by greedily demanding many features and changes as fast as possible in a continuous crunch mode, without considering long-term costs like defect rate or turnaround time on bugfix and feature implementations... or developer burnout.
It's better not to assume that developers are blathering when they ask for formal permission to carry out QA or maintenance tasks or upgrades, if they are the people who know what the work looks like on the ground level. If you are putting lots of time pressure on developers and implying that heads will roll if aggressive deadlines are missed, it doesn't make sense also to leave it up to those developers to magically find the time to fix lurking internal time-bombs.
This makes products worse, it makes developers' lives worse and developers are stuck with the blame for something they can't control. Maybe as a PM or whatever that suits you, to dump all the risk in the laps of your developers. But if you are yourself an engineer you should not make yourself part of this problem.
Similarly, living in continuous crunch mode is something I let myself get bamboozled into quite a bit in my younger days. I bake time to leave the camp cleaner than I found it into my effort estimates, and I know how not to let myself be pushed past my limits. If I worked for a place that couldn't respect that, I'd also be looking for a new job pronto.
I hope that you were being hyperbolic for rhetorical effect in your post, but if your work environment is really like the situations you were describing then I would also encourage you to be looking for a new job pronto. Life's too short to waste on that kind of stress.
It's totally possible to contain horrible sins behind a clean API. I've done horrible things, but those things don't leak all over the whole project.
On the other hand, i worked at the same place for a long time. I got to see the benefits and consequences of my mistakes and successes over years. I do think i burned out. It took me a long time to learn how to contain nuclear waste.
In the end, it's easier to just work someplace that doesn't put crazy demands on developers. There's a continuum of perfection <-> functionality. unfortunately for new developers, it takes a while to figure out where you lie on that line. Finding a place that's close to your preference is really best.
It's kind of like driving. Everyone slower is a Sunday driver, everyone faster is a lunatic with a death wish. I'm happy with homogeneous drivers - too far from the mark and i get nervous. I'd bet it's the same for pretty much everyone else as well.
Good engineers will recognize when it's "good enough". However, that requires biz folks that can provide useful feedback and actually get things in front of customers and get paid. Because, honestly, if we're like three extra features in, have no validation, and are cutting corners, you're damned right I'm going to put on the brakes and redirect the effort toward paying off technical debt.
Because, and this can't be overstated enough, whether or not the business fails in six months is not something we engineers can control--but I'll be damned if I'm going to come in and keep sinking my dwindling life force on a fucking technical lemon. If your biz folks can't establish trust in their abilities, then you might as well hone your craft.
EDIT:
Large amounts of technical debt are a sign of either rapid growth or bad management. If the company is growing, we can overlook the debt every time we, say, waste 30 minutes waiting on a build or fixing a weird nested CSS bug or some other damn fool thing. If the company isn't, then we lose respect for everyone above us in the technical leadership, because they let--by deliberate neglect!--the situation get so bad.
Some subsystems don't change much because they already do what is needed and aren't broken. Sure, the code could be improved, but it's not important enough to do (or at least, to do now).
Other subsystems don't change much because they are scary to change. If the system is brittle and poorly designed, engineers may have very good reason to be hesitant about changing it, and instead it is worked around.
So this latter case behaves very much like "technical debt" . This system costs you engineering time every time you even think about touching it. It's hard to reason about, it's hard to change, and over time these problems get worse if you introduce workarounds in other parts of the system, because now they are worse and harder to work with.
This is a very real cost.
Those workarounds that get created instead of changing the scary thing put me in mind of the "scar tissue" metaphor that SapphireSun proposed elsewhere in this thread.
I think that it often is best thought of as "real debt" in that it costs you time and attention, and (at least some of the time) that cost compounds over time. In other words, some of your engineering spend is just going to service this debt, not produce anything useful. That's a very real opportunity cost.
Speed follows quality.
If you're doing your planning and analysis right, you should hope to have identified this... so in your example, hopefully that subsystem went up in priority because it was identified as a place for gain.
I've definitely seen startups that have almost found a good idea, but can't tweak it, scale it, or make it stable, because they're overwhelmed by technical debt and even simple changes have become engineering death marches.
Letting your back-end code turn into a complicated, disgusting mess means that adding critical features may become a multi-week nightmare. Which is then followed by a series of additional multi-week nightmares for each successive feature.
Now, this isn't to say that obsessive code-polishing is a good idea, either. But startups require rapid iteration, and you can't iterate especially rapidly when your code is complicated, badly organized, and poorly tested.
Unfortunately some people don't realize that in many situations not making a decision is itself a decision, and so they don't always notice it when it happens. That can be anything from the ones you mentioned, like testing and tools, to authentication, auditing, localization, robust error handling, resource/memory leaks, or monolithic designs that prevent scaling.
Accepting large amounts of technical debt enables you to deliver faster if your initial assumptions (both about what you want and how to achieve it) are correct, but slows you down if any of those turn out to be wrong.
OTOH, it might offset that slowness in whole or in part that by speeding up your ability to determine that they are wrong.
I also don't believe it's possible to pick the 'important' areas of code a priori and make only those areas free of technical debt. Doing good work is an attitude, a mind set, and something to practice - if you practice making poor quality code, it's going to affect you across your work, not just in the places that you do it.
I think that, by being professionals and putting aside the 'hack it out' mentality, we can improve not just the code that we're working on, but our mindset towards it. How many codebases have 'that scary part' that adds background anxiety to the people that work on it? Is that something you want to invite into your life on an ongoing basis?
Especially for things as important as integrations into external APIs - sure, they may not change much, but in my experience you really want to make sure they're done right when they're how you deliver your product to customers. Given how error-and-failure-prone many external APIs are, I expect a lot of work there just to robustly handle the other end of your integration being flaky.
I think that quality is something you can't 'bolt on' later. It has to be built in, top to bottom, throughout your organization, code base, and personal skills.
And you may pick the latter, but what if management wants to make a quick prototype and turn that into a product later? At that point you'll still be having the "technical debt" talk with your manager.
That said, I'm not implying that you should slow down, just do the best possible work at all times. Don't do a crappy job just because you're time-limited - we all only have 24 hours a day, and there's always more work than can reasonably fit in. The moment you start going "I am going to lower my standards right now", you've started a trend.
I think it's interesting that people are always saying "hire the best possible people you can", and then saying "but don't ask them to do the best possible work they can do".
And then there's solution number 10, which requires the programmer to invent a new programming language, and requires at least a couple of months to develop. "You hired the best possible person for this job, so let me do the best possible job".
I just bet most people here wished they could always deliver the best possible code.
A good programmer will try and improve code the code incrementally, and won't bother trying to sell anything.
I am sure that most product/project managers want feature as soon as they can think of it
1 week today to do it properly, a couple of hours a month from now to tweak it, a couple of more hours 6 months later to add a new tested feature,
vs,
1 hour today to hack it in, 3 hours on friday to rework it, 1/2 a day next week to debug a problem, and hour the next day to fix your fix ... plus the entire following day when that didn't work properly
Followed by 1 day to do a "proper rewrite" in a month, to add the tweak that should have been easy but breaks some of your assumptions plus 2 days to debug why something doesn't work properly anymore interacting with another subsystem.
Followed by someone else spending most of a week debugging a subtle bug you introduced by stepping on some state "nobody was using anyway"
Followed by 6months later when the new feature is needed someone looking at all this and throwing up their hands because they can't understand what it's doing. At which point they say... "screw it, I can re-write this in a couple of hours".
Lather, rinse, repeat.
Most of the time, the "fast way" is pretty reliably going to cost you more time. Just not more time today.
B) You can spend an hour and a half instead of an hour and include one test and make some incremental improvements as you go.
I agree with all of your points and just wanted to add one more thing that doesn't seem to be addressed much. The author mentioned technical debt didn't have an "interest rate", but it absolutely does. As soon as you start to add debt to a particular part of the code base, it must be paid back immediately the next time you or someone else has to work on that same section. This may genuinely not matter for a one-off script, but if it's code that will ever be worked on again it can start a snowball effect that causes the interest to compound and usually manifests itself as code that's buggy, unstable, and hard to work on.
Financial debt has an interest rate, but you take it out because you believe the return on the proceeds raised adds more value than the cost.
Technical debt is meant to (ideally) be the same. As long as it's outstanding and compounding, it should be because your return on business value will be lower if you "pay it back", then if you do not and spend time/resources elsewhere.
if your discount rate is 30%+ per annum, this makes for some counter-intuitive conclusions about when it's smart or not to "finance" with technical debt.
Allow the page to break if the user's browser does not support websockets If you know that 95% of your target audience uses modern browsers this seems like a reasonable compromise.
Don't write enough unit tests Maybe you're using a library that is difficult to mock. Maybe most of your logic is happening at the db layer. Eventually you may want to thoroughly test this piece of code, but it's a ton of work to mock a database in a meaningful way.
Write a monolithic service instead of a modular one While it's bad to let a single service grow too large, when you're starting out, it can be helpful to write it all out, and _then_ see how it can be broken apart. (It's often hard to foresee everything at the design stage.)
Finally, a lot of code ends up being never used. Nobody cares about well-factored well-tested code that isn't used. Sometimes a dirty version is the quickest way to tell if a project makes sense at all.
At this point, your design is already screwed and borderline-unmaintainable, so yeah, I guess unit tests won't help much. But when people talk about Doing it Right, usually they mean having a codebase that's loosely coupled enough that tests are relatively simple to write.
My opinion is that whether it's programming or general white-collar work, most cases of doing it "the right way" vs. "the dirty way" tend to be about investing the "certainty" of lost time immediately for the "promise" of savings.
For some things, the tradeoff is pretty decent (taking time to improve core knowledge or relevant skills that are related to goals).
But for a quickly growing company, how many situations are there where a "hack it out" fix that costs time in the long run is a net win in opportunity cost and hence long-term value?
If a shortcut that will cause you headaches in 9-18 months also allows you to add critical features at an accelerated pace for the next 6 months, is it: (1) inefficient because you net lose man-hours or (2) efficient (a) maximizes your near-term growth trajectory, (b) thereby increasing potential access to resources and hence long-term growth
To be clear, I do agree with your statements about shortcuts usually not saving time especially when you deal with teams/setting standards.
My point is more that while time saved/lost is easier to mentally calculate, it alone is not really what's driving value in many situations.
There's also the inherent problem of using qualitative terms that mean different things to different people.
But hey, that's part of the fun on the internet.
"If we don't get traction/product out the door now, we won't be able to raise our next round, so we have to get this product out the door yesterday to de-risk the whole company falling apart when we run out of money in a few months." Then later you raise your round and do the same thing for the next round. Eventually, of course, it will catch up with you and slow key progress. If you're unlucky that leads to a down round and you're all screwed.
I see that as being a larger risk than competitors moving faster in the sort term (them moving faster short term may mean you move faster long term). Higher burn rates mean you have a clearer deadline for fundraising, means you're racing the clock, means you're incentivized to take on technical debt. I'd love to have a look at what really tends to kill companies more often, but it's just anecdotes and speculation from me today.
I would point out though that in the financing-focused case you gave, although you have a definitive slow down from technical debt, it's still an open question whether or not it's actually a net slowdown vs the "status quo" scenario.
In practice of course, every individual company, with different teams and different contexts, has to find the right "cognitive bias" for themselves.
Everyone thinks they can pay after the release. If you cut those corners hard, though, you'll often regret it before your release. Cut harder still, and you won't even make it to the end-of-sprint demo before there's a knock at the door.
I'd disagree, I think all technical debt accrues interest (similar to financial debt), but the interest rate is different. As the example about a usps api demonstrated, that has a near 0% interest rate. Of course if you were ever to need to update it (maybe usps updated its api) you'd get hit. A bad domain model has a high interest rate. All the ideas are the same though, i'm just being persnickety :D
In addition, the term "debt" also applies to moral aspects, and not just financial ones. A builder may come back and improve something not because it's good for the employer or has financial benefits, but because of a belief that quality is good for the soul of the builder. That is also an aspect of technical debt.
Regarding the original article, it says tech companies are "successful because they got a product to market fast and at the right time". The "fast" is incorrect. Companies are successful if they are at the right time. If a company (like GO Corporation with pen-based computing) is fast but too early, they they are not successful. If a company (like Microsoft with Excel) is slow but eventually gets there, then they are successful.
"Fast" is therefore irrelevant in the face of "the right time." It can be that the right time is "as soon as possible", but that's not intrinsic to success.
Excel was successful because Lotus decided 123 had so much technical debt it would be a good idea to "upgrade" the old assembler code to C. This took at least a year longer than planned, and introduced a lot of bugs and incompatibilities.
Meanwhile Windows had arrived, and the planned Windows rewrite of 123 never happened as scheduled.
So it wasn't so much that technical debt killed 123. But poor management of technical debt certainly did.
Effectively the C rewrite was a complete waste of time, and Lotus should have aimed for a Windows release as soon as they could.
It's nice to be able to say this with hindsight, but it was doubtless much less obvious at the time, when Windows was barely considered a mediocre visual DOS shell, and certainly not a serious OS.
Bottom line - you have to be smart, prescient, and a little lucky to manage technical debt. It's not just about getting the code right - it's making decisions with educated guesses about where you may be a year or two from now.
And that's hard.
I always enjoy Ward Cunningham's discussion of this: http://c2.com/cgi/wiki?WardExplainsDebtMetaphor (Ward first used the metaphor)
Specifically, he is NOT talking about 'dirty code' or just badly written code, that's taking shortcuts and I believe it is never a good idea in software. We're talking about building good code but doing so before we fully understand the problem, often with the aim of learning more about the problem, especially from user feedback.
What I take Ward's discussion to mean is that "technical debt" is accrued when writing code for a problem you do not YET completely understand (e.g. almost all 'first pass' code). The debt occurs when your code's existing understanding is static (because it is in source control), and your teams understanding grows. The difference is your debt. You can continue building on the code's not-quite-right understanding for the moment which is carrying the debt forward and paying the costs of fighting a not-quite-right model, but at some point it becomes worth while to re-factor based on what your team has learned about your problem, and thus pay off the debt.
But apart form that, absolutely. Technical debt in the article isn't the technical debt that was originally coined.
I still think the common meaning of technical debt is just as important though. Code that can't be taken forward, for whatever reason, costs its owner.
And it wasn't just crappy web apps or throwaway MVP stuff. The first one was in medical systems, where the suits figured we had a 6 month lead on our closest competitor, the other was in consumer electronics (navigation devices), in the time that competition was still stiff in that sector.
In both cases it was decided that releasing something was more important, knowing full well that we had to pay the cost later. Of course not everyone agreed with it, since no one likes having to work with legacy crap, and everyone and their mother always wants to refactor all the things, but guess what: code quality is not the goal of your company, making money is.
> medical systems
Oh no.
Most software, especially middleware and EHRs, are the worst sort of enterprise garbage.
If you released it today, you would own the car market and you would be best pals with Elon.
If you released it in 50 years, people would think you are quaint for playing with that old-tech for a problem that been solved 10x over.
No, what _you_ need is more hastily shipped crapware to capture markets from your competitors so you can make more money and disappear. Hastily shipped crapware has enabled lots of new and interesting things. It also keeps 10x as many engineers busy serving it as would moderately better thought out, slightly later shipped software. Babysitting crapware isn't bad money for an engineer, but it's cosmically questionable as to whether it really advances us more than its opportunity cost. And hastily shipped crapware at scale is a real giant energy problem. Take a look at the size of a Facebook datacenter and tell me again how a metric fuckton of PHP is a good thing for the environment.
1/ Poor code =/= technical debt. Far too often, poor code masquerades as "technical debt"
2/ Good engineers do the right thing which is always a judgment call based on circumstance & culture.
3/ Take guidance from your best engineers, and find a way for the business to adapt. Much easier to change the sales deck or pricing today, much harder to overcome bad technical decisions. No one ever said "I never knew that sales deck or pricing would last this long" but many say this about software. Be super careful & deliberate.
4/ Build things that are simple (not easy), and respond to demand. Focus on movability & robustness, not "perfect" design.
Ergo, in your scenario even with 10 users, the application basically died under its own weight under nominal use.
With, of course, the realization that simple is hard and that it is almost always orthogonal to easy.
Don't waste time trying out the latest-and-greatest cutting-edge development techniques. Except SPAs (Single Page Applications) because those (somehow) take less time to implement and are 'easier' to iterate than a traditional UI.
Don't waste time perfecting the data models. Unless you're talking about data structures (ie they're totally not a type of data model) because your data structures should be perfect from the start so as to avoid the technical debt of modifying them later.
Don't waste time enforcing the SRP (Single Responsibility Principle). Unless you're talking about 'siloing areas of concern into microservices' because splitting responsibilities (and designing custom APIs to drive them) isn't just another approach to enforcing SRPs. Designing APIs that connect your app to it's dependencies in a manner that won't change (and break things) later is easy right?
So, code, code, code fast. Don't waste time early on developing a sane architecture, except when you want to spent time developing a sane architecture early on to iterate quickly later.
Forgive the sarcasm, this isn't a troll. I'm just pointing out the blatant contradictions.
The only valuable advice I can grok from this article is. Disregard TDD, build a MVP, push it into alpha/beta ASAP, fix bugs later.
IMO a better context for this would be: real software engineering requires compromises. There are a number of competing factors that need to be juggled in order to be successful: time, money, complexity, quality, etc. The point should be that you need to balance these factors, not focus on one above all else.
Financial debt (contrary to the implied sense I get from reading this article) isn't always a negative -- business loans and even VC capital are forms of debt without which many companies wouldn't be able to exist, mortgage debt is debt without which very few people would be able to buy a home. These are useful tools. Likewise, accruing technical debt is often just part of the process of writing software, especially when "bootstrapping" new projects. In moderation, it isn't bad at all, but like financial debt, you do have to manage it, see it as a risk factor and not let it spiral out of control or else it will become very destructive.
The more lines of code you have, the more stuff you have to maintain and understand.
Of course this is pretty stupid, but it's stupid in a way that sometimes leads to insight.
If your application consists of 1000 lines of code, you can rewrite it in an afternoon.
Also, cleverness can create its own kind of debt. Even as you believe you're working on reducing debt by creating some clever and sophisticated abstraction, you are actually just making it worse. I think this happens all the time.
That's why my basic, fundamental approach to software development is "throw it away and replace it with a simple shell script."
There's a clue in this quote:
> The most successful cities in the world, such as New York, London, Vancouver, Sydney, etc
A big city in each of the 4 largest native English-speaking countries of the world, in descending order of population.
My startup went from a few scribbled notes on paper to a full app in the app store in three months, including 50 lessons, 6 mini games, and a challenge/achievement system.
Naturally, I had to cut a LOT of corners to do that, and accrued a ton of technical debt, but I was able to keep the high level architecture relatively supple, and as a result tackle the various messes individually rather than as a whole later.
It paid off. Or architecture has since changed twice over so none of the old code remains, but we're strong on investment, have lots of people, and are poised to take over our target market. And we got app of the year last year.
https://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_pil...
I feel like that's it's own idea and should not be confused with technical debt. The article seems to make this conflation as well.
I think developers, including myself, like to get things right before launch because fixing it after launch is harder, if not impossible once other emergency/technical debt comes in and starts layering on top of that debt.
If this goes on for 1-2 cycles, before long your app, backend and internals look like a shanty town that no developer wants to enter.
Sometimes I wish code was visual like buildings, you could more easily see quality that way. Mostly though you have to believe in your developers/engineers that what they are building is a mansion/building you want not a shanty town dwelling that is held up on duct tape and silly putty, yet the technical pressure/debt put on is largely coming from someone who can't see the visual representation of the system.
Developers can only be MacGyver for so many release cycles before instability strikes the system down if management is not careful.
There will always be some amount of technical debt but developers do love to get things done and out the door as fast as possible. We like to do so with a minimum amount of technical debt as it is a constant battle against time/market/cost with quality.
I haven't seen anyone else comment on this sentence, but I'm baffled by it. The first and second parts don't seem in any way connected, nor at all in keeping with my experience of building SPAs - the UI layer is often intricately complicated and every bit as hard to iterate on as an API!
And I accept that there are times when you can move faster by accepting technical debt.
However, I'm always uncomfortable when this view is preached, because I think that while all these things are true, they are rare. The most common case is that when corners are cut, or 'tactical' solutions are adopted, you begin paying for them within a week, often before you've even finished implementing them. Another common case is that doing something in a way that allows future change is almost no extra effort compared to taking on the technical debt, but the vision or mental effort (or political will....) isn't there to even try to see things in a new way.
So sure, sometimes we should embrace technical debt wholeheartedly, but most people are already too far in that direction already. I'd rather read articles about people who spent a little extra time thinking in these situations and came up with a solution that was quicker to implement than the original plan, with less, simpler code and fewer bugs. I think such things happen just as often as technical debt producing good outcomes.
The trouble is, I don’t know which half is which until it’s too late.
Apparently they're using Rails or PHP at Bigcommerce. If you're trying to get to market as fast as possible then Rails makes sense but what goes with Rails is TDD. You really shouldn't have too much technical debt when you're using Rails or Ruby.
On the other hand, if they're using PHP then I'd bet a lot of money that they have a lot of technical debt because they hired poorly skilled developers who were able to push through some code that looks like the feature requested.
At some point, you will be crushed under the weight of your own technical debt. The race is whether your startup will die before that point. The author sounds like he's trying to get acceptance that there will always be technical debt. How about letting your engineers working on refactoring and making things nicer?
Facebook could have been crushed by their technical debt but instead they let their engineers loose on the code base. That's how they've come up with Hack lang which is type-annotated PHP, and that's how they've come up with flow, the javascript type checker. This is why they made it a mission to allow developers to be able to fix and deploy some code within 1 week of starting a job. All of that forces them to fight against technical debt and to keep themselves in the game.
That's the real issue here; sure it sucks creating technical debt in an effort to get to market first, but it sucks even worse when as an engineer you aren't allow to tackle the debt at any point or are only allowed to tackle it very rarely. Efficiency is what will keep the dollars rolling in.
A recent example involving Google is Chrome. People are actively switching back to Firefox or another browser. There's bloat in Chrome! How's that possible?! My bet is a product manager who isn't allowing the engineers to fight the technical debt in the code.
Systems take too long to get to market, which causes
marketing efforts to stall or become irrelevant, and
competitors beat out the startup to an offering.
Wikipedia has a bunch of great links to more on the topic if you're unfamiliar with it: http://en.wikipedia.org/wiki/Product/market_fit
As a result of that, when it comes time to renovate the house, the workers doing the renovation frequently find that nothing about the construction of the original house was designed to make it easy to extend or add to the house. even things as simple as adding light fixtures can require running wires all through the house back to the electrical box, cutting holes in walls, patching walls, moving insulation, etc. When you go to add rooms, you find that the way the foundation was poured means you'll need to remove existing foundation drainage systems to add new footings, add new foundation, add new drainage etc. Then, tear everything apart to add more electrical and plumbing. and finally, tear apart a big portion of the roof to merge the shingling on the roof.
Houses _are_ static, and thats why we tolerate building things this way. We can't predict how they might change in the future, or what the needs might be. If you could/did, you could mitigate almost all of those challenges I mentioned earlier.
In software, we do know that it will change, and we frequently can predict where and how those things might attach. Because we can predict those things means that we should design our software to be ready for those expansions as well as possible to minimize major structural changes to the whole applications in the future.
I don't see taking on "technical debt" as being faster. It's just less risky. You're more likely to have something that does something after that month of work. Hence the common perception that it's faster. In an organization where there's mistrust (rational or not) it's easier (or even better) to take the safe road of "technical debt".
http://www.se-radio.net/2015/04/episode-224-sven-johann-and-...
The most important thing is always to make money. The most important money is the money you make this week, not the money you might make in a year, and you have weeks of potential money queued. It's equation is never right to spend a few weeks not making money so down the line you can potentially make other money at a slightly fast rate. It's definitely never the right time to do that over and over again to get the many small improvements that will make you much faster in the future.
That said, the only reason their business is where it is today is because of shortcuts taken in the name of speed-to-market. In their case, the end really did justify the means - eCommerce is ultimately a land-grab for market share, and BC dove in hard, spent _millions_ on marketing and customer support, and left Engineering to fend for itself. As an engineer, it sucked, but in retrospect and looking at their success to date, I'm willing to concede it was the right choice at the right time.
A good engineer can produce a good product under moderate time pressure but will accrue technical debt in critical spots if more time pressure is applied.
The great engineer anticipates the change and will not strive for perfection in parts that are going to be rewritten in the future anyways but instead just make it do the job, while at the same time keeping things simple and flexible. That's how they deal much better with deadlines and time pressure.
Consider the center-point of this article:
> I view technical debt as any code that decreases agility as the project matures.
Well, any given piece of code can have positive or negative value based on what change you need to make in the future. You can have a perfectly factored, elegant piece of code which becomes useless if the job to be done changes, at which point you are better with no code than the previously ideal code. On the other hand, you can have a module which is ugly as sin, unreadable, and horrible in every way, but it does a job that never needs changing, and it's isolated enough that no one ever even has to look at it or consider it.
So the very judgement of the scope and severity of technical debt depends non-trivially on the future. Individually design decisions can be critiqued, and the likelihood of certain types of changes can be weighed out, and certainly the best engineering and product teams are very very good at this, but the fact is that the whole endeavor is terribly fraught and difficult to concretize.
All that said, I think the author is absolutely right that engineers often have a proclivity to polish some bit of code or architecture when their time would be better spent elsewhere, but it's just a tiny microcosm of the problem facing all startups: you need to build a money-making machine, and you have infinitely many dials to turn, with only a handful of (hopefully) smart people on a very limited runway. How do you make decisions day-to-day about where your greatest marginal utility lies? I don't think there are any rules of thumb that are globally applicable. If you are lost, a blog article is not going to help you. The only hope is to gather feedback as widely and quickly as possible and be brutally honest with yourself.
1) People lie about what is required. Anyone who has been in the industry for more than a token amount of time has experienced the PM who says that they need X+100 when they really need X because "programmers are lazy". Similarly everyone has worked on a team where one or more of the programmers are more interested in building their beloved architecture than solving the problem at hand.
2)People don't listen to the needs of others. Very often they assume problem #1 exists ("They don't actually need that") and blindly forge ahead without paying attention. They play whatever political game that is required to force the other side to follow their approach.
3) People make mistakes. Often this is due to a lack of experience, but sometimes it's just because the problem is hard. Some people make mistakes more often than others. Good management will recognize those that are making good choices, but sometimes you have to let people crash a few projects before you realize that they don't know what they are doing.
4) Choices are made at the wrong level. With all of the other points I've made, it is exceptionally tempting to move all the decision making to someone who acts truthfully, listens well and has a good track record. However, the information required to make a good decision can be at many levels. Sometimes you need 100% test coverage of something and only the programmer will know when this is going to happen. Sometimes you need to deliver tomorrow even if it doesn't work at all. Only the PM will know this.
In order to build a team that executes at the highest possible level, you need to have people who work well together, trust each other and are competent. If you do not have this, then success will likely be a matter of chance. No amount of heroics from a single person can reliably fix the issue -- and often such heroics just make matters worse because they result in the behaviour listed above.
I think that discussions around technical debt really lack any mention of technical _credit_, the opposite of debt. Just like how you can take on debt in your codebase, you can save up credit, too. In an ideal world, you would always have savings so that you don't have to take on debt.
Running things over a message bus is slightly more work than one large application, but the upshot is that it lets you experiment with new technologies and ways of doing things without betting the entire product/company on it.
When you have a large enough codebase, you will always find bad code if you look long enough, it's pointless to waste time on that. You should only refactor code that you needed to touch in the first place.
However, management need to understand technical debt and their impact on timely product delivery in later stage of development.
also another mistake I've seen made over and over again is to say stupidity==technical debt. If you're doing stupid things just to 'get things done' you're doomed in the long run. If you've thought about it and made the decision to cut a corner you should know why you've done it.
On the other hand this is what I miss most about school, or perhaps just doing things out of my own interest.
There was a time when I could appreciate the beauty of code and all I could worry about whether my software was elegant. Then I joined industry and everything changed.
What's wrong with the title: he asserts that "we" all defined technical debt incorrectly, when (in my experience) there's no meaningful agreement on that term. It's one of those terms that, if it's used, requires follow-up to be sure that both parties are talking about the same thing. The author even acknowledges this lack of agreement, but he asserts that we're all using it wrong.
The author then proposes a definition: "any code that decreases agility as the project matures".
Sorry, but there's no polite way to say this: that is, quite possibly, the most useless definition of technical debt I've ever heard. Every line of code decreases agility in one direction or another, every step forward is a step past some other opportunity. Thus, a definition that conflates necessary progress with actual problems is worse than useless. It's harmful. And frankly, I have trouble imagining an intelligent person who would use it. At this point I'm attacking the author a bit, but at this point the author has attacked the entire world a bit, so it seems fair.
The author has made clear that he thinks he's smart, that we're dumb, and he's figured out things we can't understand. So gross.
Captain know-it-all then proclaims, in bold font no less, that "The Most Important Thing is Getting the Thing to the User". No. Fucking. Shit. We fucking know that. We get that. Please don't do this shit where you conflate the whole damned world with the one guy in your office who you're passive-aggressively railing against with this awful awful screed.
Then this just gets into total dishit territory. We're supposed to not worry about getting everything right, but god forbid we need some migrations along the way, no... those aren't acceptable to this bloviation blowhard, we must get them all right on version one.
I now have a better understanding why BigCommerce is about the fifth company I think of when I think of places to host a shop. They're a fifth place company that hires fifth place management.
And Shaun, if I'm correct that you wrote this because you're tired of hearing your team talk about technical debt, then you are truly a horrible manager and you should just fucking quit. I'm not a person who typically advocates quitting, but this seems like nothing more than a passive-aggressive attack on one or more of your engineers who mentioned technical debt or code quality more often than you'd prefer.
And if you really just thought this was brilliant insight: you're an idiot. You used a dumb definition of tech debt to justify an argument that showed you don't understand the original metaphor. You are an impressively stupid man.
I don't think this is what Linus was talking about...