Technical debt and tacking into the wind (2013)
flak.tedunangst.com
flak.tedunangst.com
Legacy software at first glance can seem unruly and hard to understand which gives devs a kneejerk reaction to rewrite it. But make sure you give your developers enough time to sit down with the code and understand it. Given some time to really grok the platform's idioms often you can empathize with the previous devs work and figure out how to work within their framework.
You have to take the risk of refactoring into account as well. There is no guarantee you'll end up with better software, rather than just different software. You could end up throwing away years of edge case handling and feature development just so the core product runs on some new technology the new team is familiar with.
We're getting close to release, so we have less pressing bugs and no new features to implement, so myself and a team mate decided to take a week to perhaps refactor some of the code and update it to Swift. After said week we reach the conclusion that management made the right decision: The written code handled so many edge cases that if we were to rewrite it, even converting from objc to swift could ignore some edge cases.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
It means that all future developers will have to spend a lot of time to understand complex code that could be simpler. Also, when the code is very complex, developers don't understand it fully, are hesitant and unhappy to touch it. Every new feature becomes a challenge and can break something. You need to rely on regression testing, but if you have tests, you may as well refactor.
Another reason to refactor is that you want to give new developers the satisfaction of "owning" part of the code. I read an article on software engineering at Google saying that it was ok if some parts were rewritten over and over.
> you can empathize with the previous devs work
Some devs are just bad and/or worked under strict time constraints. Code isn't only a matter of taste. There are objective metrics that tell you if some code can be improved.
> It means that all future developers will have to spend a lot of time to understand complex code that could be simpler.
Code that's complex is often complex because the domain it's written for is complex. I've heard it called incidental complexity. Could you write it more simply? Potentially, but how sure are you that you're not about to get rid of edge case handling or important features?
Tests only work if you're sure you understand the requirements well enough to write the test. If you're at that point of understanding in the platform, you've probably got a good idea of whether or not you should refactor.
> Another reason to refactor is that you want to give new developers the satisfaction of "owning" part of the code. I read an article on software engineering at Google saying that it was ok if some parts were rewritten over and over.
I think it's an attitude software learned while it's been flooded with easy money, where the tangible cost of constantly throwing away code is never really considered. Code is never considered an asset, but had you built a tractor for a company they would definitely want to know why you want to build a brand new tractor when the old one is working fine.
I have mostly worked in agency environments, and as such cost is always part of the conversation. Often the only people who want the refactor is the developers. It doesn't provide any value to the business. I often do advocate for a refactor for some of the reasons you mention, but to get client buy in you need to provide some real value, and when you have to think about that it quickly discovers whether or not your refactor is frivolous.
You're not wrong about the cost and the need to justify it, but I think you might be underestimating how often this happens to physical products. Lists like "the worst cars of all time" are full of needless reworks, frequently driven by some executive's desire to fully "own" a product. I suspect the biggest difference is just that as the cost of throwing away a working design becomes more expensive and visible, the decision to do so slides higher up the ranks.
There are still times when knowledgeable legacy maintenance is the right answer, I think. In particular, the Catch-22 of systems built without the capacity for online updates which can't be taken down to add that capacity. I've heard horror stories of the deeply-understood hacks that keep telephone networks running, because they're ancient monoliths contracted to >99.9% uptime. But in general, the argument for not rewriting legacy code is that it takes too long, or it's going to be expired eventually, and so relying on the original author or hacking in changes is more efficient. It seems like "don't rewrite it, but repeatedly spend the time to bring new people up to full expertise" has a very narrow window where it's the right choice.
I've seen this story over and over. I'd argue that the following story is much, much more likely than "devs were in a hurry and wrote bad code":
Developer comes in, understands the happy path, asks why there's so much code, starts advertising how they could rewrite it. Once they get the green light, they quickly implement that 80% happy path.
Best-case scenario for tech, business is somehow convinced that this new thing is shiny enough that they give the go-ahead. [0] Manual workarounds are implemented for the 20% of unhappy path processes. Eventually, business will start asking for the system to automatically cover more and more of these unhappy path cases. End result: The system becomes a complex mess of code to handle these multiple edge cases. But hey, it's at least a different complex mess than before!
Lesson: The code is a mess because it's encoding a messy business process full of one-offs and edge cases. Rewrites of systems like this only work if the business process is simplified in parallel with the redevelopment effort. If the business process remains unchanged, the code will reach the same endpoint. Worst-case scenario, everyone will saw-tooth oscillate every few years between "shiny, fast, and 80%" and "old, slow, and ~100%". (Worst-case because it allows for the minimum amount of ROI extracted from the technology for the same business process.)
[0] This is already pretty unlikely. More likely, business is not willing to take on that work, and will demand 100% of current coverage. End result is the same, just now it happens all at once instead of sneakily over time. It's up to political and cultural inclinations whether this becomes a kill shot for the rewrite. Some places will force it through anyway, while others will see the writing on the wall and just end it.
> It means that all future developers will have to spend a lot of time to understand complex code that could be simpler.
How did you get from one to the other? The OP's point is to give people enough time to figure things out. Not to never rewrite.
We need to agree that going to either extreme, always rewrite vs never rewrite, is not a good solution, or even a solution. We should spend some energy in figuring out why to rewrite (is this code even used? is it used more than foo / time unit, is this code causing any problems as is, is it worth the time to rewrite it) and how to rewrite (make sure the new version covers all the use cases etc). In short, we should have a way to decide what to do if you say to rewrite and I say no.
If you are new to a company and are dealing with legacy code who's logic is messy and the solution seems to be a 'rewrite', there's a high probability the new codebase will be as messy as the old one. Even if you aren't new in an org and yet your legacy codebase is a POS. It's highly likely given the same org doing the rewrite that it'll end up as a POS as well.
So if you are making the code 'simpler', getting rid of layers of abstraction to enable a pivot that never happened, you might run into a CEO who still wants that to be an 'option'. A plethora of special cases all piled on each other probably resulted from no single source of truth on requirements in the org. All of these things will still infect a rewrite from scratch.
Its much better to look into specific parts of the code that time can be taken to refactor properly. The time part is critical.
I work with software that has been continuously worked on since the 60's. It has the dreaded legacy if's like below everywhere.
if(obtuse variable) then
change a few other obtuse variables
One of my main projects throughout my stay has been to reduce these with small, disciplined improvements. I have made major headway in this fashion. You have to find the leaves of your software and work your way up while rigorously testing your changes. This is the exact opposite of the human gut reaction of a massive top down rewrite. It's impossible to tell how far you are off-course until it's done. It's the waterfall method of refactoring.
IMO: It was always hard to justify preserving legacy ASP code once Microsoft made the decision to end-of-life the entire ecosystem. Web development was much newer than, and it was common to completely rewrite web applications in the "language du jour."
IMO: It's also easy to justify "fun" projects like writing your own compiler.
Maybe I'm just being a "Monday Morning Quarterback," but would it have made more sense to build an ASP -> ASP.Net converter as a standalone product?
You're 7 years late to a dead controversy. If you were in the middle of it, it would make a lot more sense as rebuttals to specific complaints made at the time.
Since the Internet Does Not Forget, perhaps those complaints will even come back up in this post, in which case the article may make more sense in hindsight. But I'm not going to be the necromancer. If you want to do your own thread necromancy, https://hn.algolia.com/?dateRange=custom&page=0&prefix=true&... should pretty much get you there, if you ignore the articles that are about the condiment. (And the top one is something else, too; FogCreek Wasabi is a compiler, not a storage platform.)
3 years later, we were almost done.
At least it's not a 100% new language skinned to look like the old one!
I've heard so much talk about technical debt (and done plenty myself) over the years and it almost always gets used to talk about improving the messy code of the last idiot who worked on the project and didn't know what they're doing. I really appreciated reading Ward's clarification that his original metaphor was about going back and improving the code to show an improved understanding of the problem. Assuming implicitly that the code is clean and well-written in the first place. So now I don't know what to call the effort required to clean up bad code written by inexperienced predecessors. Incompetence debt? Student loans?
Does this piece of code ~spark joy~ fulfil its responsibilities concisely?
Some technical debt is just working around features that no one really uses, or don't drive enough revenue to justify their complexity.
However, often people have good reasons for why they write the code they write. For instance, maybe there are so many quirks and edge cases there's no way to write clean code without creating a massive body of abstractions that would just making development speed slower.
I'm all about clean code but as a senior developer I increasingly realize that naturally, everything is a tradeoff. While you should be biased towards neatness, sometimes writing pristine code just isn't worth the time or cost of maintaining all those abstractions.
I agree about the trade-offs, I've been there myself too.
In the tall-ship era of square-rigged vessels, wearing ship caused far less wear-and-tear on your rigging and was something you did to be more cautious or if your boat was far from a shipyard. Tacking through the wind was perhaps a bit reckless and not something to be done unless chased by a privateer or you were a warship in action.