Things You Should Never Do, Part I
joelonsoftware.com
joelonsoftware.com
- Write comprehensive test cases and run your current product agains them. Have these also benchmark the code if they're performance critical.
- During the rewrite, slavishly adhere to the test cases. If there's an undesireable difference discovered between old/new, write another test case.
- If possible, break up the original product into smaller modular sub-projects you can rewrite independently.
Most problems with rewrites come from starting from scratch - slowly replacing a code base from the inside is a much better way to do it.
This.
Many projects can be done this way. In particular, the process of figuring out the set that can be replaced at any given time can aid in the refactoring.
An example of when a rewrite goes bad is Final Cut Pro X.. Apple really leaned a lesson with that fiasco.
That said, we left the new and old systems running in parallel (all new users were directed to the new platform) for over a year before we even informed users we were shutting it off, then gave them another six months to migrate. And we never purged the old data; it's in cold storage in case someone ever comes looking for it.
Rewriting the codebase isn't something you should take lightly, but "never" is pretty damn strong word.
Technologies change. More importantly, requirements and visions change as well. Blindly throwing money at an existing system without considering alternatives is a really, really bad idea. And while you can argue that it's possible to refactor anything into anything else, you can't argue that it will always be cost-effective.
I was unfortunate enough to work at a place with a 10+ year old code base where there have been 3 to 4 software architects that have all had their own vision over the years. The code has been refactored many times already. It consisted of a buttload of singletons that were all initialized at seemingly random places. It was impossible to start or shut down the app deterministically. It was massively multithreaded (in most cases for no good reason). They had this horrible implementation of a shared cache (because many instances of this app ran across many computers that needed to share state).
It had no unit tests. Whenever you change anything it would break something entirely different. To me it was an example of everything going wrong...many times.
The best part was that it had to run for weeks because it was a critical application (for fire, police and military services). It didn't. They were instructed to reboot all the machines every day. But it wouldn't start up properly every time so they may have to do it multiple times. I did my best for the year to clean up the critical parts of it. I was contracting there but they had offered me a full time position by the end.
I finally cracked it when the original "software architect" came back because he had been fired from his old position. He was annoyed that over the years his code had been changed so he started to put back in place everything that he had done before he left. Not only that we had an angry complaint from one of the customers about a possible hostage situation that could have turned very badly because of this crappy system.
We tried to reason with the software manager by suggesting that we assign one or two people to start from scratch and take across code that can reused. It would take a bit of time to reach the same feature set as the old application but it had become so difficult to add new features to the old one that it needed to be done.
The original article does assume that you're dealing with a system that more-or-less works. If your application flatly doesn't do the right thing, than a big rewrite from scratch may really be the best choice - there's nothing to save.
But spaghetti code doesn't just fall from the sky, it occurs because of politics, stubbornness, and bad processes. Over the course of the 10+ years it took to make the code base, is it really the case that nobody but you noticed the problem? It's more likely that there's a lot of pressure going in the opposite direction, and other maintainers didn't know what to do either.
Dealing with subsystems helps you handle both problems. Management might not be willing to let you rewrite the whole thing, but if you said, "Let's just fix the boot-up process for the Cyclotron 4000 resource. Nothing else changes, just the Cyclotron boot." you might be able to get permission. In a badly-maintained project, it's hard to replace all the instances of one service - that's what makes it 'badly-maintained' - but it's still easier than dealing with the whole system in one go. And, of course, instead of 'fixing' the Cyclotron you're actually rewriting it with a new, non-wacko Cyclotron service.
Then you go back to your manager and say, "It was rough, but the Cyclotron 4000 no longer blocks the start-up. Let's get the next thing on the list." Not only do you have a slightly better project, you also have better credibility with management, which makes it more likely you'll be listened to when you say a certain technical measure is necessary. Next, fix the subsystem that talks to the Cyclotron - and so on. Pick a right time to introduce tests, code review, and all the rest.
Remember that just as you had the experience working with the terrible code base, your managers had the experience of working with the previous 3 or 4 software architects who "had their own vision" and delivered a product that doesn't start up reliably - I don't think it's surprising that there was no longer the political will to assign people to refactoring or rewriting tasks. Bad architecture uses up the political will needed to approve good architecture, because it makes all "architecture" tasks look bad. You need to regard you reputation as a finite, under-supplied resource just as much as your time and budget and plan to get more.
From your use of the past tense, it looks like you're no longer in that situation (good for you!)... but that would be my advice if you see a similar situation in the future. I've used this plan in my own career to rewrite a (much smaller, only moderately troubled) project piece by piece over the course of a year.
The place had a reputation for hiring highly motivated engineers and burning them out. Just to be replaced by another. When I left, they hired a very talented guy that I worked with for a couple of months. He left recently and the cycle begins again!
The singletons were just one of the many problems. I remember there was a "database.cpp" file which handled all access to the SQL database. It was over 10k lines of code and had hundreds of structs to represent all the tables in the system. The person responsible for that ensured he had a job by only working with that source code.
This makes me thing that Java is better for this kind of big "enterprise" application, not because it's faster or more enterprise or somesuch, but because it's more limited, and therefore less things can go wrong.
I worked in two banks, developing web banking in one and middleware service in the other, and while there were some strange things (what's with banks and XML, really?) there was nothing that terrible here.
But then I'm pretty sure that someone will share their Java horror story.
As bad as everything was it was a fantastic learning experience for me though.
Were the low memory issues due to the limitations of current hardware at the time and no longer relevant? Do many computers still have floppy disks, and would you still want to have to support that?
I think re-writes should be avoided unless you have a full understanding of the reasons for all the one-off patches. You should also have an idea of what you will lose/gain by getting rid of them.
I would interpret Joel's point as: Not shipping something for years and then shipping unstable crap is a bad idea. In this case, it ruined people's already shaky confidence in Netscape. Rewriting to improve code isn't always a bad thing - in this case it did end up paying off, just too late to benefit Netscape. Firefox probably wouldn't have happened if the switch of codebases hadn't been made. Netscape being out of the picture by this time was a good thing - since Netscape was institutionally incapable of shipping quality software it would just have screwed it up.
On another note, things like Eclipse's refactoring tools have exponentially increased my ability to keep my codebase clean and easy to read. Renaming variables to stay consistent etc is extremely easy using these tools and I have to wonder why people still struggle with issues like this.
I've spent about three years (on and off) building a Django application for a team where I work and, over time, new functionality has been bolted on as and when the needs of the team changed. We've also gone through one fairly major model change that seriously wrecked the sense of symmetry in the v1.0 I delivered all those years ago.
A bit later this year I will be making a whole lot more changes to the application and I've decided that rather than going from a v1.6 to a v1.7 this is a great time to throw away the majority of the code and re-build from the ground up so that the v2.0 is properly architected to meet the needs that exist today.
It's going to be a lot of work but I think it's worth it in terms of making new features easier to add and reducing the maintenance burden associated with a structure that has become overburdened with technical debt.
If your business needs have completely changed then it might be worth it, but at that point it's no longer "the big rewrite" but instead just writing a new application. (that isn't required to fulfill the requirements of the old app)
The same is true of software. Well written software is a joy to read and expresses an idea clearly. I love reading code like this. I love writing it even more.
If we want to write code that is easy to understand we should spend more time reading code than simply writing it all the time. Practice makes perfect but introspection reveals the path to self improvement. Absorbing the good ideas of others and filtering out the bad is part of being a good writer and is especially important to the programmer. Read more code!
The one that's written clearly. An ugly, obfuscated loop is worse than clearly-written and commented recursion and vice-versa.
If the most pressing readability concern in your code is whether an existing loop would be more easily-read as recursion, you're already done. The code is clean enough. Move on.
This reminds me of Final Cut Pro X. From an abstract code-base standpoint, it was much improved. Many people on the nets (that probably never used FCP) tried to cover for it with excuses like "it's a 1.0 product" or "just don't upgrade", but from a user standpoint, the rewrite left behind enough features that it became unusable to many people (myself included) that owned, used, and loved the previous version.
As developers, we need to do projects for ourselves (like code cleanups, and rearcitecting) but we need to understand the user impact first.
That said, there are times when tossing out the whole thing and starting over is absolutely the right thing to do. You can identify these apps by when the 8x5 sticky index cards describing defects fill a 9'x15' wall, application instability is a feature, and the "Big Ball of Mud" anti-pattern definition uses the app as its case study.
It is very situation dependent on whether the messiness is a result of bug fixes that will have to be reapplied, or because the system was built when no one had a clue how to build it.
Can you make a case for "write a new product that will (eventually) replace the old one?" After all, you know a lot more now from having written the first one. Write one to throw away, and all that.
Or if you write an X, are you forever discouraged from ever producing another, better, more relevant X?
My unfortunately long experience with maintaining code has taught me how true it is.
I would add that most often the maligned code is the only existing documentation of a company's workings.
Perhaps in some cases, rewriting your software can be a competitive advantage.
---
FWIW I hate the posting of a link without describing the content in some way.