Things You Should Never Do, Part I (2000)
joelonsoftware.com
joelonsoftware.com
This is also the core motivation behind all of the schemes to force a single coding style on all developers at the cost of performance and usually with an extremely complicated design for seemingly no reason. Usually, these turn out to be unreadable messes as well a generation later once everybody has moved onto the latest and "greatest" coding style trend.
Instead of doing rewrites, you can just refactor code. That means, among other things, deleting obsolete code for platforms you no longer support and, in the case of C/C++, converting macros to functions. You can also document the codebase (with comments not commit messages) so that developers understand what the code does and why it is there. But a full rewrite is usually a bad idea that will just introduce new bugs and remove existing functionality.
Refactor vs rewrite is basically the software development equivalent of reform vs revolution in political science and everybody with even a minimal knowledge of history knows that revolutions usually end badly with an even worse regime taking power. Likewise, the failure to accept the need for reform (i.e. refusing to allow refactoring) inevitably leads to revolution.
It means there is a shared idiomatic approach across your team and all the documentation for it is just the framework documentation. It is also easier to hire devs, if you can vet their ability to write idiomatic code in a specific framework then you can better assume they will mesh well with yor codebase.
It is a communication and empathy problem at its heart. You are being empathetic to future devs and future you by acknowledging the knowledge they would need to work on your code. Then the communication of your idioms is baked into the framework ecosystem and docs.
I'd say there's quite a bit "special" about all of these attributes, at least enough to warrant careful consideration.
If you qualify this advice to the point where it actually becomes useful it turns into, "Use code written by other people that doesn't suck." And that, of course, is indeed a good idea, but it's a lot easier said than done.
But despite that caveat, there do exist frameworks out there today that speed up development for some developers. Whether one has the time or energy or will to audit them to find one that works best for them (or learn that none of them do) is up to the individual.
I bring this up because many of us on this website are at times infected by NIH syndrome and think we can do better than a dedicated team of experienced front-end developers. And it's likely true for many frameworks, but not all. Additionally, we may want to extend that attitude to our workplace where our peers and bosses likely value development speed and cross-project consistency over code performance.
I used to eschew any and all frameworks, but now I realize that my values have changed. Projects like Svelte give me hope for a better future for front-end development.
Of course. But there also exist frameworks that will slow development down. For any X in {frameworks, languages, operating systems, libraries} there exist X's that will be helpful and X's that will not, and so "use an X" by itself is not helpful advice for any X. You have to use a good X, but that isn't helpful either because it just begs the question.
I just intended to push back against the notion that frameworks should not be considered.
Don't let bad frameworks represent all frameworks. That is a faulty generalization [0].
It's not about code quality at all, it actually doesn't matter that much if the framework is excellent or just okay to reap the main benefit, which is knowledge diffusion through the team and having a set of agreed upon idioms that there's no need to bike shed over. Not every team has a software architect skillset so it's a bit of a risk hedge as well.
There's no need to argue over implementation details, you waste less time onboarding new staff and you have that rich ecosystem of docs and community of solved problems.
That's all. There are great reasons to build your own software but if you're working in a team I feel it's an easier experience to use a known framework. Also a bit more responsible from a business risk management perspective, since you don't have as big of a "bus problem". Only your business logic is trapped inside your org.
LOL: so did you read TFA, or just come up with this yourself?
Did you read the article? He says what you wrote explicitly, in fact there is an entire section on it.
It is so easy to think "If only I could start everything from scratch, life will be so beautiful and I will be so successful and everybody will be happy. I will fix all the problems of the current codebase and people will forever sing songs about how amazingly I turned everything around."
What more frequnetly (much more frequnetly) happens is usually some mix of:
* developers not really understanding why exactly the previous version failed,
* developers not really understanding what actually worked well in the previous version,
* developers completely underestimating actual amount of accumulated knowledge in the old system that they now have to replicate. Obviously, they only discover it along the way when it is suddenly too late to fix the design to take all of that stuff into account.
* developers not appreciating the fact they are in a much very different (worse) situation than original creators of the previous version. The creators of the previous version had time to build the system and then slowly evolve it. But the current team has typically a short time horizon to replicate ALL of it.
* as the resources are shifted from the maintenance of the old system to the development of the new, suddenly the clock starts ticking and the stakeholders are impatient to see the result. The pressure grows quickly and with it comes inevitable compromises and technical debt.
* at some point development team is told they have to start maintaining the old app. The resources devoted to the rewrite shrink, the work grinds to a halt. I have seen many more systems that have been in a state of perpetual migration than I have seen ones that have been successfully completed rewrite.
On the other hand, developers writing the new system have the major benefit that the requirements are known. Second system effect is real, but if you can avoid it, and focus on addressing the real requirements (as determined by the lived experience of the first system), you can get something good. Sometimes, something really good, if the old system was built on the wrong abstractions, and the new system has better fitting abstractions.
Of course, if you don't actually know what the current system is doing, you're not really in a better place to design the second system.
For example, consider a team that has been maintaining the system for a long time. Most or all of the people that has initially created the system have moved on to other positions or companies. Many of the people who stuck with the team did so for reasons of being experts but are also unable or unwilling to communicate well (for example, because they see the value in preserving their position of being one of the few people with expertise).
In a team like that, they might have good memory of things they have worked on recently, but not necessarily things that are working and the users take for granted. Users do not typically spend much time reaffirming the features that are working well.
One of the challenges of working with stakeholders is that close to 100% of the communication tends to be on things that are not working or are missing or they want to get done. As a developer or manager of the team, you need to find some way of answering the question "am I doing well?". Because users are of course unhappy with how the application works, but to understand levels of unhappiness you need to find some other reference (working for some other projects).
I was yesterday in a meeting with developers where they complained about how manual our deployment system is.
Our deployment system requires them to press a button to deploy a version of the application to the environment. Yes, log in and press a button.
I explained, that I worked in the past for teams that had hundred page manuals on how to compile and deploy the software and that manual would be compiled again and again for each release and the process would take from couple of days to a whole quarter.
So you need to have some perspective to be able to judge that kind of feedback.
* Many developers prefer working on greenfield code rather than adding to an old codebase
* Working on a rewrite buys you some time in limbo where you can write code while being free from the burdens of production operations. This period is shorter than people dream, but it's often long enough for developers to pitch a rewrite for 6 months, code for 12 months and then skip to the next job with "Spearheaded rewrite of critical infrastructure" on their resume. Time it right and you can get out of there before you have to support the code you wrote.
Rewriting software is an excellent way to gain such understanding. I only understood what GNU autoconf was doing when I was in the middle of rewriting its core functionality in pure GNU make.
And who knows? You might actually make a better wheel against all odds.
Maybe you were thinking of GNU Automake ...
I suppose I was rewriting automake too: I started metaprogramming GNU make by evaluating templates. It's a surprisingly lisplike turing tarpit.
Things You Should Never Do, Part I - https://news.ycombinator.com/item?id=31122975 - April 2022 (7 comments)
It’s harder to read code than to write it - https://news.ycombinator.com/item?id=31117277 - April 2022 (2 comments)
Things You Should Never Do (2000) - https://news.ycombinator.com/item?id=23725867 - July 2020 (83 comments)
Things You Should Never Do, Part I (2000) - https://news.ycombinator.com/item?id=6327021 - Sept 2013 (154 comments)
Things You Should Never Do, Part I - https://news.ycombinator.com/item?id=3624830 - Feb 2012 (40 comments)
Things You Should Never Do [2000] - https://news.ycombinator.com/item?id=3449953 - Jan 2012 (2 comments)
Things You Should Never Do, Part I - https://news.ycombinator.com/item?id=608431 - May 2009 (40 comments)
https://news.ycombinator.com/item?id=6327021 - Sept 2013 (154 comments)
Every few weeks someone came to me complaining that something didnt work in "my" tool. Thanks to git I could port the old, working code over to the new tool almost every time. Eventually the 10 lines grew to a few hundred again.
Always when the discussion about an edge case came up, he'd dismiss it as "that's never gonna happen".
The of course when those never-gonna-happen things happened, he would absolutely not want the responsibility to fix them.
I suppose I think the reason people think code is a mess is because it _is_ a mess. Just yesterday I saw a team decide to override $PATH in 50 separate files because they didn't understand how to package a python library. (I'm not innocent in crazy stuff either, we're all human).
My experience is that rewrites happen when a team doesn't have anything clear to do instead, so it's more like "teams/businesses with no clear objectives" struggle rather than inherently rewrites.
Perhaps it's rose tinted glasses, but I distinctly remember that Netscape 4 had to repaint the screen on window resize. Like the code was that bad. The product was rushed to market and features were bolted on because there was a browser race/war where quality suffered greatly.
This POV would have been impossible to see if you use Netscape 6 because it was so unusable no one used Netscape 6 except for nerd points and to try out "does my browser support this CSS" tests.
For example, one place I worked as a C and C++ programmer, they hired lots of recent grads from a top CS program. Unfortunately, production-grade C code is much harder than most people think it is. The bug reports in core code used throughout the system often to lead to functions that weren't even on the right track for the quality level of C that we needed. So I started killin' code that needed killin'. Management seemed to approve of those decisions, and I don't think I stepped on any toes. (I suspect that the recent grads who'd written code like homework assignments didn't care if anyone rewrote it, so long as it was done quietly.)
Another time, there was a single Web service endpoint that pretty much had to work correctly, if the company was going to stay in business. So I rewrote it much more methodically and resilient than Web backend code typically gets written.
There was another time I simply walked away from a new consulting project, once I saw the code and realized that it was unsalvageable, and that the client wouldn't understand rewriting. (Imagine an undergrad who enthusiastically kludged together something demo-grade from numerous off-the-shelf components, and now the client "just needs someone to polish it up and extend it".) The functionality could've been recreated rapidly, and rock-solid, in a fraction of the time it would take to start to improve this code. The barrier was political, and since it was a new client, and looking like it would be a bad client, not worth the ulcers to salvage.
Except that you then have nothing that actually runs. So yes, no more bad code, no mess. But also no good code, except the theoretical possibility for shiny new good code.
Refactoring is usually the way to go. Can be painful as well, when you are in the middle of it and not sure if it works out (I am in the middle of it, while also switching to a different graphic engine, but I finally see the light) - but mostly you still have something that actually works. And then you can incrementially improve on it. Seperate things that should be seperated - remove things nobody uses anymore (tricky and hard to be sure). And rewrite parts, that need a clean rewrite.
With my limited energy now (or my more realistic conception of my limited energy) - that process is way more rewarding for me, than giving in to the illusion that if I just could do it from scratch, it will be all beautiful.
Joel mentions the "build one to throw away" mantra, and notes that it can be dangerous, but I'd go further with that: assume the "prototype" you build is going to be the product, and it will never get thrown away and rewritten. Because that's often what ends up happening, and you (and your colleagues) will be much happier working over the next many months or years inside a project where there was at least some thought given to architecture and abstractions when it was started.
The only time where I think a rewrite might be warranted is if you, no matter how hard you profile and work on it, have realized that the framework (or even language runtime/interpreter) you have built on is just not going to give you the performance you need. Even then, you should really be sure that you can't optimize it better, even if that means digging into the framework you're using to make changes. (Digging into the language runtime or interpreter might be a step too far, though, depending on the circumstances.)
Ok, I guess there's one more time: if the architecture of the code is just so completely wrong that you can't add new features or fix issues anymore, and your velocity completely drops to zero. And you are pretty sure that starting from scratch would get you to feature parity faster than embarking upon a series of gigantic refactors. But even then, I think most people underestimate how long that rewrite will take.
But otherwise... nah, it's probably not worth it, unless you're doing it on your own time, as a toy project, for learning or just fun. I might take an old personal project written in C and decide to rewrite it in Rust because I'm sick of C, and just don't care to work on it anymore, even though there are more things I want to do with that project. Now, I'm not saying that's necessarily a good reason to do it, but if I have no professional obligations around that code, it's my prerogative to spend my time however I want.
Off-topic: anyone else remember the Mozilla browser before Firefox? Remember XUL? That was with KDE and Gnome 1.x, when the Linux desktop was just round the corner. Who needs Word when you could use StarOffice....
I think Joel's advice is holding up pretty well.
I mostly agree with the sentiment. Slow, incremental changes, whether to code or even a website's style. Mostly. But sometimes you run across code so malign, so neglected, so undocumented, uncommented, twitchy, troublesome, inexplicable, that the risks of the Second System Effect are worth it.
Really, I think the crucial point comes when your system is a kind of obelisk. You tiptoe around it, make offerings, but you don't know how to appease its wrath. It's like the weather.
Then it is time.
Joel Spolsky is a smarter man than I, but his examples here are table stakes stuff compared to two decades of poorly implemented Java enterprise MVC patterns. And we don't sell software, we sell a service, and that service will continue even if our rewrite team never delivers, so the rewrite can't be more than just an expensive financial boondoggle if I'm wrong.
In my experience, the main reason a rewrite feels like a good idea is that the existing software has grown into what feels like an unmaintainable mess: changing the software in any way at all is so painful that it feels like you'd be better off starting from scratch.
Done right, automated tests dramatically reduce the cost of changing software. You can make a change and feel confident that you've not broken anything outside of the area that you're working on because the (mostly integration) test suite continues to pass.
Likewise, documentation. Without good, comprehensive documentation you quickly find that there are all sorts of areas of the project that you don't understand - which can also make a rewrite feel like it could be justified.
The catch there is that you have to _trust_ your documentation. If you (or your team) know it to be out-of-date you'll lose trust in it - so you have to get to a point where the documentation is thorough and any mistakes in it are treated as high priority bugs to be fixed.
But... if you can get both of these things in order - your automated testing strategy and your documentation process - my hunch is that rewrites will be FAR less tempting.
The new system was to support new requirements, have fewer legacy constraints, and benefit from using more off-the-shelf libraries and frameworks. It was more complex than it was bulky (i.e., cross-domain designs to nail, not tons of rote coding to churn). They had an aggressive timeline.
One risk you might imagine is Second System Syndrome. And there was some of that, more with everyone wanting to jam in every feature, and maybe also Analysis Paralysis on some parts the architecture.
But the biggest problem was that, collectively, that particular team just couldn't build a new system that would come together in a sufficiently timely fashion.
At least part of the problem was that management definitely dropped some balls they couldn't afford to. Had the engineers been organized better, and used more appropriate process, I don't know whether they could've risen to the challenge.
There is not team that is 100% bad so you don't need a full rewrite of 100% of the code base.
What people rewrite is not really bad code, it's just middling annoying muck code.
The first kind was written by people who know about software, has some attempt at following some conventions, variable names, etc. etc. It's not "clean and shiny", but you can see someone tried. Like if a bunch of first year apprentices built a house. They tried their best, and often did a half reasonable job - at least for a first timer. Plenty of companies are using this kind of software to make many of millions of dollars. I would say this is extremely common in the real world.
And then there's codebases that consist of a single source file that has hundreds of thousands of LOC and that file is called indexNEW.php, indexNEWNEW.php index2022-1.php (and so on). Nobody who works on it has ever studied any kind of software course at any kind of school, and literally nobody has any real understanding of what/why/how, because it gets passed along like a hot potato everytime someone quits due to the insanity. Nobody who works on it even knows what source control is, has no idea why you shouldn't copy and paste code, and has never heard of TDD or automated testing or bullding or literally any software discipline. This isn't like first year apprentices building a house, this is like Homer building the Canyonero - an utter disgrace that just barely functions at all. [1]
I wouldn't be surprized if Joel is talking about type one, and doesn't even really consider type two to be "software" or "a system". It's just a stinking pile of garbage.
(I worked at a very large Telco. The vast majority of code plugging the systems together was type two - it was a nightmare of spaghetti so bad that when a power outage and failed backup generator took down everything (including 911), it took more than two weeks just figure out how to start everything back up again.)
Imagine a file with 100,000 LOC, some commented out, copies of functions like "addCustomer()" and then "addCustomerNEW()" and then NEWAddCustomer() and then "makeCustomerDanGNewest()" and so on and so forth. Absolute gong show.
However there is a time rewrite is a good solution, that is when you find a lot of the functionalities the old code base provides are NOT actually needed. In this case, it is perfect (and you should rewrite it), because you are writing a much simpler software and you have a much greater chance of success.
Also how messy are we talking? I've seen code at a bank where a junior contractor implemented, in reams of code, his own database locking primitives, apparently unaware that the db could handle concurrent accesses on its own.
Rewrites aren't such a big issue these days, this seems a little dated. The real issue where we get painted into a corner now is architectural, say if you went really hard down the microservices route and tried to back up out of it.
What other good reasons can you suggest to do a rewrite?
Rewrites often have the goal of "this time we are making everything perfect".
Which leads to tons of unnecessary abstractions and endless design proposals.
For Netscape 6 it was the idea of introducing XPCOM. "Classes are stupid everything is an interface".
Which later led to the "DeCOMtamination" project.
But that's the thing: a lot of development involves not writing any code, as you re-think your abstractions, plan the architecture of the next bits you'll write, debug what you've written, etc.
So I don't think we're necessarily talking about speed when we say that it's harder to read code than to write it. I think reading -- and truly understanding -- code (especially when it's someone else's, or even yours, that you haven't seen in a long time) can require quite a bit more mental effort than writing code.
Yes, you need to know what you're doing if you plan to rewrite anything, and make sure that the people who know the little legacy details of your system are around, but it's absolutely the right thing to do in many situations.
Yet, they do useful work and just abandoning them in-place and rewriting them from scratch is rarely a good solution for these problems.
“They did it by making the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.”
some people took the “any software company” to heart and now apply the same principle even in those smaller contexts when sometimes the code really is that bad and rewriting the whole thing really is by far the best plan. I’ve seen very senior people citing the Netscape case study to support a no-rewrite position while discussing a 10kLOC system that could be rewritten from scratch in a couple of weeks by a single developer.
Sometimes stuff is broken so badly you better start over
A perfect pairing.