Why We Threw out All Our Code (And Why You Should Too) | Hacker News Reader

Why We Threw out All Our Code (And Why You Should Too)

blog.nowjs.com

92 pointssthatipamala64 comments

Throwing out all your code and rewriting it from scratch is like demolishing your building and constructing a replacement from scratch. It's usually not the right thing to do, it occasionally and rarely is the right thing to do, and it's always the tempting thing to do.

It's stupid to argue, "You should never tear down a building" (like http://www.joelonsoftware.com/articles/fog0000000069.html implies) and it's also kinda stupid to write an article entitled, "Why We Tore Down Our Building (And Why You Should Too)"

Neither is universally right. The right thing to do in a given situation is a difficult and complicated question to answer, and it depends on the building in question: how big it is, how well it was designed, how well it's been maintained, etc.

Right.

In this case it was 3 months worth of code. I think the analogy is closer to "we tore down this clay hut we built". Easy to build, easy to tear down.

If this were a mature code base, I would be a little more shocked.

Or maybe closer to "We're putting this model building in the closet since we built the real one."

How often do you build a prototype building? Or a building as a side project?

Your analogy fails, sir.

It's easier and more common to write dirty code that functions as it should than build a building with poor foundation but that looks and functions as it should.

People build temporary structures all the time -- event tents, for example. And plenty of people finish a room, redo a bathroom, build a treehouse or a shed, etc, as a side project. And believe me, it takes a high level of expertise to inspect a building. Many of the most important kinds of flaw are only evident to the eye of an expert.

Yup, but a tent is not a building. And going from a tent to a building requires you to take apart the tent in almost all cases, no?

I think it's a good one. Think of commercial real estate. You have an old shopping center that is cash flowing. Its maintenance costs are high, and the rent is low. Once upon a time it made sense for the market and the style was perfect. Now it's out of date and the units are not exactly what people are looking for.

You could tear it down and build a fancy new one that would cash flow even more with higher rents and lower costs. It's exactly what the market needs today. Green energy, nice, open spaces, new paint. Do you break out the bulldozers?

It depends, of course.

> How often do you build a prototype building? Or a building as a side project?

This is one of the most amazing things about rapidly growing cities in most of the world. Neighborhoods that start out as informal shanty-towns full of lean-tos with leaky roofs become, over the course of years, first neighborhoods with modest but permanent cement block buildings and tin roofs, and sprout all the typical urban amenities like power lines and sewers (often at first pirated ad-hoc from other nearby neighborhoods), have their roads paved, etc., and a few decades later look like normal neighborhoods, sometimes even surprisingly gentrified ones.

Absolutely, as someone who grew up in a city like Mumbai, I know much about shanty towns and from my knowledge of them, they are far from ideal. ie. They are not know for a strong foundation, cleanliness, longevity etc. They are constantly hacked and patched by its residents to ensure basics like protection from rain, electricity and basics like a toilet.

And that's how New York and Rome and Shanghai grew into world cities. They didn't pave them over and start from scratch. They grew out of shantytowns.

But wait, we were talking about buildings, not cities. A city is made up of many buildings. And buildings are torn down across cities on a daily basis.

Right, and chunks of code are thrown away on a daily basis.

The question is: which is the more appropriate metaphor for a project? A building or a city?

I agree with the position that it depends, code may be a building, or a city or any number of things.

I disagree with the OP that deciding to recode can simply be compared to tearing down and making a new building for no reason. That's oversimplification.

First, there are times when buildings must be demolished and built from scratch. The OP makes this idea seem ridiculous but it happens everyday when buildings are in fact demolished and rightly so. Second, fewer buildings are demolished and built from scratch relative to programming projects demolished and built from scratch due to the inherent differences between a coding project and a building project. For one, the former is more regulated and carries more physical risk than the latter(a coding project).

People might think coders are a bit self-important if we start analogising our project's as cities rather than buildings.

Maybe some projects are more building innovation centres, that later go on to be used within actual buildings, and some don't get used.

My local zoo actually built a full-size prototype of their new penguin exhibit ahead of building the exhibit itself. Once people could actually walk around in the space, some improvements were obvious.

Here's an example:

http://www.vt.edu/about/buildings/surge_space_building.html

"The building provides space to temporarily house academic or administrative units that are displaced because of renovations to their home buildings. The facility was designed using a pre-engineered steel frame with its exterior clad in metal panels and precast concrete-like panel accents. The building is designed to be disassembled and recycled in 10 or 15 years.

One floor; 45,000 square feet. Find a public access defibrillator in the hall intersect."

>"like demolishing your building and constructing a replacement from scratch. It's usually not the right thing to do" Actually, tearing down a building and rebuilding is very often the right thing to do, but isn't done for the same reasons people don't toss their code and rewrite from scratch - sunk costs.

Unlike software, new buildings are almost invariably significantly more efficient and less resource intensive than old ones - and green building has, of course, gone from a fringe movement in the wake of the Rio Summit, to the mainstream and is now standard for public and corporate buildings.

Like poorly written software old buildings are expensive to operate and maintain (and unlike well written software buildings still deteriorate over time when left only to their own devices). Finally, like software one must go through old buildings and figure how systems were designed and installed and patched in order to modify them - and this is very time consuming. Per square foot costs of renovation are commonly as much or more than new construction.

They threw out 3 months of work... That's not a rewrite, that's a prototype. People do this all the time.

+100 - the biggest questions (to me) were HOW MUCH code did they throw out, and how old was it? Throwing out a few months of code you just finished writing (and which you're still intimately familliar with) is vastly different to throwing out a code-base that would be entering middle-school if it was a person.

I don't know what the old codebase is like, but the current one is just 2145 LOC, not counting tests. 62K total.

I know that startups do way more that just write code, so I'm not criticizing them at all. And it seems they're not afflicted with the typical engineer modesty when it comes to their own codebase, which is a good thing. On the other hand it's not exactly a superhuman hacking achievement to rewrite this from scratch.

So if we're charitable and say the new codebase is, oh, three times as compact, that means they threw out about 10% of their code.

Netscape 6 is (or was) the oft quoted example of why not to do this. All that old code may seem old and boring, but it has a huge amount of knowledge, wisdoms, learnings, fixed mistakes, performance improvements, great ideas, deleted bad ideas and security fixes built into it.

When you throw out your code you're throwing out all that knowledge on the assumption that you can build it all back with improvements.

Evolving an application from a known good state to a known better state is usually the best approach unless your codebase is small or your app has no traction.

Rewriting your app from the ground up is like getting divorced and dating a new girlfriend. The first few months are a lot of fun until you figure out how much you lost.

Without Netscape 6, we wouldn't have had Phoenix/Firebird/Firefox, and probably wouldn't have IE7, IE8, IE9 and KHTML/Webkit would have come about very differently and all these would have had much less competition.

Netscape 6, or at least the mere fact that it existed and worked well enough and the fact that it was the first browser ever that did standards well and focused on not much else other than standards, made it the most important browser ever created.

Netscape 6 was the most revolutionary browser project ever created and had the development effort behind Netscape 6 not been done, we would possibly have had a much worse very IE-only web today.

Never mind that beyond all that, Netscape 4 was terrible and deserved to be thrown out.

> Netscape 6 was the most revolutionary browser project ever created

I dont disagree with the general sentiment of your comment. But as someone who attempted to use Netscape 6 at various stages of its development, I fear that statement might be stretching facts a little bit.

A few people have pointed out that it’s easy to throw out your code when you (a) have tests for it, and (b) only have three months invested in it. I agree with both of those points, but want to tip my hat to a team that rewrites their code after three months.

The reason some code bases can’t be rewritten after a few years is because when the code was only three months old, the team passed on the chance to rewrite it while it was still a tractable exercise.

Very much agree. An extra layer on the danger cake (for web apps) is if your user base grows slowly while keeping the business profitable. Suddenly (sic) you hit real capacity limits after n years, and have no idea how the "organically grown" code in question works.

I've never used NowJS nor looked at the source for it, so I can't comment about their justification for a rewrite.

But most programmers I know don't need any encouragement to rewrite stuff, myself included!

That's the attitude I am trying to avoid these days. When I look at something that I want to rewrite I now ask the following questions,

1. Do I have sufficient tests and testing to do this without breaking things?

2. Do I want to want to rewrite because I cant be bothered understanding it?

3. Is there going to be any gain now or in the near future by rewriting it?

4. Will I be able to rewrite it sometime in the future as part of another piece of work which actually delivers value?

Unless I come up with Yes, No, Yes, No I tend to leave it alone. There are other things of value I can do with my time.

EDIT - Formatting...

Thank you, these criteria just saved me from tearing down a building.

3. Sunk costs and egos get in the way of good software

Egos are what drive people to throw away their entire codebase and start over.

Depends on the angle. The person that wrote the codebase is usually the one holding onto the old codebase due to ego. On the other hand, new people that come in and don't like things can be full of ego in the sense of, "I could program this better without any limbs!"

Egotistical engineers always think this time things will be different, because they're smart enough to do it right. The reality is that every rewrite brings a new and different set of problems that need to be addressed. You don't escape problems by scrapping it all and starting over and this is one of the most valuable lessons to learn as an engineer.

You don't need to be egotistical to see that you can do things better the second time around because you've learned stuff along the way. In this case, the writer provided multiple examples of how they were able to use the lessons they'd learned from the old codebase to improve the new one. They did, in fact, escape some of their problems. Did they introduce new problems? I'm sure they did, but they're probably still better off overall.

Sometimes ego is involved, but sometimes rewriting is just the rational thing to do from a cost/benefit standpoint.

Egos and lack of skill.

It's so much easier to write from scratch than it is to refactor. So many programmers have no idea how to tackle an existing codebase, even if it's one they've wrote themselves.

Very important clarification, you are quite correct.

Justified or not, I think a project owes it to its users to change major version numbers when a major rewrite (much less a complete one) has occurred.

As perfect as your code might be, customers deserve a hint when major changes were done under the hood. Moving from 0.6 to 0.7 screams "just a few changes, some new features", not "we threw everything out and started over". This should have been made into "1.0beta", I would think.

There are other factors involved when deciding a version number than the code under the hood: API changes, new features, is it production ready, 'this is the now stable API that we'll be maintaining for a long time', etc.

Edit: for perspective, a thoughtful proposition on how to approach software versioning: http://semver.org

I agree that there are many factors. I'm just saying that some changes are important enough to trump anything else; the presence of a rewrite should decide the version all by itself.

sigh; a headline like this is little more than HN linkbait. As others have mentioned, throwing out all your code is generally a bad idea, especially if your project is of significant size. How many lines is NowJS anyway? I doubt it was much of a loss.

I'm still trying to figure out what NowJS does other than expose an RPC call using socket.io. All of the features on the website are actually socket.io features.

Throwing it away is easy when there's not that much of it.

My codebase is 10s of thousands of lines, it evolves as I get better but I'm not going to scrap it.

Please bear in mind the counter article everyone is citing (http://www.joelonsoftware.com/articles/fog0000000069.html) was written 11 years ago. I'm not saying that it's wrong because it's from 11 years ago, but you really want to consider its central example.

Netscape 6.0 was a complete rewrite. The article claims that this was a mistake.

Now, if you would recall, one of the reasons Netscape 6.0 was completely rewritten was to use Raptor/NGLayout. Raptor/NGLayout was later renamed to Gecko. Gecko is the currently developed Firefox rendering engine, and Netscape 6.0 marked the start of the Mozilla foundation.

I would harshly argue that Gecko would not have taken off nearly as well without Netscape's rewrite. But this is definitely not a story about Gecko's popularity. It's about the reason why Gecko was the decision they took at the time, and the current impact of Gecko.

Besides the fact that Gecko was one of the early open-source success stories, the reason why Netscape decided to scrap everything and start from scratch was simple: Netscape 4.0 was horrid. Remember back to the days of IE vs Netscape. Netscape was significantly slower, and had no dynamic HTML among many other features. It was buggy and would crash frequently. And the development on the "continuation" of 4.0 code was simply too slow to compete, because of outdated, not modulated code, and stuff stuck in that simply "worked," and never changed to allow for future changes.

Now, people attribute Netscape 6 to Netscape's death because of this radical decision. But take a look at Netscape's usage statistics: http://upload.wikimedia.org/wikipedia/en/1/16/Netscape_Navig... . Hey, guess when Netscape 4.0 came out and IE started to dominate? 1996-1997. By 2000, when Netscape 6 was released, Netscape's usage already dwindled down to nothing.

Now, you're thinking, why didn't Gecko save Netscape? It takes too long to reverse these tides. Netscape was already dead. AOL acquired Netscape in 1998, to combat IE. Then, in 2003, AOL won an antitrust suit against Microsoft, and allowed AOL to use IE royalty-free. Netscape was pretty much scrapped at that moment because the damage had been done.

But wait.

Mozilla released Phoenix in 2004, based on Gecko. You should now the story from there. (Clue: Phoenix -> Firefox)

Disclaimer: I'm not saying that Firefox > IE or Chrome or whatever. I'm saying that the codebase Netscape 4.0 was based on was doomed to fail. Gecko still lives on today, and was started as a rewrite of Netscape.

Anyway, my point is, there are times to rewrite things, and there are times to keep them the same. If you have a good, legitimate reason to rewrite, as Netscape or NowJS did, you should. If you are just the stubborn programmer that always implements his own Queue or QuickSort when libraries exist, don't.

Holding onto code just because is a foolish notion. Sometimes approaching things with a fresh perspective is what's best - it allows you to apply what you've learned so far and actually improve not just one section of code, but the whole kit & caboodle.

This is one of the reasons you should write tests as you develop - it ensures you have a consistent baseline you can apply.

As a developer you are constantly learning & evolving. If you're not confident enough to draw a line under what you've done, step back and evaluate your position and rewrite it when needed - that's fine, but others find it an effective approach to building something bigger, better and more sustainable.

I think that this applies to the technology you're using as well.

I've been thinking about this because I'm writing an application in Rails and realized relatively recently that it would be better suited as a Node application. My motivation for using Rails was that it was what I knew the best. Luckily for me, the most time-consuming part was the front-end, not the Rails part, so I'm hoping my switch to Node will be relatively painless.

Throwing away Netscape 4 and rewriting it from scratch wasn't a bad decision because nothing good ever came of it; it was a bad decision because it killed the company. The shattered remnant of Netscape that hobbled across the finish line to release "Netscape 6" wasn't even Netscape anymore, it was a rapidly fading division of AOL, IIRC, that just stamped its name on a Mozilla product it had little to do with producing at that point, to widespread derision.

As bad as Netscape 4/5 may have been, while I can't guarantee trying to refactor instead of rewrite would have gone better, I can guarantee it couldn't have had any worse a result than the Total Death that was the actual result.

And the phoenix-return of Mozilla is the exception, not the rule.

To be pedantic, there is at least one fate worse than the Total Death we observed: Total Death + no Gecko.

Even though ths external case is stronger, this holds true even from the company's perspective. AOL continued to release Gecko-derived browsers through 2007. They got something from that (even if it was just AIM installs and propping up the fading Netscape brand for other purposes).

In support of your comment, I think some are forgetting how horrible Netscape 6 actually was to use. (Or am I just getting old? It wasn't that long ago!)

Compared to earlier versions it was incredibly slow—even by the standards of the time—and crashed a bit more frequently. It took much longer to load or perform some common tasks. Sure, the rendering engine was better for many pages, but browser sniffers were still common, and (like now) weren't updated often enough, so bad versions of sites got served occasionally too.

There were a host of reasons why Netscape 6 was bad from a user's perspective. Spolsky has it right when he says Netscape 6 was a piece of junk. Let's not allow time passing to permit us to remember it more fondly.

Sure, future versions got better, but that was due to he creation of a new "corporate" memory. It took a while to get over having discarded so much learning.

Well, just because some prominent luminary says it shouldn't be done never means it's always a bad idea. A lot of the basis against rewrites are for relatively mature projects still on a high growth trajectory or large installation base. However, there are exceptions to that as well. Foursquare went from LAMP to Scala/Lift, PostgreSQL, and MongoDB with minimal complications and it's paid off handsomely to support their amazing growth.

Also, the money quote: "It's managed to survive our vicious benchmarking tools and our ridiculously comprehensive test cases. " My assumption is they are using the same languages and tool sets (not porting from language X to Y).

I'm curious as to how long the rewrite took though.

Rewrites are always fun and I applaud any man brave enough to undertake such an effort.

I've often been faced with a situation warranting a rewrite, but I never mustered the courage to actually go with a full rewrite I don't think. Gradual rewrites are much more my thing.

I've been really really tempted to do one, but sadly the pressures of the job take precedence and so it usually ends up being the case that you stick with the code you've got and try and hack extra things in, knowing full well you really don't want to and aren't at all happy with it.

So my 'rewrites' sometimes come in the form of superficial organisation. Looks moderately nicer, doesn't work any better.

I guess it is courage though. You don't really know that your rewrite will be an actual improvement, or if you'll even pull it off. Brave move indeed.

It's managed to survive our vicious benchmarking tools and our ridiculously comprehensive test cases.

I guess they did not rewrite those from scratch. If so, I would guess they only rewrote, at the most, 25% of their code.

Once you start making revenues from your code, a complete rewriting is probably the worst thing you can do for your company. Sometimes it might be warranted, but customers care about solving problems, not about technology.

Chances are if you wrote properly modularized code, you can rewrite modules one by one, which is perfectly legit. But just dropping everything and trying to rewrite a product from scratch is a mistake that a company usually only makes once (before they die).

Especially when getting into a codebase that is not yours you are tempted to think "why dont i just rewrite this in a better an cleaner way". Then you go on todo that and fall into most of the pitfalls the original author fell in before, have to rethink every idea behind and it usually is alot more work than you think. That said it depends heavily on the size of the project and a bunch of other things, for a 3 month old project it might make sense.

You should read the book, "In Search of Stupidity - Over 20 Years of High-Tech Marketing Disasters"

Refactoring is almost always preferable to throwing out all your code, imho.

I'm glad they're happy with the rewrite, but honestly, this makes me feel a bit nervous about updating my Node app utilizing NowJS to the 0.7 release.

Maybe it is better. Maybe it is more stable, smaller, and faster, and well-tested. But it hasn't had as much "real world" use thrown at it as the previous codebase had.

I'm not saying I won't upgrade, but I'll likely wait a bit until I do and stay with what I know works right now, at least in production.

Frequently, throwing out "all your code" is a sign you did pretty much everything wrong to begin with. Personally, the sign of a decent design is when you can wind up doing the old "Grandpa's Mattock" trick:

"This is grandpa's mattock; my father replaced the handle and I replaced the blade".

I'd be dubious that they actually rewrote everything - having working stuff lying around that already worked, some of which was probably just fine, I'd hope it got copy-pasted into the new code...

Relevant: Things you should never do - http://www.joelonsoftware.com/articles/fog0000000069.html

Whether their decision was right or wrong, they made a decision that big companies tend not to make. Startups should probably have a bias towards decisions like that.

An attention grabbing headline but all the same I have been using now.js and its been a great tool (although a cut down on the print statements would be nice).

If you're dealing with human DNA, a complete code rewrite will likely fail.

But if its just an amoeba and you have a lot of time, go for it.

It's probably easier to do this sort of rewrite if you've already got "ridiculously comprehensive test cases".

Anyway, congrats on 0.7 Sri!

Be ready to throw away your first version. I wish I had the right quote on hands. The K in K&R said that.

Its not always the right thing as there are years of issues/corner cases that have been resolved.

Reply on news.ycombinator.com