How many startups are actually killed by tech debt?
I saw many successfull companies with shitty software.
I had the impression the business part of things killed much more companies if they didn't get it right.
I saw many successfull companies with shitty software.
I had the impression the business part of things killed much more companies if they didn't get it right.
[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-...
> If you are writing code experimentally, you may want to rip up the function you wrote last week when you think of a better algorithm. That’s fine. You may want to refactor a class to make it easier to use. That’s fine, too. But throwing away the whole program is a dangerous folly, and if Netscape actually had some adult supervision with software industry experience, they might not have shot themselves in the foot so badly.
But I wouldn't go so far as to call it the sole reason the project failed - that just sounds too easy an explanation.
The from-scratch rewrite is a classic response to declining productivity. In the metaphor of technical debt, it's equivalent to declaring bankruptcy.
As Jamie Zawinski wrote in 1999: "We never distributed the source code to a working web browser, more importantly, to the web browser that people were actually using. We didn't release the source code to the most-previous-release of Netscape Navigator: instead, we released what we had at the time, which had a number of incomplete features, and lots and lots of bugs. [...] The code was just too complicated and crufty and hard to modify [...] By being a cleaner, newly-designed code base, so the theory went, it was going to be easier for people to understand and contribute. And this did get us more contributors. But it also constituted an almost-total rewrite of the browser, throwing us back six to ten months. Now we had to rewrite the entire user interface from scratch before anyone could even browse the web, or add a bookmark." -- https://www.projectseven.com/grafitti/october_2001/zawinski....
He doesn't use the "technical debt", because that didn't start becoming popular until 2004 or so: https://books.google.com/ngrams/graph?content=%22technical+d...
but if you look at Ward Cunningham's original 1992 description, it sounds pretty congruent: "Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation [...]"
> There’s a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:
> It’s harder to read code than to write it.
> [...] Even fairly major architectural changes can be done without throwing away the code. On the Juno project we spent several months rearchitecting at one point: just moving things around, cleaning them up, creating base classes that made sense, and creating sharp interfaces between the modules. But we did it carefully, with our existing code base, and we didn’t introduce new bugs or throw away working code.
> A second reason programmers think that their code is a mess is that it is inefficient. The rendering code in Netscape was rumored to be slow. But this only affects a small part of the project, which you can optimize or even rewrite. You don’t have to rewrite the whole thing. When optimizing for speed, 1% of the work gets you 99% of the bang.
> Third, the code may be doggone ugly. One project I worked on actually had a data type called a FuckedString. Another project had started out using the convention of starting member variables with an underscore, but later switched to the more standard “m_”. So half the functions started with “_” and half with “m_”, which looked ugly. Frankly, this is the kind of thing you solve in five minutes with a macro in Emacs, not by starting from scratch.
Once the technical debt is so deep that bankruptcy seems like a good option, digging out is also not a great option. It's certainly possible now if it wasn't then; people like Michael Feathers have done great work charting paths out of an accumulation of garbage code. But at the time of JWZ's resignation letter, Martin Fowler's book Refactoring had been out for all of 3 months.
In particular, Netscape was faced with a choice: they could either do a total rewrite quickly, or take much longer to do a slow rewrite while still pushing forward, but at a much reduced development speed. That's fine if you're some large enterprise in a stable field with a solid revenue stream. But a startup competing in the then-fast-evolving world of the Internet? Competing against the monopolistic giant of Microsoft, who was desperate to dominate the space?
There's little reason to think a slow cleanup would have been any better than a fast rewrite for Netscape. Especially if their solution involved open-sourcing things, which a) requires a non-awful code base, and b) reduces the pace of refactoring and cleanup. So I think it's fair to say that either way Netscape was killed by technical debt.
Nothing has fundamentally changed between now and then. Sure, the information may be more readily available, and we can argue that it's unfair to judge Netscape by today's standards, but even so, I think the article you linked makes a strong argument that technical debt did not kill Netscape, and that their approach to managing it did. This is why it goes on to discuss other, more successful approaches.
> In particular, Netscape was faced with a choice: they could either do a total rewrite quickly, or take much longer to do a slow rewrite while still pushing forward, but at a much reduced development speed.
Were they actually faced with this choice? The article you linked points out there are at least 3 reasons people choose to do rewrites. This is one of them ("architectural problems"). Even still, the article's argument is that this is a false dichotomy, and that there are better alternatives.
I also think "total rewrite quickly" is an oxymoron and a bit disingenuous. Your article points out it took them three years. In my experience, they typically take full-blown products at least two years - if they are successful at all.
If you are worried about Microsoft beating you because you are slowed by technical debt, you certainly need to be worried about Microsoft beating you because you didn't ship anything for three years. That is the entire point the article makes. The very first sentence reads:
> Netscape 6.0 is finally going into its first public beta. There never was a version 5.0. The last major release, version 4.0, was released almost three years ago. Three years is an awfully long time in the Internet world. During this time, Netscape sat by, helplessly, as their market share plummeted.
I find your next point surprising, given your citation:
> There's little reason to think a slow cleanup would have been any better than a fast rewrite for Netscape.
The article you cite makes this exact argument:
> You are putting yourself in an extremely dangerous position where you will be shipping an old version of the code for several years, completely unable to make any strategic changes or react to new features that the market demands, because you don’t have shippable code. You might as well just close for business for the duration.
It then goes on to list alternatives, including a "slow" rewrite, during which time you can still ship code.
None of this is to say that too much technical debt can't slow you down to the point that you cannot keep up with your competitors - but it is certainly not the argument your citation makes. Its argument is tangential, and related only in the sense that technical debt is presumably what pushed Netscape to make the decision it did.
I have to say, I am fascinated by our difference in takeaways from this article.
A total rewrite, if done well, is going to take less resources and way less calendar time than a refactor-and-cleanup approach done equivalently well. That's what I mean by quick. Was Netscape's rewrite done well? I doubt it. But there's no reason to think their refactor-and-clean up would have been any better. Which again leads back to my point: they were probably fucked either way at that point. Note that in the ensuing years, nobody has made major direct revenue from selling a browser.
The thing that has fundamentally changed versus 20 years ago is major advances in dealing with existing code. Back then, writing tests was extraordinarily rare. Now if it's not the norm, at least it's common. Things like continuous integration are common. Builds and tests are much faster, and the amount of computing power available to an individual dev is hugely improved. The tooling advances are incredible as well, from debuggers to languages to runtimes to IDEs with built-in understanding and refactoring of the code. All of this makes incremental cleanups much easier that was possible in 1999 with a 1994-era mishmash of C and C++.
> A total rewrite, if done well, is going to take less resources and way less calendar time than a refactor-and-cleanup approach done equivalently well.
This is totally up for debate, and largely what Joel's article is talking about. Does "less resources" include opportunity cost? You may be able to get something out in less calendar time, but will it be as feature complete? Will it get rid of the old bugs that plagued the product? Will it avoid introducing new ones? There are many reasons to believe that a refactor would be better - for most projects this is probably true (at least in my opinion, and evidently in Joel's too!)
I, like many others, have seen the disastrous results rewriting from scratch can produce - if they produce results at all.
I'm not sure that the time builds take has any impact on which approach is faster. Certainly tooling advances might - but it's unclear to which approach the advantage is given. And while tests certainly make refactoring far easier, this can often be done as part of the refactoring.
And let's also remember that the article cited is written 20 years ago - the timeframe you're referring to. This idea that rewriting from scratch is dangerous and costly is not predicated on recent advancements in tech.
We can argue whether Netscape's approach to rewrite from scratch was the right approach to take, and we can also argue about whether rewriting from scratch is generally appropriate, but the intention of my original comment was to point out that Joel specifically makes the argument that it was rewriting from scratch that killed Netscape - not technical debt itself - and that there are alternatives which would've been far less risky (from his POV.)
It’s difficult to imagine Firefox having the success it had (being a browser for the modern web) if it was based on the Netscape 4.7 source code. I really think they had no other choice.
> Even fairly major architectural changes can be done without throwing away the code. On the Juno project we spent several months rearchitecting at one point: just moving things around, cleaning them up, creating base classes that made sense, and creating sharp interfaces between the modules. But we did it carefully, with our existing code base, and we didn’t introduce new bugs or throw away working code.
> A second reason programmers think that their code is a mess is that it is inefficient. The rendering code in Netscape was rumored to be slow. But this only affects a small part of the project, which you can optimize or even rewrite. You don’t have to rewrite the whole thing. When optimizing for speed, 1% of the work gets you 99% of the bang.
Quite frankly, Netscape was very good for its time. Firefox today is also drastically different than Firefox 1.0 ("It’s difficult to imagine Firefox having the success it had if it was based on the Firefox 1.0 source code").
Day 1
Stakeholder: "The app is down, what is happening?"
Engineering lead: "We are working on it. Do not worry".
Day 2
Stakeholder: "The app is down, what is happening?"
Engineering lead: "Do not worry, we got this".
Fast forward to Day 15:
Stakeholder: "Can you explain why the app is down again?"
Engineering lead: "We got this"
Stakeholder: "That is what you said last 2 weeks ago. We will send someone to take a look at it"
External auditor: "wtf wtf wtf. These guys do not know what they're doing."
Day 16:
Stakeholder: "Say hello to Bob, he is your replacement. Please work with him making sure he has all the context he needs during this transition".
Engineering lead: "But, but... it's just technical debt"
Stakeholder: "You may leave now"1) Some people mistakenly interpret "maximizing the work not done" with maximizing tech debt.
2) Power is the ability to make people do what they do not want to do. Saying "no" is a way to exercise that ability. Some leaders become addicted to the feeling of exercising their power.
3) Cohesive groups of people tend to prioritize cohesion over rationality during decision making (i.e.: "groupthink"). Groupthink makes teams unable to make correct decisions or assess risks in a rational manner.
Many startups suffer of issues 1, 2, 3. And some even perceive those weaknesses as strengths. When that happens, the only way out is to decapitate the team, punish groupthink and reindoctrinate the team members.
Basically 0. If you have a good idea - you can overcome. If you have a crappy idea and an amazing engineering team you are doomed.[1]
Technical debt, if it's relevant at all, is relevant at much later phase than that.
It wasn’t even technical debt in the sense that it was bad code, it was because they prematurely tried to optimise everything, so adding a feature would take twice as long. We also couldn’t add new devs to the project because the team lead’s decision to not document the code, because “code should document itself”.
Bugs could only be identified by the dev who wrote that feature because the code was so obscure and undocumented.
I am looking for projects that are very small.
In projects with a lot of business logic, or projects that require understanding of a specific subject, it’s often better to write comments to clarify what’s going on. Sometimes code alone isn’t enough.
- Performance optimizations (always worth mentioning why you ugly'd up some code for a necessary speedup)
- Kludges where you're working around something in an external library or service
I've also had this experience, where a product accumulated so much tech debt (in part, due to my decisions) that it was simpler for the business to outsource the functionality. That wasn't the only reason, but it was a big one. When tech debt sand gets in the software delivery gears, the allure of vendors increases. Sure, the business won't get things exactly like they want them, but they won't have to deal with missed deadlines and degraded functionality either.
Well put. Could not have said this better myself. I've experienced situations where the tech debt sand is a consequence of fairly unique business and internal system constraints. The allure of vendors did increase and the company did eventually research vendors, but nothing addressed our snowflake use cases perfectly.
I don't think that's necessarily a positive reflection on our product though. If anything, the lesson I take from that is to build your business processes and teams to reduce as much accidental complexity as possible.
I can’t blame them for doing this tho, I often felt like we spend most of our time discussing implementation, doing code reviews and optimizing code for the sake of optimizing it. Towards the end it was also clear the business just didn't want to deal with our team anymore, they were annoyed we couldn’t just build the feature they wanted.
Code tells me what the coder was doing. It does not tell me why. If the why is not blindingly obvious, it deserves a code comment.
Too much debt slows you down, and if you don't move quick enough as a start up, the money stops. It's that simple.