Hacker News Reader

"However, startups don't fail because the codebase is ugly, or convoluted, or not following best practices."

Yes, they do.

The obvious one is one senior developer who writes a bunch of trash code to get stuff done in a hurry. Later is asked to maintain it and add features. But it's no fun cause it's a pile of poo. New shiny attracts his attention and he moves on (cause, you know, he delivered at his current job!). New developers try to pick it up, including a new hire, try to work with it. Warnings about runway loom. Support is swamped and many of the tickets get kicked up to developers because support can't answer because they're obscure bugs. Most of developers time is spent trying to fix the worst bugs, but things just get worse because each bug fix introduces new bugs, cause the code-base is well neigh incomprehensible. Some developers see the writing on the wall and flee, leaving even more work for the remaining developers. No money for new hires. 3 months later, layoffs. 1 month later, closed. One poor guy is laid off 4 months after being hired.

Lather rinse repeat.

The upshot is crap code makes a crap product. Just like crap engineering makes a crap car. Customer do care about that. They'll get tired of the bugs and the infrequent updates and the poor support and eventually they'll move on.

> The obvious one is one senior developer who writes a bunch of trash code to get stuff done in a hurry. Later is asked to maintain it...

There's some survivorship bias at play in this; it disregards all the startups that never reached the "later" point because they were too busy polishing the code.

Someone should publish a series on such startups. Intuitively it seems there might be some. But I haven’t seen any articles naming names, describing details.

1) One SF wifi mgmt. software startup had a 50,000 LOC product with 250,000 lines of test code.

They seemed content, but obviously a lot of resources went into tests.

2) Many late startups in SF spent a lot of time and effort on perfecting CI/CD software (multiple years), or struggling with k8s in the early days (1 year to finish one service.)

3) Often post-founder programmers these days have a lot of process to overcome before shipping. I know one startup that hired dozens of programmers, but the founder (alone) still writes most of the code.

We can debate whether struggling with k8s and CI/CD would fit in the category. These are tooling not product but all part of the use of engineering resources. See the “choose boring technology” posts/articles.

I've never worked on a project o bad I could add features fix bugs to it.

And I worked on a project that literally saved the HTML on the page into a varchar, had 15,000 lines of JavaScript, and all the business logic was written in awful stored procedures.

Whatever speed quality trade-offs a good engineer is making will be worth it when a startup is still in the "I don't know if this company will exist in 6 months" phase.

Well when you can't deliver a working product to clients because your code is so bad it can really matter. And the thing is any half decent developer is not going to code that way even within hackathon time scales

Can you name a company that failed because of a bad codebase as the number one reason?

I feel like people walk through hypotheticals like that, but I've not heard people say "Company X failed because of that scenario."

Friendster imploded during its growth stage due to either bad code or not enough spending on servers.

+1

Having worked on the support side of this dynamic, mostly in problem and incident management, I have honestly seen 12 months of product roadmap completely blown up by stability issue after stability issue that our support staff could not possibly workaround. It's really corrosive to the morale of the support staff to just be unable to do anything to help.

In my case this was a relatively mature company on a version 2 reimplementation of an existing product, so it wasn't a complete death sentence. It eventually led to a complete house cleaning of CTO and product management in the end though. But, the company sure didn't fail!

> In my case this was a relatively mature company on a version 2 reimplementation of an existing product, so it wasn't a complete death sentence.

OP specifically referred to startups, though, not mature companies. Most startups only have 12-24 months' runway to begin with, and lacking clean code/testing/best practices/etc won't (usually) kill them in that timeframe.

Lack of product-market fit, on the other hand, is the one thing guaranteed to kill a startup in that first year or two (lack of financing is the other, but this is usually fixable if you've addressed product-market fit). It's why startups spend every single developer hour on rapid prototyping/iteration/features, rather than refactoring, testing and stability.

It's not black-and-white, though. Some code is so bad that it costs time in the very short-term, because noone can figure out how it works. Some functionality is so central to the product that tests are needed to confirm it's actually functioning as intended.

There's a vast spectrum between "fully tested, clean, reusable code" and "held together with sticky tape and segfaults every other minute".

I'd say that the one key skill for a startup CTO is deciding where the team should be on that spectrum on any given day.

The software is not really doing what the customer wants if it does not work reliably. The company dies because their the product they sell does not do what their customer wants. If you buy a car, you care about the quality if the parts indirectly. No one wants a car that works 50% of time. Same goes for software. So yes, poor code can kill the company, because the product does not work.

"However, startups don't fail because the codebase is ugly, or convoluted, or not following best practices." Yes and no are both right I suppose, depending on the viewpoint.

Jeff Bezos seems to disagree with you. Ask people about the first 10 years of Amazon. Mark Zuckerberg also disagrees.

Wouldn't this just be a classic case of survivorship bias? For every Amazon and Facebook there could be a number of companies that did similar things and fell apart.

Good code quality doesn’t guarantee success. However, those two data points indicate it may be necessary if not sufficient. There are only 4-5 tech companies operating at that scale with proven business models. Google and Microsoft being the primary others.

Last I heard, Amazon does a 25-way join on their customer tables to print a shipping label. Now that's quite an albatross for developers and DBAs to carry around!

Mark was a perfectionship in releasing a working product. That doesn't mean it was good from a programming standpoint, but it was tested and always worked well.

eBay is the poster child for shitty software. At one point their CTO was requesting one Sun E10k per month ($2 million each fully populated) because of how slow and leaky their Windows code was. The board said no and they had to fix it.

> The upshot is crap code makes a crap product.

You took the sentence out of context. Obviously the product must work, must not have bugs, and must be straightforward enough to be maintainable.

If your code is so bad that the product doesn't work, then obviously no one is going to say you're doing the right thing.

I was speaking to walking the line between good-enough code and perfect-code, not advocating that people write code so bad that the product doesn't even work.

It seems similar to how people usually interpret MVP, which all to often is executed as MP and lacks Viability.

"The code should be bad enough that it allows you to ship as fast as possible" - I agree with the sentiment here, but not the expression. Speed and "validation" is obviously the outcome, and the ends should justify the means. But to say that fast must necessarily equal "bad" I'm not really sold on.

Coupling is an obvious example. If you want to retain agility in your product, the various components inside should be pretty loosely coupled and have restricted domain of responsibility. When you find out the product needs to change - massively - you already have a lot of the bricks you need to make the new product.

On the other hand, it's clearly wrong (in my experience) to go full-bore microservices out of the gate, because you end up spending a lot of time on wiring and infrastructure.

So for me, the question isn't "what's the minimum level of quality?", because at every stage the quality should in practice be pretty good. I know people say "be embarrassed by the first version you put out" - but people also say to be obsessed with the product/problem. Product teams with high standards (in general, but including code) tend to be more successful. If quality work starts taking up substantial effort, that's a pointer to other decisions (choices of technology, for example) being wrong. Again, all IME, YMMV - I don't think there's a clear rule of thumb that applies to all startups.

> But to say that fast must necessarily equal "bad" I'm not really sold on.

In a perfect world, you'd make all of your decisions along the Pareto frontier of speed-vs-quality tradeoffs.

In the real world, you don't have perfect engineers, and the Pareto frontier isn't immediately obvious.

Generally speaking, the more experienced engineers are better at getting closer to that Pareto frontier of speed-quality tradeoffs from the first iteration. It's the less experienced engineers who end up somewhere less efficient on the 2D spectrum of speed/quality. This is where experience pays off the most.

I'm not sure about that either, tbh. Once you're in the process of managing a successful product and have a well-defined value prop; sure - pareto efficiency is interesting. Before that point, I don't think it is - at least, not in any meaningful way.

When you're building out product and figuring out where it fits and what it should do, you want to retain a lot of optionality. If you don't have the ability to change the product quickly in an agile fashion, your development process way well end up being more pareto-efficient overall but your potential customer base would be much smaller. Better to be less efficient and grow a much bigger base.

I think the pareto curve still covers that idea. You want to be farther along the curve towards speed.

Also you may be thinking in terms of a different optimality criterion for quality, which as the above poster noted may be unclear, e.g. a lot of effort in one direction that seems like the best decision in the short term but actually hinders needed flexibility in the long term. This just means the short term improvement wasn't really towards optimal after all.

In the beginning architecture tends to be a form of procrastination for many people. IMO, when you have clear goals for improvement you should aim for somewhat worse than acceptable quality. Not because that’s the most efficient option, but because it for forces you to be solving customer problems.

That itch to improve things is more focused when revising whatever is the worst parts of the entire project rather than various pieces being decent when there are some horrific bits of the code base. Because those horrific bits end up eventually infecting everything else.

This changes a project grows to the point some code is old enough you don’t remember all the details. At that point you need to use best practices simply to make reasonable progress.

If you only use good practices at the end it is too late - you have the anchor of the old, unmaintained and untested code around your neck. Additionally, engineers who know they are doing crap work are generally less motivated.

I'd argue that the difference in time of writing tests is balanced by less time debugging, and the moment you want to re-use the code you know it already works in all the contexts the tests use it in, in fact it was already designed for re-use because the tests and the application use it, so you avoid major refactoring and instead just assemble your Lego blocks in a different shape.

It’s never too late, in the absolute worst case you bin some or all existing code and start over with very clear goals, and an understanding of any needed external API’s etc. Even better you can almost always extract plenty of useful bits here and there. On the other hand if you start every project thinking I need test cases, separation of concerns, continuous integration, scaling etc you’re working without a clear understanding of the end goal or how you’re getting there. Find out some tool your depending on is insufficient and you may need to toss most of what you’ve been working on down the drain with nothing to show for it.

Jumping right into whatever is the most difficult problem also means your model of the best architecture is likely to evolve as your working on the problem.

Granted this should all be on a sliding scale depending on how novel the project is.

I often feel like I should write a "things you can skimp on, and things you can't" for a startup. There are are some things that if you do it right will immediately start paying dividends. For example, I worked at a startup that didn't take the small time to invest in automated deployment. There was one guy that knew how to manually deploy things, with the only copy of the correct prod configs on his machine. And since there were no standards for deployment, the 5 different sub-apps/services were each deployed differently. They could have literally spent less than 1 day building a proper CI/CD system which would have immediately made all their deployments painless and immediate. Instead when I got there it took many months to finally get CI/CD in place because (a) everything was always on fire so it was difficult to carve out the time to do it, and (b) retrofitting all the different systems to use one way of deployment meant I needed to retrofit 5 systems.

More importantly, my time at that startup was some some of the most stressful in my life, and for what, someone else's shitty code? I refuse to be a "code janitor" anymore. Of course all systems build tech debt over time, but it's not that hard to see which systems were built with a modicum of forethought, and which were just slapped together. Imagine if people building a house got a group together, gave everyone some tools and some lumber and said "go!" instead of actually creating a design, laying a foundation, etc. That's what a lot (not all!) of startup code is like, and I for one refuse to touch that from now on.

> However, startups don't fail because the codebase is ugly, or convoluted, or not following best practices.

Startups do fail due to a failure to execute. Best practices aren't always about aesthetics. Ignore them too much and you end up with write-only code. You can compensate in various ways, such as designing the code so that it's possible to throw away the ugly parts later.

You are correct that good senior engineers are apt at striking this balance. I do not agree that the typical senior engineer is competent at this.

Right, which is why my sentence right before what you quoted emphasized that the product must work properly and not be full of bugs. I considered combining them into one run-on sentence so people wouldn’t be tempted to take it out of context.

Obviously, the product must work, must not be buggy, and must be maintainable enough to move forward.

However, there’s nothing wrong with what you call write-only code in certain circumstances, especially at a startup. I’d much rather have some backed together, single-use code that lets us prove demand before investing resources in a proper rewrite. The project goals will likely change after first contact with the customer anyway.

One of the biggest time wasters I’ve seen at startups is engineers who lose track of the goal and instead start writing reusable frameworks, or over-generalized code, or separating architecture into modules so they can open-source part of it on GitHub.

Startups are trying to prove a business model as fast as possible. Once you prove demand, you scale it with the proper code backing it up.

Curious when you've worked with write only code what was the language? I've been on like 30 different projects over the last 10 years and never encountered write only c#. But I have come pretty close to encountering write only javascript.

It seemed like static typing, "find all references" and the limited use of reflection style coding made the difference between bad code being annoying and completely unworkable.

I second this rather strongly.. The harder it gets to "stop" and clean-up shit, the quicker it gets to just lose it.. Especially relevant in environments where the authors of the write-only codebase are not around anymore. Somehow it is hard to blame/hate someone you go out with for lunch or beers.

What really worked great for my teams in the past is to game the transition so that the team as a whole sees it as an internalized mission, usually much stronger than that of the company. Then killing X lines of code from the old mess and beautifying the codebase becomes a shared and extremely satisfying exercise

I've seen write-only Java where the design was fragmented over so many classes that nobody could really make a meaningful design change anymore. Classes with dozens of methods, gross violations of SOLID principles, etc. It seems entirely possible to do the same in C#.

That being said, languages in that space seem generally less prone to write-only code, so I'd agree that language choice has a big impact. Especially when an entirely novel language or framework enters the picture.

Regarding b): Bad code quality tends to lead to bad feature velocity fast, and it gets worse over time. Maintainability and extensibility is a major factor to consider when doing design- and architecture decisions.

Something I’ve been thinking about a lot lately. Bad code has a real world business cost, and velocity is it.

Suddenly you will see you can’t just scale the team up because it takes new team members a long time to grok the codebase and work within it.

Similarly, a disconnect happens between business and engineering where building features that appear similar to already existing features still take longer to build since the existing implementations were never built to be extendable or reusable.

While all this is happening, the team is busy working around the codebase, not with it, to the point where they can’t apply solid refinement to the end product. Adequate time for bug fixing/testing is not allotted since the team is fighting the codebase to just get features out. Your product ends up lacking quality.

Developer quality of life goes down, which believe it or not, impacts velocity. Suddenly it’s not pleasant for the team to casually peruse the codebase. Suddenly those time estimates are all in the mid-high range, they can’t get a breather from the cognitive overload.

Finally, those little tweaks business wants to make become the stuff of nightmare. Stand-up becomes a call for mercy, ‘Making that change is not straightforward’, a constant compromise, ‘Can we just settle for this alternative instead?’.

A shitty codebase has an outsized cost for sure.

B-b-b-ut this is diametrically opposed to other axioms I follow, like "fail fast", and "pivot hard", and "agility"

Are there more colors on the greyscale than 000 and fff?

> However, startups don't fail because the codebase is ugly, or convoluted, or not following best practices.

Could they fail because they fail to meet deadlines due to convoluted or buggy code or having to invest more money and time to fix those issues and bugs introduced by them?

Yes, totally.

They could also fail because code/architecure is overengineered, not invented here syndrome or because of premature optimization.

In the beginning, being able to adopt and/or pivot quickly might be much more important than to have 100% test coverage, keeping up with the latest JS framework or being able to scale to 100M users.

Unit Tests are invaluable in writing more quickly difficult features. You can make sure that your existing behaviour is not broken by fixing a bug or adding more functionality. Every day you spend fixing bugs because you didn’t want to write proper unit tests reduces your chances of getting that next founding round. If you write trivial code the tests are obviously superfluous, but you should ask yourself what is the added value of your product if you only write trivial code.

It’s interesting work for mature companies and seeing something resembling the Big Bang, where you can peer into deep distant code that is of a very different nature to recent code. That distant code being hacked together in startup mode does what the heck it wants and is ugly as f. It might even be in VB!