Code Is Never “Perfect”, Code Is Only Ever “Good Enough”
exceptionnotfound.net
exceptionnotfound.net
“Good enough” is really the type of attitude that leads to this unfair balance of effort.
Unfortunately, we live in a world where criticism is levied very heavily on where you are now, and not on how you got there. What, version 2.0 of the product sucks now? Screw it, “1 star”, boycott this; fire the team responsible! Never mind the fact that version 1.0 was a rickety mess, all of those guys left, and anyone would be lucky to get the thing working at all, much less in a way that does not introduce any new bugs.
One of the big benefits of open-source is having a chance to understand what’s going on. Would as many people invest in a product, despite promising behavior in version 1.0, if they could see a critical review of how that version was actually implemented? Ideally, we can see not only how a product appears to work, but how well-engineered it is according to experts.
There's a reason for this. The "real work" you mention comes into play only after the product has reached product market fit and needs scaling. Before that phase, it's all about struggling to make things work, and people don't have time to refactor everytime they need to make a small tweak and experiment. So naturally by the time the "real work" guys come in they will think "WTF the code is so shitty, whoever wrote this is an idiot". But they should know that they're there only because of that shitty code. If they were in that same situation (and rational) they would have done the same thing--write a consequentially shitty code.
Startups are the opposite of having clear requirements. There is a lot of uncertainty. In my opinion, it is usually orders of magnitude harder to figure out what to dot hat gets suers than building something "perfect" that nobody cares about
It doesn't take that much longer to write great code when it's just part of the first-principles of your engineering process. You just do it as you go, but best-practices need to be scaffolded and modelled for everyone beforehand to give them a solid starting point for what "minimally viable" actually looks like. A single well-placed sentence that describes something necessary, but non-idiomatic or non-obvious, that takes 10sec to write can save you hours or days of tracing and debugging at that crucial moment right before you're onboarding a pilot customer or pitching investors.
Very often the bar of "minimally" is set way, way too low and the trope of that just being "startup life" is usually just laying problems at the feet of engineering that don't belong there by justifying, sometimes horrifying, technical debt which are really process and management failures, and later eventually scapegoating engineering for it.
Unlike what you say, it does take much longer to write great code when you're trying new things every day. Every time you try something new, by definition you're building something that you hadn't planned when you first started. Which means if you really wanted to be perfect you would end up refactoring every day. This is not a small effort in total.
You could say you should have prepared for that scenario from the beginning, but I don't think it's a good idea to believe that you can predict the future especially when you're building something that hasn't existed before.
Literally every situation I've ever seen in 20 years across all manner of marketplaces and every imaginable stage of development has had the immediate faux-gains made from ignoring spending 10%-20% more time now (to produce something at least coherent, commented, and composable) ultimately overwhelmed by the soon-to-be-immediate costs of having to live with the consequences of the dubious "shortcuts" taken.
I agree with you that it's extremely hard to achieve maintaining that kind of discipline when under the gun, but that's a people management and expectations problem, not an engineering effort problem. I'm 100% certain that in any given window of time, no matter how crunchy that crunchtime is, that if my team spends 10-15 out of every 100 man-hours producing a higher-quality "minimally acceptable" artifact that they'll both materially accomplish more in that first 100 hours than if they'd spent all of it hacking together a barely working dumpster-fire in the same window of time, and that it will pay glaringly obvious dividends for their productivity in the next 100 hours after, and the next after that, and so on.
Lacking that discipline is neither a necessary condition nor a near or long-term benefit to a prototyping or product development cycle. It's purely a function of leadership, expectations management, and experience.
(I'm even giving the benefit of the doubt that what's being done "hasn't existed before", which is overwhelmingly not the case. More typically it's different people and different teams just painfully learning the same hard lessons in parallel, often because there's an assumption/delusion what's being done is a legitimate pioneering effort instead of the probable reality that it's really a minor variation on something else that already has well-established successful models available to learn from and mimic.)
The problems is at the very beginning, all (or most) of that code will be thrown away because you really don't know what works. How many cycles of wasted extra 20% effort would it take for you to change your mind?
This is because the friction and pain of technical debt doesn't wait until the end of the 100 hours to create problems. It's biting you every second of every minute of every hour and dragging down your productivity and velocity as you're just plowing forward soaking the noodles.
On the other hand, I have also made mistakes of over-engineering to make the code maintainable. What happens then is I can't really detach myself from the original idea no matter how much evidence there is to show me that it's not working, since I've invested so much in the technology. In that case the time I spent to make the code maintainable becomes a mental liability rather than an asset.
I don't doubt you are a great developer and have great experience, and I don't doubt that spending 15-20 hours out of 100 hours when you've reached product market fit is something you should do, but I can say from my own experience and experiences from other successful/failed entrepreneurs I've met that it's absolutely a bad idea to do so when you're just 1-3 guys bootstrapping. In the beginning the only thing that matters is getting people to actually use your product, so you should spend 100% of your resources on getting there, not on optimizing maintainability at all.
If you have actually been in that situation (a founder who's built his own product from scratch by bootstrapping) and still think it's best to spend 15-20 hours out of 100 hours working on maintainability, please let me know. I personally don't know any founder who thinks that way.
It can also be 15-20 hours making sure you're following language/tool idioms and conventions so that you can iterate quickly instead of fighting against yourself and your tools being used incorrectly.
It can be about making sure you aren't making leaky abstractions so that when you suddenly realize you need to make big or small changes to address market needs you're able to do it quickly and efficiently rather than having to rewrite a big brittle pile of al dente noodles with every shift and change you're presented with.
It can be about making sure that the little idiosyncratic things you had to do have some explanation around them so when your fellow boot-strapper has to come in and make a change or understand something you worked on, but you're home sick as a dog because your kid sneezed straight down your throat, your fellow boot-strapper can understand what you did, why you did it, and can carry forward with that change without depending on your availability holding everything up, and because you didn't make a bunch of leaky abstractions on top of it all they can make that change without breaking 99% of the rest of the prototype/product.
It can be about making sure a sufficient amount of what you're doing is done so with composability in mind, so that when you figure out that thing you did that you ended up throwing away, but now turns out would be supremely useful to meet current demands, both still exists someplace and can actually be used without having to spend 10x as long rewriting it.
Having great scaffolding, best-practices models, and good process makes everything go faster... not slower. Learning how to properly use your VCS is an enabler, not a drain, but it takes time away from just hacking away on something. Setting up a CI pipeline is an enabler, not a drain, but it takes time away from just hacking away on something. Building scaffolding tools that can just generate for you the first 20% of a project template are an enabler, not a drain, but it takes time away from just hacking away on something. Learning your tracing and debugging tools is a massive enabler when trying to understand and prototype something, but it takes time away from just hacking away on something.
I should add the caveat that all those things are enablers... only if you do them well... otherwise they're a giant drag because they're hurting you, not helping you, and they become a part of your technical debt.
Do you spend literally no time on any of that because it takes time away from hacking? Do you think having none of those enablers in-place is helping you all work faster?
First principles works fine on CRUD apps, less so new concepts. If you going to toss 90% of your code as useless then an extra 20% is simply wasted time. I am going to try the same thing in 5 library's to see which one I like directly means tossing 80% of that effort away.
Keeping documented feature branches of all those versions you tried before landing on one is an absolutely critical result of any exploration process, as well as some aggregation of information and context in a central place that points to and describes those various versions and your final outcome. This helps you keep from repeating mistakes, it provides context to you and others in the future for why things are the way they are, and it provides an archive of workable variations that you can quickly migrate to in the event your "final answer" turns out to be wrong.
I haven't written a "CRUD app" since 2008, and since 2010 nearly all my work and the work of my teams has been in greenfield development (in some cases literally inventing new never before seen things). I see it time and time again. We move much, much slower as individuals and as a group when we take shortcuts under the auspices of time-pressure and don't take the time to be thorough even in prototyping.
So we spend the same 100 hours hacking away to create a dumpster filled with spaghetti as fast as possible or we spend 100 hours designing, building, implementing with intention, and every. single. time.... the latter process actually gets us further toward prototype, product, and/or market validation goals. The former process ensures that we have 0% of anything that is useful no matter what the results of the exercise are. The latter process ensures that we can legitimately iterate on what we've done based on the results of the exercise.
Iterating on a trash heap is extremely expensive, not a way to run faster.
So, clearly we are talking about something different.
I'm talking specifically about using a different language, a different OS, a different deployment pipeline, new testing methods, new security constructs, new infrastructure orchestration methods, entirely new architectural paradigms, etc. etc. etc.
In fact using that as an example. Say it's a new programming language. You will absolutely waste more time dealing with the pain of doing things epically wrong, fighting with your tools, doing really crappy kinds of log-print debugging, etc. than you would taking 10-20% of your time up front learning some normal idioms and conventions for the language and learning to use richer tracing and debug tools to find problems and to instrument your prototypes.
Just because you sat down and started coding and debugging without having any idea what you were doing, but at least you were "doing", emphatically does not mean you are getting to a goal of minimally viable test, or prototype, or product faster. It just means you started typing sooner.
"Wait, I need a Mac to write iOS code, ouch."
Ideally you should be producing far more research than code. Your talking logging, and I am talking the "H" part of "Hello, world."
PS: If you can get away without writing code so much the better.
But, as you've pointed out, the real work is in maintenance, durability, and quality, not veneers.
I’m generally surprised at how little interest VCs have in this, though. I think a sizable percentage of their misses could’ve been avoided altogether if they had just audited their prototype, or whatever they’re claiming qualifies as their MVP.
People generally don't understand how many different scenarios software needs to account for, and how quickly those inputs can exponentiate into a very big and complex task. Creating the appearance of functional software takes a very small fraction of the time and expertise that it takes to actually make something production-ready, but I don’t think a lot of non-developers understand just how big that gap is.
Those VC's can pump and dump: promote the stock stock, then dump it before it dives.
The people that have to care are the company officers who want to grow a successful business. If those people are cynical, then the venture is probably doomed.
No user cares about any of this. Fix it. Don't try to rationalize shipping garbage.
I'm a lot more impressed with the evolution of the human species than how doctors have been able to maintain those running systems.
You want to design a human from the ground up using proper infrastructure that doesn't have ailments or diseases go ahead. you'll be lucky if you end up with a fruit fly.
I'm not even sure why I responded to your comment, since it was so random and unrelated to the rest of the thread.
Doctors are necessary and important and deserve to be paid well, since they deal with life. But that can be done by anyone really. I'm not saying any idiot can become a doctor, it's that most doctors do what any other doctor can do.
I'm not interested in those things either. I would rather work on something that can change how people live their lives--which one could arguably say is close to evolution.
>The human body is the result of 1,000,000,000,000,000,000,000+ organisms dying before or after reproducing.
I guess I have to spell it out. "Bad, unmaintainable code" is the result of 1,000,000,000,000,000,000,000+ random requests being shoehorned in without proper design of the whole thing, but just on the basis of whether it still works or meets some last-minute requirement. I hope this makes my analogy clearer. (I realize the number is an exaggeration, I am referring to the process involved in "evolving" code, rather than "properly designing" it.)
Those are good examples of things that seem easy to write perfectly until you change some requirements that seemed like they would never change.
Take your list manipulating library and throw in a requirement that all node allocations have to be pooled, or that it needs to work in a multi-threaded environment. All of a sudden you have to rewrite them all. (for example, see: STL vs EASTL)
Someone already mentioned floating point, but math can definitely get tricky if you have to be exact. An example from my own past experience - it's really easy to write code to cut a 2d triangle using an arbitrary line as the cutting axis. It almost always works, and it usually works flawlessly. Until you get close to degenerate cases, skinny triangles or overlapping vertices. Throw in a requirement that degenerate cases must resolve correctly, and suddenly you have to start over and use a different representation of numbers. Make it a little more general to handle 4+ sided polygons, and before you know it, you spend a decade trying to get it right. (for example: http://www.complex-a5.ru/polyboolean/comp.html)
That list manipulating library might still be based on concepts which are understandable and communicable to their users, allowing us to reason at different levels on their internal and external behavior. This in turn allows us to say, at a certain abstraction level: this code is complete. Thinking of Occam we might continue: given our preconditions, our abstractions, our algebra: this is the simplest.
It is fair to believe that 'perfect' means that all unnecessary complexity is removed. But, I would like to argue againts its usage, as the word carries too much emotional baggage in this context.
In that sense, there is no perfect code, nor will there ever be. There will only EVER be good enough.
The purpose of words is to share information. If you have write a paragraph describing your definition when you use a word, use a different one.
If software functions correctly but is unnecessarily hard to use due to (eg.) bad layout, would you still call it perfect?
>literally trash
Early on, every app is laden with technical debt. They barely work. When "barely works" becomes "doesn't work", we fix them just enough so they barely work again, and start piling on more features and functionality. We may build some scaffolding and do some refactoring to prevent dropping into doesn't-work-land, but that's not the same as robust! So software - all software - hovers around that barely-works/doesn't-work borderline.
If you care about the outcome of a craft, you'll get craftsmen, and they'll produce things and you'll pick the best ones.
Imagine if a chef had to serve the first try at a new dish, if a photographer had to use his first shot, if a director had to use the first take, if a visual designer had to use the first concept. These processes are easier for people to understand because a layman can tell if two flavors didn't combine well, if a model blinked, or if something looks off. People (even technical ones) can't experience technical problems immediately or directly in the same way. Other fields solve for this with engineering, but ours rarely is allowed.
We often ship the first architecture that gets brainstormed, and the first implementation that works. It's never "We implemented the MVP using patterns/frameworks A, B, C, D, here's a table of the strengths and weaknesses as we see them, statistical summary of the time to complete each feature, and of defect rates. This next slide is a plot of the relationship between request latency, capacity, and estimated infrastructure cost of each implementation". It's rare to even have refactors scheduled. Just cleaning up your shit is "wasting time", never mind how long it takes to ship new features given how cluttered things are.
Sure, engineering is costly, so is not engineering - you just aren't projecting or measuring the latter costs. It's all trade-offs, if you aren't considering expected values of different options, with numbers, then you haven't optimized, you just rolled some dice. Sure, of course it's always worked, everything always have worked for the survivors. There's a big difference between what's unforeseeable and what has merely been left unforeseen.
Time to market and allocation of resources are serious issues for management. They may not understand the tech, but I've found engineers rarely understand the business context, either. "Good enough" is good enough. And we've seen process get more and more focused on this since the late '90s... Agile, Lean, etc. Get a product out the door, even if it's buggy and flawed. Get feedback from actual customers. Find out what's really a problem and what we only think will be a problem.
Engineering isn't the latter end of that, it's precisely just the process of figuring out the trade-offs, the diligence of figuring out where on that spectrum (highly-dimensional continuum really) of process your project should be to get the greatest chances of success. Of course it includes time to market and concepts of "good enough". But it's math and planning and risk accounting and not the cargo culting, handwaving, business platitudes, salesmanship and politics of authority that get utilized in the majority of cases I've seen for making those decisions. These result in monuments to compromise -- the output of arguments between "captain cowboy" and "doctor diligent" (neither of which correctly account for real-world incidence of risk or what that costs the organization which employs them), arguments of engineering without any engineering being done.
Engineering isn't disconnected navel-gazing perfectionism, it's how you prevent it. Engineering is both why we don't live in Kowloon Walled cities, and why most places have water and plumbing and transportation despite a wide variety of obstacles and economic realities that would preclude "ideal" solutions.
The fact is: software is a lemon market and teams/companies are rewarded accordingly.
This is the biggest issue, software is seen as something that is built, not something that has to be maintained, improved and extended.
It's amazing how companies can pour tens of millions into the build step and lose that investment by not maintaining and improving it.
Another one of those real-world insights I've had is a variant of Conway's Law... the software built by a team reflects that team's communication structures, but if you dissolve the team and hand maintenance to someone else, they'll never make sense of the original code, because it doesn't match the communication structure of the new team.
Then when they need to update it (say for a new version of IE) they can't find anyone competent to work on it because no one competent wants to work on such an outdated POS. A simple bug fix or added may involve upgrading a tonne of dependencies and code changes in the project is effectively dead.
By that point if it's worth sorting out it's already paid for the people to clean it up.
For example Coding Standards from NASA's Jet Propulsion Laboratory (JPL) for C[1] and Java[2]
This will also give you an idea of just how tedious it is to do so
I am not even sure if enforcing those strict standards leaves a Turing-complete language
[1] http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf
[2] http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_Java.pdf
So what's left probably isn't Turing complete.
> important general differences from the JPL institutional C coding standard for flight software references (JPL-C-STD) are: (1) the Java standard allows dynamic memory allocation (object creation) after initialization, (2) the Java standard allows recursion, and (3) does not require loop bounds to be statically verifiable. Apart from these differences most other differences are due to the different nature of the two languages.
That said, the standard in question is explicitly only intended for ground-based systems.
"Piece of [bleep]! I know you [bleeping] work! WHY WON'T YOU [BLEEPING] WORK, MOTHER[BLEEPER]?!"
With that kind of attitude (trial-and-error?) you will never be able to consistently produce correct code, let alone perfect whatever that is. Though I think it's not typically the attitude that is the problem, but rather the underlying lack of skill/intelligence which makes people resort to such approaches.
Sometimes software development process can be very frustrating and I fully understand the feeling when you're overwhelmed and confused and feel frustrated because you don't know how to approach the problem. Working with black boxes usually causes the feeling in me, or having an elusive bug that could come from literally anywhere from a huge system and having to debug it with seemingly random reproduction steps, that always gets me.
It's no surprise that there's no such thing as perfect code because code is mere reflection of a real-life requirement, which is always somewhat inconsistent and ever changing. Sticking to one static point is, well, pointless.
Improving code beyond the point where it gives material benefit is not really a problem except for the fact that it deprives you of the opportunity to work on crappier code. Especially when working with new code, I find it useful to concentrate on process rather than end product. If you write tests like X and refactor like Y, how does it affect the result?
In some ways this is really freeing because you get over the fear of writing crappy code. If it's crappy, you just get to improve it. It also allows you to explore ideas and ways of doing things without worrying so much about the result. Concentrating on producing good code can often (in my experience) lead to achieving a local maxima but not does not allow you to kick out of it to achieve something better.
I'm with you on losing the fear of writing crappy code being freeing. At some point I realized that everyone's code including mine is crappy code, and there's no way around writing crappy code or introducing bugs. Realizing that did make me a better coder and a better manager. It let me be a bit more humble as well as empathetic to others, and led me to think more about the importance of testing.
Edit: Maybe one day people will look back and say, "you shipped code without formal correctness proofs? Were you all high?"
And there's other code that pushes against physical or economic limits: processor speed, memory available, network latency. In those systems, writing in an inefficient, high level language is just as wrong as writing the first type of code in a low level one.
Fast, complex, difficult to fully verify code is not bad coding practice any more than a rocket is a worse vehicle than a Volvo because it explodes more. Volvos can't go to space.
(Of course, the situation gets fuzzier when you must account for other factors: work/dollars/lives lost due to bugs in dependencies, the integral of maintenance costs over time, etc.)
"...The entire music distribution world had changed, and we were still working on the same goddamn rewrite. I said fuckit, and we abandoned the Rails rewrite."
It made a bunch of people mad, because (I think) they were reading "Rails couldn't X, and Rails couldn't Y, and Rails couldn't Z, but PHP could totally Y Z and Y, so I used PHP and it was a big success".
But having been in this situation, I think what he was saying was "I couldn't X, and I couldn't Y, and I couldn't Z, so I said fuckit and did it the way I know how..."
It has nothing to do with rails or PHP, or node, django, react. Programmers who have never finally said "fuckit" and abandoned "the true and right way" are programmers who haven't released.
Not saying you should stick with ancient technology, but "good enough" is an inevitable ingredient in any software that ends up being useful to someone.
The important part is the flipside, not the presentation: Given a problem domain and a context there is an optimal or "best" solution or there is not, and we can get at the answer for most cases.
So code can be perfect if we understand perfect to mean optimal. If we understand it to mean spaces/tabs probably not, though given an input and desired output we can even prove some of these as well (for instance, underscored function names are more quickly processed than CamelCase : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.158....).
>Results indicate that camel casing leads to higher accuracy among all subjects regardless of training, and those trained in camel casing are able to recognize identifiers in the camel case style faster than identifiers in the underscore style.
Some code is perfect, but it's only perfect because it was define with extreme mathematical precision (Ex: Haskell Prelude code). The value of this type of code is in direct proportioned to how often it's reused. Often the specs are based on timeless and well defined ideas, and the implementation crystallizes.
On the other end of the spectrum is loosely defined ad hoc code. The specification behaves more like a living thing and the code often tries to adapt to the moving specification over time. This type of code tends to bring a lot of value to the business and often relies on the foundations laid by perfectionist code.
Yes, the Haskell's Prelude is perfect when matched with an (artificial) specification that requires exactly it. Yet, on practice I don't think I ever saw any Haskeller that does not want to add or remove something from it. Partial functions are almost unanimously disliked, ditto for String functions; not to say about the universal complaints in how it is badly generalized.
Sure, people may disagree on what is good enough. But hopefully that disagreement will be easier to resolve than one about what the perfect solution might be.
On a related note, I feel another analogous statement that I now see used dangerously is "Premature optimization is the root of all evil". Yes optimizing prematurely can be paralyzing but the other extreme where one refuses to do back of the envelope calculations and wilfully writes code that will be slow is damaging as well.
Another important consideration is the level at which the quality exceeds the needs of current and future users of the software. Past this point (which might be impossible to determine or fully agree upon), one must acknowledge they continue to iterate and polish for purely personal reasons. This can be fine in certain situations, but it may be a detriment in professional endeavors.
And of course, like always, one must consider opportunity costs.
Edit: For example, the HN code was obviously not "perfect," since we just got several new features. :)
Of course you can argue that "good enough" for them is "damn near perfect".
http://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-s...
From the [summary][3]:
> Our culture’s default assumption is that everybody should always be striving for perfection – settling for anything less is seen as a regrettable compromise. This is wrong in most software development situations: focus instead on keeping the software simple, just “good enough”, launch it early, and iteratively improve, enhance, and re-factor it. This is how software success is achieved!
[1]: http://simongriffee.com/notebook/europython-2013-notes#goode... "My notes. Biggest takeaway was to use 'perfect' as a verb rather than adjective."
[2]: http://www.aleax.it/ "Alex's homepage."
[3]: https://ep2013.europython.eu/conference/talks/good-enough-is... "The lecture video is also on the page."
Non-trivial codes can either be consistent or complete, but not both (within a system of abstraction).
This explains Joel's "Leaky Abstractions" essay as well as the conjecture that code cannot be perfect.
However, "good enough" thinking is often just sloppy thinking. Even if mathematics has boundaries, it doesn't mean you just shrug and say "a nod is as good as a wink".
Craftsmanship and quality are perceived in terms of utility over time. "Good enough" often doesn't meet that bar.
Engineers overbuild to support capacity and plan for the worst. "Good enough" tends to under-build at or below capacity and hope for the best.
"Good enough" is a just reaction to over-engineering (the adding of unnecessary fixtures and 'perfection'), so the warning has value and there are some engineers who use it in this way.
But others often mistake over-building for over-engineering and cripple their organizations and products by seeking "good enough".
The only software that is ever "done" is code that has been abandoned and is no longer in use.
I find that most of the development environments that I have worked in, the focus usually ends up being on getting new features completed in order to help facilitate the sales department to make sales. It's a never-ending cycle that will gradually accumulate technical debt such that pushing new features out the door will eventually come to a halt.
Yes, I agree that code can never truly be perfect. However, as developers we should always strive to make our code better.
"Good enough" is not good enough as a target. It's uninspiring and unsatisfying.
I've seen (and written) plenty of code that was less than good enough.