Perfectionism – one of the biggest productivity killers in the eng industry
newsletter.eng-leadership.com
newsletter.eng-leadership.com
And most people are wrong about which group they're in.
Perfectionism doesn't necessarily mean better implementations, good practices, or even doing the right thing. Chasing and stacking needless abstractions to make things appear clean is an example of such perfectionism that actually harms the industry and erodes standards.
Other engineering professions do have precedent shown for this.
I'm not convinced that either certification itself or the market represent the most effective means to achieving societal good. I think what history shows is that real consequences for shoddy work are required [1]. To date, software makers haven't really been held liable for consequences, but I expect that will change if we don't clean up our act.
[0] https://blog.codinghorror.com/do-certifications-matter/
[1] "229 If a builder builds a house for someone, and does not construct it properly, and the house which he built falls in and kills its owner, then that builder shall be put to death."
- Hammurabi's Code, ~1750 BC
Allowing individual engineers to be de-licensed creates a culture adherence to norms is the priority. If you ever worked with a civil engineering organization… it’s not exactly a hotbed of innovation. Alot of the professional associations spend more time creating make-work regulations than actual safety or engineering. For example, in New York, something as trivial as refurbishing a moderate volume street crossing in an urban area requires $750k-$1M of services to address ADA compliance, multi-modal transit accommodation and other factors.
Crowdstrike had a sloppy process without meaningful testing, only parser validation. Assigning that task to tens of thousands of admins, most of whom have no idea what they are doing just shifts blame, not accountability.
There’s an epidemic of “minimum viable” when it comes to software, and that results in flying really close to “not viable”.
And what do they all say? “We should have launched sooner.”
That’s what happens when the goal of the vast majority of startups is not to build an actual business. Their goal is to “get funded”. It just feels rotten.
Perfectionism is a disease
That results in other shipping sub optimal stuff
It’s perfectly ok to ship 50% of the features 100% well and keep shipping
I got you! It’s relatively new, published four days ago. The relevant section is called “what went wrong and why”:
https://www.crowdstrike.com/falcon-content-update-remediatio...
It’s a hard problem - the whole point of Crowdstrike is to quickly respond to threats. (And they are very good at it) So you want time to market for marginal changes to be very fast. Normally I’d give them the benefit of the doubt, but their arrogance and poor response triggers my spidey-sense.
Assuming it is following, paradigms, design patterns and making the code "cleaner", then I would agree with the article 100%.
However if it was about actual engineering concerns like better error handling, performance, test coverage, nailing down invariants/validation/parsing, etc. Then I would say that we should probably be more diligent rather than less.
The issue is that the former gets too much attention and the latter too little.
I've seen first hand how someone writing a 500 line function struggles to test it. But splitting it up into more manageable chunks of functionality makes it easier to write a test.
Imo perfectionism is chasing that last 5% performance.
Clean code, design patterns etc are just good engineering practices... Not perfectionism ....
The reason I wrote the above comment is because I've seen this being flipped on its head too often by myself and others.
This is what I don’t like about software engineering. The dogmatism. The absolutism. The more I work in the field, the less I talk about “clean code”, “ddd”, “good practices”. I find myself saying more and more “it depends”, and more often than not what the business requires is not 5 layers of code in which the concerns are separated among dozens of files and following good design patterns, but a damn single file that get things done.
There’s place for what you call good practices and clean code, but many times those things don’t lead to good products.
Every single startup I’ve worked for has eventually come to terms with needing some best practices. In my view, there actually is a baseline: CI, partial coverage unit tests for the most easily testable code, and modularization/loose-coupling so that unwinding tech debt has linear rather than exponential cost. But even that is driven by my opinions about ROI, not theory in isolation.
Clean code for a bad product won't turn it into a good product. That's the core issue. These are engineering principles but you are looking at them through business logic.
The best metphor here is to compare clean code to insurance. You don't want to insure this? Okay, you can take that risk and maybe save if nothing happens. But if/when something happens it'll be more expensive than if you paid the insurance up front.
Meanwhile, Clean code from a good product will make it resiliant to becoming a bad product. I'm sure if you played any modern video games that you've had at least a few games that you loved but had horrible technical hiccups. Unoptimized, untested logic, crashing. The issue is some of these games still sell, so maybe the business logic prevailed over properly engineered code. But they are entertainment at the end of the day, not Crowdstrike.
Just don't do them for their own sake.
But the bulk of my experience is with small startups where the stakes are high; tensions around productivity are natural, and working through them is essential.
I've been using it for many, many years and it has saved me a countless hours by postponing abstractions, modularisation, etc. to the point where it actually emerges as a pattern in the software I'm working with. Let the code repeat itself, copy and paste, and only after some pattern repeat itself for the 3rd time I start to think on how to abstract, or modularise it.
It's always so much easier to rework repeated code/patterns into abstractions than fixing misguided abstractions done too early.
Now for a proper engineering answer, there's a few (but not limited to) factors to consider when determining software rigidity:
- Halo effect: How much code does this module affect? The more things that can break when the module fails, the more need to get proper testing/architecture.
- Frequency (in both ways): How often are you changing the code (is it cutting edge, or bog standard 50+ year old algorithms?) as well as how often the application calls the code. The more times its called or changed, the more you may appreciate ways to test new implementations with minimal impact to the current iteration.
- Understanding: pretty straightforward. The less you or the team understands the code, the more need to validate that code through tests. This often falls into frequency, because inevitable the less you understand the more you will need to tinker, possibly for unknown unknowns.
It's still subjective at the end of the day, but there are metrics you can use to make your case.
Given the choice between fixing a quality issue that can be easily measured/detected and one that cannot, a lot of people will prefer the measurable, easy-to-detect one.
So the folks on the 'code quality' committee end up introducing tools that make it impossible to merge if your markdown file has two consecutive blank lines - rather than spending the time on performance or security which are much harder to keep on top of with automated tooling.
sigh
Jordan seemed to be talking about it: prioritizing; helping, rather than burdening others. Gregor seemed to be describing a lack of diligence: not fleshing out learning, fretting over less relevant details.
In my mind arguing for dilegence is a better here. It implies that the product matters, even if it is imperfect. Arguing against perfectionism is bad. To some it will imply that the product does not matter, that sloppy work is okay if you can somehow justify it (getting the job done, doing more, etc.).
On the other hand, I couldn't count the hours of my career that I've wasted perfecting some piece of functionality - either at someone else's behest, or due to my own motivation/professional pride - that ultimately nobody gave a shit about.
I don't really want to do that any more, and I get pretty fed up pretty quickly when some PM (or whoever else) pushes me to do something that I know ultimately isn't going to matter. I can add a lot more value to a team than by simply churning out code, and it's frustrating being in situations where I'm not allowed to do that and am instead forced to waste time endlessly fiddling with stuff that doesn't add a lot of value for users and customers.
And I'm not talking about compromising on UX or anything like that: I mean actually spending time building functionality that doesn't get used. Or spending too long on barely used and non-key workflow aspects[0].
What's the point?
[0] A really great example of a barely used but ABSOLUTELY CRITICAL piece of functionality is the export/publish functionality in Ableton Live (it's a while since I used it so I can't remember exactly how it's labelled). This is a piece of functionality that is barely used compared with other functions, at least by many of us, but it's also - in some sense - the whole point of the application in the first place because it enables you to export or publish a finished piece of music, so it needs to work and work well.
It wouldn’t have existed lol
If you want to build value you must ship, even if the thing isn't ready yet because otherwise it will never ship and there will never be any value realized.
Sarcasm aside, you're not wrong about people being caught up in perfectionism but that doesn't make you right about people/managers conflating perfectionism with goodenoughism. I'm sure with this error (and many others) you can find someone who was trying to solve the problem/s and someone else that accused them of perfectionism. I rarely see actual perfectionism, but I do frequently see people disagreeing what is good enough. I also frequently see problems that would have been small at the time but to fix later on require massive expenditures. The problem is that not-good-enough compounds while good-enough is neutral. You need to also dedicate time to maintenance and repair, because that's what fixes not-good-enough, which frequently happens simply because we aren't omniscient.
If you're going to measure the dollar value cost of their errors as a cost of CrowdStrike being in business, shouldn't you balance that against the dollar value of savings from the attacks they've stopped?
Just comparing against the attacks they stopped is a naive and gross overestimation because it requires an assumption that if a customer did not use CrowdStrike they did not use _any_ alternative. In reality, Crowdstrikes customers would have used some other product. So the attacks that competitors could have also defended against cannot be accurately attributed to Crowdstrike. i.e. if Edison didn't invent the light bulb someone else would have. We know this because he didn't invent it, though he did make the first practical version[0].
But since we're on topic, I'd call what I suggested a "good enough" estimation. We have a reasonable estimate under reasonable constraints. It's more complex, but the amount of accuracy gained is tremendous. Arguably the other model is so inaccurate it is less than meaningless (it isn't even "back of the envelope") because its conclusions would lead us to believe things that are not true, and far from the truth. But if we want to talk about _actual perfectionism_, well this gets much more complicated much more quickly. In fact, gets exponentially complex in exponential time.
A true analysis needs to consider the counterfactual question of what if Crowdstrike did not exist at all. This is far more complicated and contains many subltities that could dramatically change the results. Because we need to understand things like how other companies would have advanced and developer were the money/manpower/resources that were allocated to CrowdStrike were allocated to others. To understand how CrowdStrike both creates and discourages competition in the space. Systems like these are (mathematically) chaotic. Who knows, maybe a manager at Crowdstrike pissed off an employee who then turned blackhat. But maybe Crowdstrike gave a job to someone who would have turned black hat would they not have gotten hired. Maybe because they were laid off from a competitor! Realistically performing these calculations to a solution that is accurate (which would still be probabilistic in nature) is intractable. Of course we can improve our earlier model by using some of these factors, but I want to illustrate that there's a difference in perfectionism which is unachievable from being reasonably accurate ("good enough").
[0] Left unknown is how quickly someone else would have come to similar practicality but my best understanding of this specific story is that it would not have been long. This is not an uncommon story in science and technology and if you look hard enough you'll almost always find that any discovery or invention was also being pursued by another group. There are exceptions, but they are rare. You just never hear about the other groups, but you can confirm this by watching advancements that happen while we're alive and can see the competition in real time. Specifically in domains where you are close to.
In other words, it's not perfectionism that's the productivity killer they've identified, it's inexperience.
I'm a perfectionist in the clinical sense. Shipping something that I know is imperfect gives me a high amount of anxiety. But I also have enough experience to avoid wasting my time on things that I now understand don't matter.
I don't get anxiety about not having the perfect abstraction anymore, I get anxiety about things like whether we have enough monitoring in place, whether our release plan will cause problems in prod, or whether I considered all the edge cases in this critical piece of security-related code.
The perfectionism isn't gone, but it's not really hindering my progress in my career because it's focused on the things that actually matter. I'm working on addressing the anxiety aspects of my perfectionism, but that's a mental health issue, not a software development cycle issue.
The problem is that software engineering is still, in many ways, in a pre-scientific era [1]. For example, civil engineers know how to build bridges that are good enough, i.e. that barely stand.
In the software field, we don't really know how to proceed in the same way. We have formal methods, but they are quite costly, and therefore not used very often.
And "good enough" is a relative and changing thing. What was "good enough" thousands of years ago is nowhere near standards today. This is even true a hundred or 50 years ago with the specific example of bridges. Engineers learn this the hard way because the results are visibly catastrophic and usually through the loss of many lives. But I think one of the difficulties of software is that failures are often less apparent and can be difficult to accurately attribute. But then again, when a bridge fails there are both internal and independent investigations that are extremely thorough and result in updated policies. This does not seem to be the same in software (at least to nearly the same degree), in part due to the abstracted nature of how software can cause harm. It is actually harder to attribute error (like you said, formal methods exist, but are costly and not just in the monetary sense).
I think there is an irony in this because often someone managing a group will accuse them of perfectionism when they themselves do not have the appropriate context to understand that the arguments are really about goodenoughism. I do think this is why companies that are fairly transparent to their employees have been successful. Because it helps them understand the balance and constraints of which they are operating under. To have an environment which is closed and groups are unable to communicate (e.g. in a highly classified setting[0]), the person managing the groups needs to have intimate knowledge of all the pieces coming together, because somewhere along the line someone needs to glue all the puzzle pieces together.
So I wouldn't call what you express perfectionism. Perfectionism doesn't really exist because perfection doesn't exist. Rather it sounds like you are detail minded. It also sounds like you've learned to understand what details can (or have to) be ignored, what details can be temporarily ignored but need to be revisited, and to understand that keeping track of details is still important to understand how one needs to improve and mitigate future catastrophes. Obviously there is a balance, but I think this is very different from someone who is trying to turn a map into the territory.
[0] I should mention in many classified settings people are still encouraged to communicate and bounce ideas off of one another. Just to do so exclusively in secure areas and to ensure the other person also has the requisite clearance.
> the tendency to demand of others or of oneself an extremely high or even flawless level of performance, in excess of what is required by the situation. It is associated with depression, anxiety, eating disorders, and other mental health problems.
At my worst, every single mistake I made would give me hours to days of stress and anxiety. I've gotten it much better under control in general, but it still does cause a lot of stress from time to time.
This is why I object to people equating "perfectionism" with "being bad at prioritizing truly important work". I'm pretty darn good at prioritizing what's important. I actually perceive wasting time on unimportant work as a mistake, the opposite of perfection, and if I do it and realize my mistake I get stressed about it.
Perfectionism is holding yourself (and/or others) to excessively high standards, often met by anxiety or depression when those standards aren't met. The amount of perfectionism someone has is completely orthogonal to the question of how you measure perfection—a perfectionist with a definition of perfection that is well aligned with reality may accomplish a lot while a perfectionist with a definition that's less well aligned ends up wasting a lot of time.
And while you're right that perfect doesn't exist, that doesn't stop a perfectionist from pursuing it.
To be clear, I was trying to make the point that this is ill-defined due to what is a reasonable level of acceptance is difficult to define and how the common usage of the terminology can be ill suited due to this being dependent on understanding what one is conditioning on. I'm not saying you don't place too strong conditions, I don't know you. Despite running into you here often, I don't know that side of you and it isn't something I can reasonably conclude in either way. But I do know it is common that too lax of conditions are placed. It's a spectrum after all and either end is bad.
> every single mistake I made would give me hours to days of stress and anxiety.
I can definitely empathize with this and have been there.
I think you and I are decently aligned in our belief. Maybe we're bad at explaining and/or bad at interpreting. But there are enough similarities that I think we should not discount. But I do think we were using the word perfectionism in different ways. I was trying to use the version we'd see in normal language and how it is used in the article to agree with you in that this actually isn't perfectionism (certainly not the clinical definition). Though you're right to also point out that the clinical definition of perfectionism is not about actual perfection. But I don't think this invalidates my point that the true difficulty is in determining what is appropriate levels of standards, that this is a difficult balance to strike, and that many conflate comments of details/nuance with excessive standards when they may or may not be, including that the recognition of details is important in determining this. And I do think there is a natural bias in that it is easier to identify excessive standards than it is to identify insufficient ones.
experience without growth is just inexperience with more "years of experience".
This is pretty much what happens in more technical parts of industry when you have your best workers jumping every year, poor documentation of proprietary tech, and a lack of proper training because you expect "experienced" developers to simply grasp your entire tech stack from day one. You have people falling into the same pitfalls every year (or few months even) and building on a foundation that is not fully understood. Because there is no time to properly understand.
Maybe there should be some "perfectionism". Maybe not in shipping "things that actually matter" (as business defines it), but in the boring stuff that keeps a ship afloat. They could also simply give raises to their best workers instead of having them walk, but clearly that's not in the cards.
1. Crowdstrike 2. Microsoft 3. The companies that think it’s a good idea to run what is essentially an enterprise rootkit
I would argue that perfectionism is also about an inability to prioritize, something which, in the real word with all its time limits, will bite you.
this was not the failure of a single company.
I think we see the results of rushing in all the software around us much more than we see the delays of perfectionism.
Thankfully the market correction made this much less of an issue.
I'll take a 95% solution delivered in 2 weeks over a 99% one delivered in 2 months.
This depends a lot on the context. If I deliver the 95% in 2 weeks can I ship an update in 2 months that delivers the 99%, or are we stuck forever at 95% (due to technical limitations or business reality)?
If I'll be able to deliver the 99% soon, then sure, let's ship the 95% sooner rather than later.
But if for whatever reason I realistically cannot do that, then you have to consider the context. What is left out in the 95%? How soon will it be a problem that we're missing that remaining 4%? Who will notice the missing pieces? Will they be able to work around their absence?
Many times you may still come to the conclusion that the 95% is the right move, but sometimes it's better to not offer the two-week plan as an option if the fallout from that missing 4% is going to be high.
E.g., the Twitch codebase
They can't change it or add new features or fix problems because it's too much of an unholy startup crack-fueled mess.
Not even companies who stole the leaked codebase can improve the horrible user experience on their Twitch clones whatsoever.
I practice my perfectionism at home, on hobby projects, where I can run lint with every option if I want, and resolve every warning, static analysis issue, style issue, I can profile my code and find/fix slow parts. I can run valgrind on it and find memory leaks. And I never have to release the software! None of these things were common practice in any software job I've ever had. It was always "Get it to barely work and then ship ship ship!"
Maybe it's the pessimism of age, but it feels like that describes all biology, economies, politics, etc...
While the article doesn't offer specific examples, it's probably talking about this sort of stuff. e.g. "perfecting code, often code that hadn’t been touched in years and didn’t need to be touched". This sounds like a classic "oh, this isn't how I would have written it, so let me just refactor everything here".
"Not how I would have written it" does not equal "bad".
This isn't making the code better, less bug-free, or "more perfect" in any way. Actually, it often introduces bugs.
Give the engineers clear requirements and let them interact with the stakeholders. Get rid of the intermediaries and knowledge brokers.
Maybe I've just had good luck with product managers, but I tend to enjoy my work the most when someone else is filtering the stakeholders' long and constantly changing wishlist for me and turning it into something coherent and actionable.
Now there are these Product Managers who have absolutely no clue how software is built.
Interestingly I'm seeing similar trend in many other fields; my car goes to servicing and there's some dude who just knows jargons interfacing with me, for the minutest of my questions he will "get back to me after consulting the line workers", same with my house construction. I had to get through 4 layers of people to finally get to the person who is actually doing the electrical wiring to clearly explain what is it that I wanted.
That's because he's not there to explain what they are doing. He likely doesn't know a spark plug from a fuel injector. He's there to sell you more services.
There are several reasons why this is not a good idea. And those reasons show itself usually quickly once that is tried.
"You know, the whole thing about perfectionism — The perfectionism is very dangerous, because of course if your fidelity to perfectionism is too high, you never do anything. Because doing anything results in … It’s actually kind of tragic because it means you sacrifice how gorgeous and perfect it is in your head for what it really is." - David Foster Wallace
Anyone that has built and shipped something has likely struggled with it. You have a great idea. You build it and it's never as great as it was in your head. You need to ship it anyway, but doing so means getting over that perfectionism. Some can. Most can't.
The "we'll fix it later" turning into "no one ever fixed it" is way too common of a pitfall.
Then the lore came out that the original programmer had slapped it together over a weekend. It worked in the lab, and mostly worked in the field but failed sometimes in weird ways.
Then engineering tried fixing it. They failed three times, so they decided to replace that feature. It took two years to replace that one feature and get it to work right. It didn't kill the company, because it was a less used feature, but it reduced trust in the marketplace so growth took a hit that let other take and keep the lead.
Now that the prevailing school of thought is that shipping crap is OK we have a checks notes Boeing Starliner.
The thing that's killing productivity is the pushing out of half baked garbage.
There's simply no solid foundation to build upon.
OK short game but horrible long game.
On an organizational level, perfectionism is a red herring. Yes, there is such thing as debilitating perfectionism, but 95% of "perfectionists" are really just people who care a lot about doing things properly (providing all the benefits that begets), but who aren't 10x greybeards, so they take longer to do it than a cheap dev would to shit out what looks like the same product at first glance.
(You can say the same thing about "premature optimization", perhaps to a lesser extent.)
Nothing wrong with “perfectionism” but it should be achieved as a team and that requires getting good people around you. Unfortunately, in my experience, that’s where it usually falls apart.
Clueless management continues to think hiring at the bottom of the barrel and “training” them is the right way to go. There’s a reason why they are taking the lowest possible bid…
Half the nation's personal data is breached every week and we're over here talking about how we need more productivity and less perfection.
The desire of money people to sell hack/slash/burn kind of products to customers is so pervasive nowadays, that there is no wonder why we have:
- the Boeing travesty (people actually die because of this) - the CrowdStrike debacle (who needs functioning airports and hospital appointments?) - the AT&T hacks
And some manager type (who probably did a 1 month business course and is also a certified Scrum master) is gonna tell you to "ship faster, value, value, value". The disdain I have for them has no bounds.
Last year's CrowdStrike crapware incident made critical infrastructure crash, flights delayed, hospital appointments delayed.
I'd say to the business people to f*ck off and leave the engineering to us, the engineers. Stop trying to squeeze every drop of "productivity" from everything.
It's done when we say so, and it's good enough when we say so. Go sit in some Zoom meetings or something, and leave us to work.
I would also say being able to identify where you should be consistent can be itself a huge productivity enabler.
In this case, how do we know it is an exponential chart at all or there is not discontinuous jumps in value at “perfect”?
We are not even close to having any noticeable perfectionism in the industry.
Over-engineering is not a form of perfectionism, it is a lack of empathy and lack of professionalism.
- Write down your priorities
- Deliver small units of value
- Seek feedback early
Are solid bits of advice. I felt the rest can be skipped.
Blaming people for not being perfect - no.
Software done right from 30+ years ago is still in broad use. There is huge value in that
Doing things right involves prototypes, iterations, and experiments, so high velocity is good. However, if a prototype ships, you often get on trouble.
While perfectionism is not good, I've rarely seen this be an issue except in maybe young engineers. But part of the problem with perfectionism in them is that they don't even know enough to understand the whole picture and the reality is that "perfect" doesn't actually exist. Solution spaces are too complex so recognize global optima are rare.
Instead, what I see far more of is not understanding what "good enough" is. Not paying attention to details. AND dismissing details with some comment about how you shouldn't be a perfectionist. This is a much more nefarious problem because shittiness builds over time. But the damage it does is incredibly costly. The temporal component and the fact that you yourself are often not impacted by your own actions make this much harder to recognize. Worse, people often get rewarded for short term shortcuts that lead to catastrophic failure in the future. But I hope we all know that a little maintenance is far cheaper than repair. It's why insurance companies want you to do a yearly checkup and get your teeth clean. They know its cheaper.[0]
What really matters is a very difficult balancing act. You need to balance your short term progress with your long term progress but humans are bad at long term planning. You need to move fast enough to meet your short term goals and deadlines but you cannot forget what the end goals are. This is hard because the end goals are fuzzy, change, and only come into resolution as you progress. So you should "move fast and break things" BUT you also need to slow down and fix things. This is part of why the software world is ruled by the lovecraftian god of spaghetti and duct tape[1].
I'll give a quick example: You can write sloppy code to get something working, but spending a bit of time to clean it up and add a few comments is always worth the effort. The problem is you might not feel the pain from the lack of cleanup or comments. What's the worst thing that happens? You spend an extra hr because you realize there's a logic flaw when writing up the doc? Or if no problems, 10 minutes to write words? We've all been stuck trying to understand what something does and why (sometimes we wrote that code[2]), especially when onboarding to a new codebase (think about the connection to the 80/20 rule). Which takes more time? Doing cleanup and commenting as you code or the time lost by every person who gets stuck on that line? If you've ever onboarded into a codebase that is well documented I know you know. And you can probably see how the costs are outsources from you, but not necessarily your company. It's the inverse side of what makes software so powerful: scale.
Perfectionism is rarely a problem and is a junior problem that's about not realizing there's no global optima. But (not)-goodenoughism is common and rampant at all levels. If it wasn't, we wouldn't have daily comments on HN about enshitification[3]. But if we were better at understanding the latter I'm sure the former would also be less common. Move fast, but also make sure you slow down. Never forget details, especially when you need to disregard them to move forward.
[0] I have one example at a company where I worked where I was told I was being a perfectionist and then months later our product literally exploded. An engineer wanted to extrapolate data, I said he was extrapolating too far to be reliable, boss overrode because he was more stressed over the timeline.
[1] Duct tape is a terrible tape and you have no reason to own it. Fight me.
[2] At the time of writing only god and I knew what this code does. Now only god knows.
[3] Enshitification is not happening because people are being malicious. It is because "good" exists in unstable equilibria and part of long interacting chains that all have to go right.