The Cult of Best Practice
domk.website
domk.website
If you look at an industry way of doing something and decide it’s wrong, you should have solid reasoning to back it up.
I’ve seen so many companies decide that they have some unique requirement to do X that every other company out there doing X doesn’t have. It leads to reinventing the wheel, but generally in a worse way.
All that said, the most dangerous words in engineering are “because we’ve always done it that way”.
And I'd further counter that: yeah, actually, you and your company ARE special. Every company is. There is no One True Way of running a business, or any individual aspect of that business. Everyone's triangulating around resources, people, industry, corporate vision, etc. If anything I'd say the argument that "you are not special" is the fallacy, as it leads directly to the kind of cargo-culting that you see so many failed companies do.
Finally, when we see successful companies, how often is it because they just mimicked what everyone else was doing? Not very frequently, as it turns out. Well-behaved companies seldom make history.
I don' think this is quite right - it's probably important to do something differently, but likely equally important not to do too much differently.
They never do.
I recall talking to someone who was very very happy to brag that his company was doing "everything just like Google" and following the exact same practices he read the company did on various engineering blogs.
I asked him what was the compensation like and how he managed to get kids from Stanford to work at his company instead of their startups and he just gave me a blank stare. Said something about how they were not in Silicon Valley and paid median salaries and that he had "no trouble finding Google caliber talent in the local market." and that "it's not really the local market's customs to give stocks to coders".
I think someone ended up buying the whole thing. Not the company but the office furniture they left after going broke.
Absolutely make your USP special.
But logging? DB access? Deployment? Those are generally solved problems and unless you have a very good reason not to... "doing what everyone else does" is going to be a lot less pain in the long run.
Best practices aren't the same depending on the caliber of the engineers you have.
I agree! Now can we also agree that often what is touted as "best practice" is not industry practice, but rather built on "authority" as the article rightly claims?
The number one thing I hate about "best" practices is it heavily implies there is one and only one "correct" way to do things, and every other way is wrong.
Here's one: if you copy them in every way, you'll just be a smaller and later version of them and get no users or sales. A-priori, you're going to need to find an edge somewhere.
Is SOA appropriate to every org? What does it solve or permit? Is it appropriate at the scale of a system with 10 developer? Or does it become effective at some scale between 10 and Amazon?
Much of industry treats git as the de facto version control system. Why not another system? What makes git better overall or is it?
While I'm a massive proponent of unit testing, there are systems where they're actually very impractical. Should we contort our software and environment to support a popular, best practice testing method when neither support it in their existing or natural states?
Should everyone really be contorting HTTP to be the base protocol for every application? Does it make sense as a messaging system protocol? As your email protocol? Would more dedicated protocols be better?
The weakest arguments against best practices, to me, is a pedantic argument. There isn't a "best" practice there is only a state of the art. Structured programming in its infancy, for example, was not widely adopted and not considered in the state of the art until programming languages adopted its ideas and went into widespread use. During that time it was still the state of the art to write code with goto. Perhaps I'm alone in this but I don't think most people confuse, "It is best practice to avoid character requirements in validating passwords," with "this is the objectively best way to validate passwords." They mean, "this is the best way we know how," and are referring to guidelines like the NIST SP 800-63 publication on digitial identity management.
[0] https://en.wikipedia.org/wiki/Software_Engineering_Body_of_K...
Goto’s can actually be really useful, their simply not worth it today. It’s a subtle yet important distinction.
Unfortunately you have been quite lucky. The phenomenon of blindly following whatever some blog post said is all too real.
Second this. I have no issues with trying to improve things, and that especially includes meta things like processes. What I take issue with is that the very word, "best", heavily implies there is one and only one "correct" way to do things.
This is what people have to realize, when you put in place "Best Practices" you're doing something that is "average".
To do something "better" than that, you need to specialize your practice for your given context. Unless you happen to have the exact same context from which the "best practice" originates from, chances are it is not optimized to your context. And even if you have the same context, time has probably gone by and more "practices" might have developed since that are even better to that context.
Like the article says, you can only optimize to your context if you understand the pros/cons of each practice, so you can customize to your needs.
I have definitely seen situations where developers push for "best practices" when the current practices are already specialized to the context, but because they are not the "best practices" from some internet blog, or famous book, the developers think the practices must not be the "best", in reality they are pushing for worse practices as they are pushing to de-optimize the practices from one context to some other context, bringing the team back to the "average". They lack the understanding and rationale to asses what is "better" or "worse" for their context, and that's why this happens, as they cannot reason about comparing their current practices against the "best practice" ones, thus simply believing in the social trends and norms over whatever the team had worked to specialize into.
Now many times, a team will have below average practices in place, and that's when putting in place some "best practices" will help, as it drives the team processes up towards the "average" or better (given how similar or not the team's context is to the best practice).
Just wanted to say thank you for writing this.
I think this is really important. A lot of today's "innovators" and "disruptors" are just too lazy to study and understand the current state and create a mess that way.
I think software development would be in a much better state if we didn't constantly forget and relearn old lessons.
"The Cult of Best practice" is just a symptom of mental laziness. You should understand why you're doing something while you do it. This doesn't come with experience (i.e. years in the industry), it comes with practice.
I've always wondered why most developers (myself included) prefer to follow popular opinions instead of thinking by themselves.
This post just frames what I've experienced in two decades of working in IT.
If you don't have enough expertise, you imitate those who do. As you gain knowledge, you can evaluate others' ideas, and then roll your own.
It's kinda like "enterprise". Originally the word implied a desirable goal and risks taken to achieve it, but "enterprise" software is aimed at businesses which are notoriously risk-averse and don't have much in the way of goals to achieve beyond their own continued existence and profitability.
Business speak has eroded the meaning straight off English words.
The author seems to be quite confused about functional programming.
That other languages allow something as innocuous as `getName()` to also call `launchMissiles()` under the hood is wrong. Pure FP langs slapping you on the wrist for trying that is part of their value proposition.
It's akin to saying, "why does C worry so much about memory allocation? It should know when I'm done with resources!"
You can have a strict language with an IO monad, and some people actually prefer that.
* One man's Lindy effect is another man's survivorship bias.
* Along the same lines, the only ground truth is the Pareto distribution; choose your 20% wisely.
* There is virtue in starting with "best practices" if only to start somewhere. It's certainly the path of least resistance to the "green light" of doing whatever you want.
> The next step is to make our own mistakes and learn from them. ...The trial and error learning involved gives us knowledge much deeper than what we would gain by following the rules.
As a learner, I find best practices to be a very useful starting point, which the article mentions.
As a mentor, on the other hand, I've struggled with how to provide "best practices" that are short and sweet, but still convey some of the hidden trade-offs. I write docs for a few public libraries and some users just want to get up and running quickly. They don't have time or want to go through trial/error and explore the nuances.
Maybe there should be a greater burden for the givers of advice to acknowledge the trade-offs in their "best practices. Then again, answering a question with "it depends" can be a big turn off to some people, to the point where they stop listening/reading and look elsewhere for help.
You can (and should!) make getting started simple and easy, but there should be next steps that gradually explore the nuances and the background.
You don't see any "best practices" for making algorithms faster or more memory efficient. We have complexity theory that tells us the metrics definitively. This just means for algorithm complexity formal rigor has been established.
For areas where we use the word "design" or "best practices" or "code organization" it's mostly people just randomly guessing things because there's no quantitative metric to tell people whether or not a guess is actually better or worse. What's the Big O of FP vs. OOP? Who knows?
Nobody is clear about what is being measured here. What makes a "bad design" bad and what makes a "good design" good? Once we figure it out you will see that the word "design" will largely exit the field and be replaced with something along the lines of "calculation." It is better to "calculate" the best way to organize code rather then to "design" it because the word "design" is a synonym for "random best guess".
The lack of formal rigor or some statistical metric means we can go into circular arguments about these topics forever.
This may be a bit philosophical, but couldn't "formal methods" and the "scientific method" also be considered a best practice? (and thus susceptible to becoming a cult?)
Throughout the history of science, there has been many methods for doing science that have since been revolutionized or abandoned (e.g. cosmology, non-Euclidean geometry, eugenics, quantum mechanics). Thomas Kuhn is a notable writer on such paradigm shifts in the history of science. [1]
My bias does not allow me to think of a better way of verifying reality by assuming observation, probability and logic is recursively true.
Is there anyone here that doesn't share my bias and if you don't, why?
A lot of our material progress comes simply from selecting different benchmarks and methods of observation.
When we write an API function we want other people to use it. When we write internal functions we want other developers to use it. When we write programs we want some customer to use it.
A lot of usability design has been somewhat formalized. We describe design in terms of signifiers, affordances, constraints, mappings and feedback.
A lot the bad and best practices can be rephrased in these terms.
If your program is "stringly" typed it's unclear which affordances your functions offer, that's bad.
In OOP a lot is hidden from the user, if this is well done, you have constraints that keep you from doing something wrong, if it's badly done you have big objects with some state that's unobservable (lack of signifiers) so you can't understand what it does (lack of affordances).
In FP you do this through function signatures, purity and strong typing.
The type signature together with the function name can show you the affordances of the function. By constraining inputs and making outputs explicit (e.g. result types) you provide constraints and signifiers about their usage.
A mapping is what we expect, e.g. from a knob. If you find 4 knobs and 4 hot plates on a stove, you instinctively know what they do. If the hot plates are somewhat shifted laterally against each other it becomes even easier to identify which knob is for each plate. In programming this could be the name of a function. A function called printX() shouldn't do anything except printing, otherwise it breaks the instinctive mapping that you expect. A function named isX() shouldn't return void. It should return a boolean.
Of course you can't condense this down to a simple number, but at least you can describe why something is good or bad. The groundwork has been laid already.
It's the price we pay for the amount we can do with software. It's very powerful but also orders of magnitude more complex than anything we can reason about formally.
> At least with our current tools
We should be aiming more towards making projects with a scope that we can actually manage, instead of trying to build huge castles of sand and failing spectacularly. And of course continue to improve our tools to properly manage more complexity.
Sometimes you just hit a mother lode, and keep digging, preferably doing proper geological research along the way. But to postpone practical applications before a complete theory is ready is to lack business sense.
The difficulty in software is its detachment from physics (when compared to other disciplines categorized as engineering). There are few physical couplings you can make with your transmission system or steering wheel. But with software, your couplings are essentially unbounded. This permits radically unsafe and unmaintainable software, where an equivalent physical system likely wouldn't even move. The nearest, in physical engineering, is electrical systems. But even then you still have to drive power through a constrained system and either can't power everything or set something on fire.
This forces a practical limit on how you design physical systems. A kind of simplicity often falls out as a consequence. But in software, complexity still reigns supreme because it can. Without a deliberate effort, it is easy to develop a complex system that is hard to maintain, hard to verify, hard to validate, and hard to extend.
Its rarely the developer who decides the size, scope, and complexity of the project.
Reliability/extensibility/dev speed/complexity all trade off against each other, but if you're working for someone else, you have to present those tradeoffs to the right stakeholders, not just always choose simpler.
There's also the "knowing the future" problem - the right design for today might run into a wall when the assumptions or business goals change tomorrow.
What is the best sort? It depends on the size data you expect, locality, initial ordering. What about CAP? Pick two. Best practice is probably choosing a quicksort for now looking what happens. Maybe for CAP give up on immediate consistency.
Best practice gives you a direction that will most likely get you to your destination. If you don't care about the destination you can take forever on the path.
We've formalized how to define the "fastest" algorithm and for that we have definitive answers on what is faster and what isn't.
Our notion of what is "best" is the thing here that has not been formalized. "Best" is a vague word it needs to be defined before it can even be formalized.
Either way for the fastest algorithm out of the set or sorting algorithms that are known.... Most likely it’s a family of algorithms that share the same speed metric of O(N). There may be an algorithm or several algorithms that are faster. Look it up, you might find it.
What I can guarantee you can’t find is the best design pattern or style of programming. There is no theory here let alone a definition for “best.”
It’s obvious your question is trying to take the conversation in some other direction. Let me try to guess it. Your probably going to talk about other intrinsic downfalls and costs to sorting outside of the speed metric like memory or the domain limitation for O(N) sorting algos. To which I respond memory has nothing to do with the speed of the algorithm and is off topic from my claim. We know the fastest algorithm(s) by definition, because the theory of what is faster and what is not is fully fleshed out.
The theory on which is the best algorithm to use in a problem given the speed and memory cost is something we don’t have a theory for yet, hence the usage of best practices, which further proves my point. This is where you conflate things, you didn’t separate aspects of areas described by theory and aspects not yet described by theory. You mistakenly lumped everything together.
Hopefully I anticipated your actions correctly and you are the one who is taught a lesson here rather then your intention of teaching me something via your trap question.
There's the best practice. It's such a strong and useful one that it doesn't feel optional any more.
Best practice is definitely a weasel word. Something that should be qualified with what is being optimized, what prompted its adoption, and comparisons to other approaches that highlight/justify its use.
It also falsely implies there is no better candidate out there (another computer science irony: your "best practice" has been proven via magical super-turing computation to truly be the best ever?).
It assumes false authority.
Really, they are only good for setting a reasonable standard and moving forward rather than constant bikeshedding.
Where it's not rare is medicine. You regularly hear the term spat out like a weapon, particularly in big hospitals.
It's a very poor way to justify something. It's a statement of the obvious: everyone agrees you should use the best or most efficient way of doing something. What people don't agree on is what that is. So it's "we all agree we should be doing it the best way, and I say the best way is ...". You get no indication of why it's best practice, or how it's been chosen, or who chose it. Its more an appeal to authority.
There is another term in medicine that's also bandied about frequently: "evidence based medicine". Unlike "best practice", it assums the obvious: everyone agrees we should be using the best way. It's telling you how the best way way chosen: from the evidence. More importantly it's also giving you permission to improve on it: find your own evidence. If you asked for to see evidence for "best practice", it's often seen as an attempt to undermine authority. If you asked to see the evidence behind evidence based medicine, the reaction your likely to see is "oh, you get it, here is what I have".
When I hear "because it's best practice" or "this is best practice" now, I shudder. It's not an invitation to be part of a team working to produce the best outcome. It's telling you you're a minion, and you'd better follow the rules.
But I think you are right in one sense, the reason we see it used far less in software development is we can often shoot something down with a few sentences of tightly reasoned logic. Evidence is often easy to come by. Medicine doesn't have that luxury, the body is too complex to reason about. It does have rigor and scientific analysis, but it takes a big studies and a lot of effort to gather.
Logic and Science are different in the sense that Logic does not require any evidence. Logic uses simple assumptions called axioms and then theorems derived from those axioms to make broader statements about the system. A computer is basically a system molded around logic so that you can use logical reasoning to derive statements and ignore evidence. This is what I mean by "rigor." Ironically, for convenience, testing (aka science) is used in computer programming even though formal analysis is a stronger (but more challenging) form of proof.
Science on the other hand is more evidence based. If evidence contradicts logic than the evidence takes precedence. The medical field is more science than it is logic hence the distinction. A computer programmer can use heavier reliance on axiomatic reasoning to derive statements but the medical field will defer more to the black box statistical results of evidence.
The difference is literally the same difference between math and science. Computer Programming blurs the line a bit but overall programming is closer to axiomatic reasoning than it is to science.
There are natural patterns in business problems, so there are natural patterns in solutions, which can be encapsulated in best practices.
Companies don't have the time or resources or development skills to come up with elegant, boutique solutions to every problems. Even FAANG companies with billions of dollars to throw at problems hit a natural wall.
So, they resort to proven best practices to solve most problems. It's wise for smaller companies to analyze proven best practices and incorporate them as needed.
> There are natural patterns
You just named the alternative. We don't see people say "Singletons or GTFO!", we have a toolbox that we apply to problems as needed. "Best" practice implies there is one and only one "correct" way to do things. "Good practice" would be a better term.