A related principle is what I call code locality. Instruction locality is the grouping of related instructons so they can be the CPU's cache (can an inner loop fit all in cache). Similar for data locality. Code locality is for humans to discover and remember related code. As an example of this is those times you make an internal function because you have to but it has a terrible abstraction (say a one-off comprator for a sort), its best for comprehending the caller to be near where its needed within the same file rather than in a separate file or in a dependency in another repo.
Applying code locality to SPOT, when you do need multiple sources of truth, keep them as close together as possible in the code.
Not that such tools don't have their place, but I've seen too much convoluted code that has broken simple things into little interconnected bits for no reason other than someone thinking "this is how you're supposed to write code" and having fun adding abstractions they Ain't Gonna Need.
If I understand some of it correctly, I was contemplating this when I started writing functions for "single functional concepts" like, "check for X; return true or false", then called each of those functions sequentially in a single "run" function. Is that what you mean?
I found that approach much easier to test the functions and catch bugs, but your comment seems to go against that.
Dividing things into methods mostly gives the benefit of naming and reuse. If you can skip the method's content and reasonably assume it works in some way, it can make reading easier.
If the reader instead feels inclined to read it, they now have to keep context of what they read, jump to the function (which is at a different position), then read the content of the function, then jump back while keeping the information of the function in mind. In bad cases, this can mean the reader has to keep flipping from A to B and back multiple times as they gain more understanding or forget an important detail.
The same thing happens with dividing things up in multiple classes. Don't need to read other classes? Great. Do need to read other classes? Now you have to change navigation to not only vertical, but horizontal as well (multiple files). Some people struggle with it, some people hate it, other people prefer it.
You're basically making a guess what's best in this scenario, and there isn't a silver bullet. The extreme examples are easy to rationalize why they are bad. The cases closer to the middle, not so much.
I would say some junior devs can get too fixated on hierarchies and patterns and potential areas of code re-use, though; instead, they should try to write code that addresses core problems, and worry about creating more correct abstractions later. Just like premature optimization is the "root of all evil", the same goes for premature refactoring. This is the rule of YAGNI: You Ain't Gonna Need It. Don't write code for problems you think you might have at some indeterminate point in the future.
When it comes to testing, TDD adherents will disagree, but if you ask me it's overkill to test small private subroutines (or even going so far as to test individual lines of code). For example, if I have a hashing class, I'm just going to feed the hash's test vector into the class and call it done. I'm not going to split some bit of bit-shift rotation code off into a separate method and test only that; if there's a bug in that part, I'll find it fast enough without needing to give it its own unit test. That's what debuggers are for. All the unit test should tell me is whether I can be confident that the hashing part of my code is working and won't be the cause of any bugs up the line.
Obviously I'm not in the "tests first" camp; instead I write tests once a class is complete enough to have a clearly defined responsibility and I can test that those responsibilities are being fulfilled correctly.
>I would say some junior devs can get too fixated on hierarchies and patterns and potential areas of code re-use, though;
Agreed.
> instead, they should try to write code that addresses core problems,
Still with you, agreed.
> and worry about creating more correct abstractions later.
I don't quite agree. I agree you shouldn't spend too much time, but I think "don't worry about it" gets you a hodgepodge of spaghetti code and half-baked ideas that no one can maintain in the future.
One of the most important things someone can do when adding a feature for instance is understanding the surrounding code, it's intentions, and how any abstractions it may have work, and then add their feature in a way that complements that and doesn't break backwards compatibility.
I'd go as far as arguing that only ever MVP'ing every story without a thought to design or abstraction is one of the major problems in industry alongside cargo-culting code maintenance rather than ever trying to form deep understanding of any meaningful part of the software.
> Just like premature optimization is the "root of all evil", the same goes for premature refactoring. This is the rule of YAGNI: You Ain't Gonna Need It. Don't write code for problems you think you might have at some indeterminate point in the future.
YAGNI is far too prescriptive and misses the point of programming that literate programming gets right:
Programming (like writing) is about communicating intent to other human beings in an understandable way and shuffling complexity around in the way it makes the most sense for:
- The typical reader skimming to understand something else (Needs to know: what does it do?) - The feature adder (needs to know how it works, so they need a high level, then easy way to understand the low level as needed) - The deep reader (needs to be able to take in all of the code to deeply understand it, needs straightforward path to get there)
What you describe sounds like no abstraction and just throwing all of the complexity in front of everyones faces all at once. I can appreciate the habitability advantage of that, but I think that discarding context of acquired domain knowledge as you work on issues is too great of a cost.
In case it's not obvious, the words came to me for the point I'm trying to make: Domain knowledge you acquire while working on something should be encoded in sensible abstractions that others can uncover later on, peeling away more and more complex layers as needed.
> When it comes to testing, TDD adherents will disagree, but if you ask me it's overkill to test small private subroutines (or even going so far as to test individual lines of code). For example, if I have a hashing class, I'm just going to feed the hash's test vector into the class and call it done. I'm not going to split some bit of bit-shift rotation code off into a separate method and test only that; if there's a bug in that part, I'll find it fast enough without needing to give it its own unit test. That's what debuggers are for. All the unit test should tell me is whether I can be confident that the hashing part of my code is working and won't be the cause of any bugs up the line.
This view sounds like it may be a direct result of a YAGNI/avoid abstraction style to me actually. If you avoid abstracting or code-reuse quite a lot (or even don't spend enough energy on it), you lose one of the largest benefits of TDD:
regression testing
If nearly all of your functions are single use or don't cross module boundaries... then the value add of TDD's regression testing never really has a chance to multiply.
For OOP I feel like this would be reflected in terms of testing base objects the most or static methods. For functional code, it would just be shared functions.
> Obviously I'm not in the "tests first" camp; instead I write tests once a class is complete enough to have a clearly defined responsibility and I can test that those responsibilities are being fulfilled correctly
I'd argue you are testing in your head anyway. Visualizing, conceptualizing, and trying to shape the essence of the problem into something that makes sense.
The problem is sometimes our mental compilers/interpreters aren't perfect and the mistakes are reflected as kludges or tech debt in our code.
If a lot of conditions are checked sequentially, why not write down exactly that? The sequence itself may well be what a reader would like to know.
The one benefit of the added function calls would be that the definition order does not need to change even if there are future changes to the execution order. But that is exactly also what adds mental overhead for the reader.
If the names add context, then that's a perfect use for comments.
They're making a connection between non-jumping code with a list, and jumping code, like classes, functions, etc. with a graph. A list can be iterated from beginning to end and read in sequence, or can be jumped into at any point in the code at arbitrary points. Similarly, if I open a file and want to read code, I can jump to any arbitrary line in the code and go forwards or backwards to see what the sequence of code is like. I can guarantee that line n happens before line n+1.
With functions, classes, modules, whatever, the actual code is located somewhere else. So for me to understand what the program is doing I have to trust that the function is doing what it says it's doing, or "jump" in the code to that section. My sequence is broken. Furthermore, because I am now physically in a new part of the code, my own internal "cache" is insufficient. It takes me some effort to understand this new piece of code and re-initialize the cache in my mind.
The overall thrust of DRY is overrated is that we often times take DRY too literally. If I have repeated myself I MUST find a way to remove that repetition; this typically means writing a function. Whereas previously the code could be read top to bottom now I must make a jump to somewhere else to understand it. The question isn't whether this is valuable, rather, whether it is overused.
The graph data structure, I believe, is the most generic of data structures, and it can be used to represent any other data structure. This arguably makes it both the most powerful data structure and the, IMO, the one of last resort.
To be clear, I (like the parent post) was using data structures as a metaphor for the organization of source code: "trees" being code that is conceptually like a branching flowchart with many separate nodes, and "lists" being lines of code kept together in one function/class/file (which the parent post points out has the benefit of "locality").
Trees invite recursion, and as the logic grows telling what is going on gets more and more difficult. While it's easier to process trees recursively, it is possible to iterate over it. It's easier to process lists iteratively, but it is possible to handle them recursively. Some people do, much to the detriment of all of their coworkers.
I'm trying to unwind some filthy recursive code on a side project at work. There were two bugs I knew about going in, and two more have been found since, while removing mutually recursive code in favor of self recursion, and then modifying that to iteration.
Iteration is much, much easier to scale to add additional concerns. Dramatically so when in a CD environment where a change or addition needs to be launched darkly. Lists can help, especially with how carefully you have to watch the code. They're not the only solution, they're just the easiest one to explain without offering a demonstration.
If you want a "different truth" create another object. And never assume that the truths of different objects must be the same.
That is a difficult condition to achieve in practice we often code based on what we know a method-result "must be, therefore it is ok to divide by it etc."
This is also why using layers as your primary method of organization is usually a bad decision. You're taking things that change together and strewing them about in different top level directories.
In this case, I sometimes prefer a slightly different approach; let's call it CRAP - Cross Reference Against Protocol: instead of definitions effectively referring physically to the same point of truth, they are designed instead such that they simply cross reference against a protocol, warn if there has been a deviation in protocol giving the developer the opportunity to go back and correct as necessary, or otherwise allow it and manually sever the link to the protocol if one is no longer desired.
This prevents against SPOT's weakness (at the cost of some extra manual work / due diligence by the developer), which is accidentally enforcing future convergence in situations when previous convergence was incidental/accidental, and divergence should have been allowed to take place instead.
Just watch what happened when the OpenVDB project won the 2014 Academy's Scientific & Technical Achievement Award hosted by Margot Robbie and Miles Teller on February 7, 2015 at the Beverly Wilshire.
In your CRAP model it seems like you’d be relying on tests or assertions to verify the “protocol”, if I’m understanding that correctly. Which means finding all the places where something needs to be true and then testing for it. Seems like a lot of work and wouldn’t necessarily catch all those places.
Feels similar to the monolith/microservice discussion in that it's mostly context sensitive. I think the term "programming principle" is misleading, these are tools with specific applications.
But I definitely understand not everyone wants to write all of their backend in JS (or even TS). (Node ORMs sometimes don't have the polish of their cousins in other languages, for instance.)
There are great opportunities here for language mixing, however, in writing that complex algorithm in the same language for both frontend and backend but the rest of the codebases may not be in the same language.
1. V8/JavaScriptCore/SpiderMonkey are all very easy to rehost as "scripting languages" inside many languages people prefer to use on the backend. It's often easy to find wrappers to call JS scripts and marshal back out their results inside the backend language of your choice these days. You pay for transitions to/from JS to your primary backend language, of course, so it takes careful management to make sure these "business rules in JS scripts" aren't in any hotpaths, but a backend may have the cycles for that anyway.
2. WASM is offering more opportunities to bring some of your preferred backend language to the frontend. You probably don't want to write your whole frontend in WASM still today, as WASM still doesn't have the same DOM access, but using WASM for single point of truth business rules has gotten quite viable. You still pay for the startup costs and marshalling costs between JS and WASM, but in many cases that's all still less than network call. (I'm still skeptical of tools like Blazor for .NET, especially those building full frontends with it, but I definitely appreciate that reach of "write once" business logic for both client and server.)
(In fact, JS is the only "universal" scripting language in mobile due to Apple's fun restrictions that JavaScriptCore is the only JIT engine allowed on iOS. It really is the strongest option today for language to write things in if they absolutely need to be shared across all possible operating environments.)
But yes, if you have truly separate codebases it becomes more difficult, and protocols need to be quite stable and changes to them carefully managed.
Or you derive one of the implementations from the other, or both from a third one.
But yeah, heterogeneous environments have a tendency of creating unnecessary code duplication.
Checks don't have to include the definition of knowledge independently, so multiple checks against the same rule don't need to be a violation. As a simple example, if you have a JSON Schema, that is the single source of truth for validation, and you can validate against it in 16 different places at different stages of processing and you haven't violated the principle that each piece of knowledge should be represented once in a system.
I'd also say there is a lot of nuance about what "truth" is (i.e. is a pizza crust/sauce/cheese an essential truth that should have a single source).
Some DRY definitions I read actually tie in SST but I think many devs don't bring that nuance to it.
“Every piece of knowledge must have a single, unambiguous, authoritative representation within a system” is the verbatim definition of DRY from when the DRY principle was first articulated.
Truth implies facts, knowledge implies understanding the meaning and associated course of actions.
This is the second time in as many days that I've read something purporting to go beyond some original. The one yesterday was "we need more than the four types of documentation." All the examples fit into the four types as originally defined.
In HR training, there was even an entire segment on the "Platinum rule" because the Golden rule isn't good enough. Yet anyone who works to understand the Golden rule to any depth knows that it encompasses every "enhancement" the Platinum rule intends without any of the side effects.
What kind of failure is occurring such that definitions don't function any more, I wonder?
One could also use “SPOTify X” to mean “reducing X to a SPOT”. :)
The mechanism to "SPOT that code out, bro!" can be applying varying techniques for reducing coupling, and tightening cohesion. A proper review on a merge request should be making more specific comments about how, not a hand-wave to say "DRY that sucker up."
One final comment: DRY is three characters, therefore obviously more efficient :p
But why call it "Don't Repeat Yourself" if it actually means something somewhat more subtle than that. I firmly believe many junior developers don't grasp the nuance and based on the comments I'm not the only one who thinks this. So if DRY is widely understood by developers to mean literally "don't repeat yourself" and nothing more, does it really matter how the formal definition phrases it?
In any event, if SPOT / SST and DRY do mean the exact same thing, I like SPOT / SST better because the names encode the essential concepts of the principle.
Rules are like alarms they draw our attention to some peculiar condition which gives us pause to think about if it's kosher and if not why not.
Someone says "never do this" or "always do that" and you can apply those rules with abandon (often leaving a maintenance nightmare in your wake).
There are no rules to programming.
Another metaphor I like for programming is Chess. Any line you add to the program constrains its future development, becomes "weight" or "force" that pulls your development into some direction. Sometimes you have to sacrifice features like pawns. Sometimes you may sacrifice security, you may think it is secure enough. The outcome of this game is often a draw, or stalemate. And the same game can continue for years.
Any principle like this that is applied to code is wrong.
It is very hard to find out if the definition already exists or not in the codebase. This can lead to multiple definitions of the same thing or the truth.
anyone has a good way to deal with this?
It should be an automatic thought when implementing some logic to think about which other parts of the system need to be consistent with that logic, and then try to couple them in a way that will prevent them from inadvertently diverging and becoming inconsistent in the future.
In terms of software design, a more general way to think about this is that stuff that (necessarily) changes together (is strongly coupled) should be placed together (have high cohesion).
That's why I'm pretty dogmatic in applying DRY to infrastructure as code, for example, as opposed to generically for every code base, because finding or depending on repeated identifiers that have to be the same is such a source of error here.
</inevitable_joke>
This is not a reason to avoid SPOT altogether, but one think through that situation as part of their mental calculus on pros & cons.
Writing code such that it’s reasonably easy to decouple or recombine existing uses is mostly orthogonal, I think. Usually you can just duplicate whatever is at the SPOT when the need for more than one truth arises.
Nit: the article’s problems also have an obvious solution—named parameters with default values when not provided.
I can think of using a Rule Engine but not sure if there are any performant ones and they don't seem to be used much.