Putting something behind an interface / abstract factory / whatever ironically often makes it harder to change because you have to read more lines of code before you can start modifying the program. Programs with a lot of lines of code are intimidating.
There are situations where you have clean module boundaries, which lets you separate your code into separate independent modules. (Eg POSIX separating user applications and libc / the kernel). But the downside is that once you've separated your code into modules (with documented APIs) the module boundary itself becomes harder to change. You're essentially betting that you have the right API. You lose flexibility at the boundary in exchange for flexibility in the code on either side of that API.
But all that said, I don't think there's any way to learn this stuff without experience. "The dao that can be spoken is not the eternal dao". There's no set of rules for this stuff that can be explained simply, and which will give you the ideal outcome in all situations.
I think the “secret” is proportionality. A useful abstraction hides significantly more complexity than it introduces. That could be a single function, if that gives a name to a simple but common algorithm on a particular data structure like map or reduce. Or it could be a 100 function API with 100,000 more lines behind it, if that provides a correct, efficient implementation of some entire field of programming that your application rests on. But it’s probably not a one-line function that needs a longer name to describe what it does in words than its whole implementation, and it’s probably not a library with a 100 function API that only obfuscates a few standard data structures and custom types that are locked away in the implementation for little benefit.
Thanks for expressing this - I think this is a fantastic insight. I love C as a language, and I've been really enjoying Rust lately too. Both languages make it easy to design around simple, raw structs. And a simple struct with 3 public fields is often a much easier interface to work with than 100 getter & setter methods hiding the same struct which has private fields.
When I try to build similar abstractions over business processes, they typically don't survive the next wave of changing requirements.
I concur. If we look at abstractions that have stood the test of time, they are often relatively simple to describe, but they express powerful ideas that are widely applicable. A few examples come to mind:
• Structured programming
• Iterators and generators
• Promises
• Atomic transactions on databases
• Model-view-whatever patterns in UI code
• Regular expressions
• Pipelines
• Continuations
• Haskell’s standard typeclasses (particularly Functor, Applicative, Monad, Monoid, Foldable and Traversable)
Many of these have proved so useful that popular languages now directly incorporate them or perhaps, for the most abstract ones, incorporate features that are more specific instances of the general idea.
I think the last example is particularly telling. The patterns Haskell developers work with are in one sense very abstract: each is ultimately just a short list of mathematical properties that some type can have. This turns out to be quite a high barrier to entry and understanding their true nature is something of a rite of passage that many fledgling Haskell developers never complete. It also took some very smart people many years of thought and discussion before the community eventually reached the list above.
However, once you do recognise the patterns, you see them everywhere. They create some very clear seams in your software design that allow separation of concerns and composability to a degree that less powerful abstractions only dream of. And because they’re rooted in relatively simple properties, they’re about as stable and future-proof as anything in programming can be.
Maybe one day they’ll be supplanted by a more effective abstraction with better language support. Effect systems, perhaps? But for now, you can express ideas in a few lines of Haskell that would take 10x that in many other programming languages, because you can compose tools from a vast toolbox of data structures, algorithms and design patterns in almost arbitrary ways. But you have to learn some abstractions that aren’t widely known and have silly names first. :-)
> I have more success writing reusable libraries that do very, very fundamental stuff
p.s. I also wrote that same library when go generics landed.
You might be able to do that. That code (most likely) still needs to solve a complex issue and needs documentation to survive next to it. Also, there is a good chance that the next guy coming to fix it is not going to investigate the extra time to fully understand the code if he can fix that one crash by throwing in a hotfix at the specific line accessing a nullpointer. Lastly, at some point, if you need to solve a difficult problem, the code to do so is likely going to be more difficult to read, too.
It's not impossible to do this, but having the time to do a clean solution, having the foresight to be right about the necessary complexity and also have the environment for it to stay clean is simply rare.
This is true for systems where you already have working code and you have multiple concrete examples of code that would benefit from refactoring or rewriting a reusable module.
I think the skepticism is more toward the idea that you should design for reuse up front before you have working code. There are two failure modes. One is that you spend time making something reusable that you only end up using in one place. The other is that you bake assumptions into the code that later turn out to be false. In both cases, the resources spent on reusability could have a net negative impact on the project.