Death by design patterns, or On the cognitive load of abstractions in the code | Hacker News Reader

Death by design patterns, or On the cognitive load of abstractions in the code

alentred.medium.com

178 pointsalentred169 comments

It's not the number of abstractions, but the quality of abstractions. If abstractions reduce cognitive load, they are good. If abstractions increase cognitive load, they are bad. Often times good abstractions are obvious within the problem domain.

To this end, I think Haskell, in its pursuit of the purest, can lead to incredibly-high cognitive load abstractions that make reading code daunting. It feels good when writing it, but feels awful when reading it. Sometimes applying the DRY principle can be bad if the thing you are generalizing will lead to higher cognitive load for people referencing it.

Also I think abstractions vary in quality depending on the language the abstraction is embedded in. Functional paradigms in C++ are thorny to work with; even though functional paradigms are good abstractions in general, if the language doesn't support it first-class (i.e. the cognitive load of using them is high), then they are not good abstractions.

You can also consider maps in a language like Clojure, where maps are almost "zero-class"; it's like the language was built for maps. Nested maps have incredibly low cognitive load in Clojure (not only because the syntax supports it, but also because functions are standardized). Nested maps in Common Lisp, however, are not as nice as those in Clojure; you have to import non-standard libraries in order to deal with them (and even more obscure libraries to support Clojure-like read macros).

The key question is: when someone else reads this code, how much do they need to "reconstruct" the context? Can they get away with not reconstructing the context? And if they need to, how fast can they do so, via appropriate naming, comments, and types?

The key determinant, imo, is precisely and comprehensively descriptive names

Even if this leads to humourously Java-esque code, it's worth it

An abstraction that has one purpose and is named to make that purpose crystal clear is a good abstraction

> Even if this leads to humourously Java-esque code, it's worth it

Some people seem to get mad about the verbose naming common in Java - but it’s one of the biggest blessings I’ve ever experienced. If you name things after what they do, and that name is stupid, then it’s the quickest indicator of bad design I’ve ever seen. Good design is where every name is patently obvious and encompasses the entire purpose of the class / record / method / whatever.

I think it's not the verbose names that are the problem in stereotypical Java-esque names, but rather the large amount of "scaffolding" classes that implement design patterns - a wealth of middleware that becomes apparent thanks to the verbose naming. That is, it's not the ProcessedWidget::QueryStructuralIntegrityPercent() that's the issue - it's the associated WidgetFactory, ProcessedWidgetBuilder, ProcessedWidgetProcessingManagerDelegateProxy, etc. that bloat the code and tax cognitive memory, and which exist only because the language isn't (or used to not be) expressive enough to express those patterns without giving them a name.

There's nothing like ProcessedWidgetProcessingManagerDelegateProxy in Java or the standard library. I feel that Java gets whacked with a cudgel that should be aimed at middleware authors.

There's nothing in JavaScript forcing you to use 500 dependencies across 100 000 folders either. Opinions on a programming language are often, for better or worse, opinions on the language plus its community/ecosystem. And correct me if I'm wrong, but isn't the company responsible for maintaining Java (Sun, now Oracle) also one of the worst offenders in terms of naming things in its middleware?

> If you name things after what they do, and that name is stupid, then it’s the quickest indicator of bad design I’ve ever seen.

I agree it’s a smell, but I’d caution that it’s also very situational.

Sometimes the domain has a recognizable category of like-things which don’t have a name for their likeness within the same domain, and choosing even a stupid-sounding name is a good start towards choosing a better name.

Other times, a targeted portion of an existing codebase might have a common theme which doesn’t necessarily line up with the domain, but needs to be named something to untangle what outlived its WET shelf life. You can recognize it’s a stable abstraction, you can recognize it's entirely divorced from any domain concern except by happenstance. You just have to give it a name, or let it be an increasingly unnecessary burden.

It’s easier to take such a hard line on the design quality implications of naming when you have a vocabulary to reference and/or compose, or when you have extant design attitudes relatively aligned with that principle. It’s especially difficult to apply the principle in systems with extant design problems of this nature, because whatever established or emergent abstractions do exist might not align with any principle you’d apply, and you can’t move them in that direction without some intermediate step no matter how flawed. You have to name what is before you name what should be.

> If you name things after what they do, and that name is stupid, then it’s the quickest indicator of bad design I’ve ever seen.

As a reader of the code I think 'FooAndMaybeBar_SpecialBazSupportV2 ()' is a blessing.

Aiming for cleaness when the problem is not clean is a worse sin than messy code that could be clean.

Doesn’t matter how good your names are if your control flow is incomprehensible or if the data representation is some bloated or mangled monstrosity.

and every time i complain about names during code reviews i get accused of bike shedding

The values you're espousing here are the core lesson I took from John Ousterhout's A Philosophy of Software Design.

> To this end, I think Haskell, in its pursuit of the purest, can lead to incredibly-high cognitive load abstractions that make reading code daunting.

Hard disagree. The Haskell abstractions in the standard library are (mostly) great. Classes like `Monoid` or `Functor` as as basic as they get (in the sense that they describe a very small set of behaviors) and include laws.

Of course, user defined classes can incur in high cognitive load, but that is applicable to any language that supports abstraction.

I feel the problem is more that Haskell is one of those languages that push against the unmovable barrier of dimensionality. Yes, monads and related ideas that get increasingly adopted by other languages are great. They let you express some things more clearly than before - especially when it comes to cross-cutting concerns like error handling - but only to a point.

At some point, you're trying to optimize readability of code across multiple concerns, each having a good claim to being of prime importance. But you can't. You can try and invent increasingly obscure mathematical abstractions to make some concerns special cases of a more general one - but in the end, there's only so much you can cram into the same piece of plaintext. You have to trade readability for some readers (e.g. those interested in "golden path" business rules) against readability for others (e.g. those interested in error propagation).

Or, you can double down on further abstractions and make everything unreadable for everyone equally :).

The underlying problem is partially about plaintext format itself, but mostly about the fact that we always work on the same, single representation, and demand it to be everything for everyone. I feel the only way to progress is to finally give up the constraint of directly reading and writing the final "single source of truth" code. It will definitely simplify things day-to-day, as it'll allow you to plain ignore and hide things you don't care about at a particular moment, instead of trying to keep them concise with design patterns, clever syntax, advanced algebra, etc.

I think abstraction is promoted way too much in programming. People use lisp macros as an example of too much abstraction since it allows you to write a DSL that only you understand yet they celebrate the poor man's DSL they create using their OOP abstractions.

I like the math homework approach to programming. First you get in there with a vague idea of what you need to do, you bang your head against it until it starts to make some sense. You then move on to more complicated examples only to realize that what you thought you knew was wrong. After going through many of these iterations you might start to notice some patterns common to all examples and if you're lucky they might turn out to be true. This is rare and only happens when you have a very good understanding of the problem.

  s/People use/People who have never written a line of code in it use/

Abstractions, like so many other things, can be "difficult" along two axes: they can be hard to learn, or they can be hard to use. The former increase cognitive load as you're learning them, sure, but they can also pay off massively once you do—abstract math is hard but also lets us manage complexity and express ideas we would not be able to handle otherwise. On the other hand, lots of abstractions aren't particularly hard to learn (usually because they aren't very abstract!), but either don't do much or carry an ongoing amount of complexity as you use them. It doesn't matter how well you've learned about pointers or malloc + free, non-trivial code using them requires a lot of care and is easy to mess up. (This is a controversial point, but it should be clear given the number of bugs and security vulnerabilities we find due to memory safety issues in real code!)

I see it as a trade-off between abstract thought and working memory. You can spend time up-front to reduce the amount of details you need to keep in your head as you go along, or you can continue to juggle more details to learn less up-front.

The problem, of course, is that most people don't differentiate between the two. In some sense, it's only fair: if an abstraction carries some up-front cognitive cost as you're learning it, how do you know it'll get better? Do you even have the time to learn something new right this moment? But, ultimately, it's a massive difference and avoiding abstractions that are hard up-front is self-defeating in the long term.

Haskell abstractions mostly fall in the former camp: hard to learn, but powerful once you have. I've worked with some pretty poor Haskell code and the abstractions that seemed hard when I was a beginner were exactly what made it easier to understand messy code! Turns out that expressive types and effect management mean that I don't have to carefully understand which parts of the code can affect each other indirectly and which parts can't; it's all explicit in the structure of the code. I had to learn the language and the concepts to understand it but, once I did, I could read it directly from the program rather than needing to simulate the code in my head.

I've found that, most of the time, the KISS design philosophy is heavily weighted in the opposite direction: it's all about avoiding concepts and abstractions that a reader would need to learn, but at the expense of having everybody keep more details in their head as they're writing and reading the code. The small pieces might each be easier to understand, but there are a whole bunch more pieces to track for the same amount of logic!

The problem with a blanket "how hard will somebody else find this code to read?" is that so much of it rides on familiarity rather than anything fundamental to the code. And that's what matters in any specific situation, to be sure... but it doesn't answer the broader question of how we should be writing (and reading) code. After all, even if familiarity dominates in the immediate short term, it might still be worth up-front learning to have a better experience in the medium and long term.

I fully agree with you. You mentioned that sometimes applying the DRY principle is a bad thing. Really, people often adhere to the DRY principle far too dogmatically.

It’s better to have some duplication than to end up with a wrong abstraction.

I like the AHA principle much more. It suggests:

  - Avoid Hasty Abstractions
  - Prefer duplication over the wrong abstraction because duplication is far cheaper than the wrong abstraction

I found it here: https://kentcdodds.com/blog/aha-programming.

The point about duplication being better than wrong abstraction is made here: https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction.

And the principle doesn’t suggest avoiding abstractions altogether. It’s really only about avoiding the hasty ones. Wait with the abstraction until you feel it’s necessary or when duplication itself becomes a problem. At that point, you’ll have a clearer understanding of how the abstraction should function.

> people often adhere to the DRY principle far too dogmatically

This is a problem with all rule of thumbs for programmers. It is like we as a group are far more dogmatic than normal. Or our rules are worse. Or both. Dunno.

I really liked the way Sussman discussed repetition in the SICP lectures. Something like, “If you find yourself repeating the something, that’s often a hint that there could be a useful abstraction.” It’s a far less prescriptive (and more appropriately flexible) way of expressing the same general idea than DRY.

> The key question is: when someone else reads this code, how much do they need to "reconstruct" the context? Can they get away with not reconstructing the context? And if they need to, how fast can they do so, via appropriate naming, comments, and types?

That is indeed the key question, but the answer will have a lot to do with their experience as a developer, their knowledge of the domain, and their familiarity with the code.

I'll take poor quality abstractions any day, provided they are coupled with amazing documentation, even if in the form of well written comments, that describe the reasoning behind them.

I always told my devs: imagine the dumbest programmer you ever knew. Take me, your manager, for example. Will I be able to understand and fix/modify your code 10 years down the road when you all have left?

That pretty much excludes using any npm library like redux or webpack.

I was coming here thinking the same thing. The important thing isn’t abstractions in general, it’s whether those abstractions are useful (defined here as reducing the cognitive load of a reader, but there’s probably dozens of other meaningful definitions of a “useful” abstraction). That can depend on a number of factors, e.g. the particular programming idioms, flavor, style, etc. of a team; the quality of documentation and onboarding of new engineers to a team; the specific task at hand, etc.

Worst offender for me is when I can't access the implementation by just clicking on the type or variable, because it is an interface.

For example with dependency injection in languages where devs put Interfaces in constructors and some magic injects the Class that satisfies that Interface at runtime. When you click the type or variable, the IDE opens the Interface file which is just a bunch of function signatures, no implementation. It's infuriating when all you want is to see some concrete code. You just want to see exactly what code is being executed and then carry on with your day.

I get the benefits of dependency injection but most of the times a single class implements that interface and yet i'm forced to scavenge the code to find it. I also know some tooling is powerful enough to list the Classes that satisfy that Interface. But it's not perfect and it's still a layer of indirection adding to my cognitive load.

I agree. In Visual Studio I’ve gotten used to using “Ctrl F12” to go to “implementation” instead of F12 for go to definition. If there’s only one implementation it takes you straight to it, and if there’s multiple it lists them so you can pick one.

Visual Studio has “Go To Definition” and “Go To Implementation” - the first will take you to the interface, the second will take you to the class that implements the interface (or show results for multiple implementations if those exist). I would try to find that functionality in your tools because it makes a world of difference.

This is one of the nice things about Visual Studio: F12 jumps to the declaration (the interface) and Ctrl-F12 jumps to the implementation (the service) or presents a chooser if there are multiple implementors.

I find that shortcut easy/powerful enough that it's no additional burden on my cognitive load. At least, it's no more taxing than inversion of control is generally.

My IDE (Jetbrains) has a „jump to implementation“ feature. If there is just one class implementing the interface it just jumps there. If there are multiple you get a drop down.

> I get the benefits of dependency injection

I don't.

I think dependency injection encourages thoughtless code. It makes it easier to "magically" make your dependencies appear wherever you want them without having to think about anything except the lifetime. Once you're project gets significantly complex, it becomes very difficult to understand what your web of dependencies is actually doing, and to comprehend how the lifetimes all interact.

Screw dependency injection. Do it the "hard way" and make the pain points ridiculously clear so that you can see where the flaws in your architecture are revealing themselves.

I am with you. The analogy I came up with is "map layers". Usually on a geographical map the features are drawn using contours, colours and accompanying names. Such map is, certainly, not a "territory" but is a good representation and easy to work with. Using (capital DI!) Dependency Injection is like a having a map "lobotomized", split into layers where you can never see them together. You either see contours devoid of names, or names hanging in the void. Bonus point if names don't even follow the contours. For any non-trivial number of classes/concepts my brain explodes.

Another bonus point when DI is used for some deep-context "dependencies". Hey, I need a HttpClient object here, to post something somewhere. Why, instead of having a URL and instantiating my client right here, I need to dig through the sands of DI bindings to find a way to "configure" HttpClient for this particular instance? Well, I understand theoretical reasons, but in practice they never make sense and only add pain.

Another bonus point for run-time binding errors. Those sometimes slip through even when there is a "test" in the CI/CD suite for that. I think dependency injection has its place, but as used presently it resembles a cargo-cult.

For me, peak-design-pattern hit right about the time I worked on two different JVM-in-Java projects (2002-ish and 2008-ish). The problem, I finally realized, is that Java programmers are taught that every class is a special snowflake in need of armor and adornments. A good Java class should be honking and hollering! It should never expose a field, never have coupling with another class that couldn't be interposed upon, and should always factor commonality into a superclass. At a minimum it should have fifteen methods, repeat the same names at least 7 times each, and be JavaDoc'd to the point your eyes are bleeding. Even if it just holds three fields and does nothing at all. ESPECIALLY IF IT DOES NOTHING AT ALL. It must pretend to be important! It should be long enough that you spend most of your time scrolling. If you aren't scrolling, you aren't coding!

What. A. Nightmare. It's like trying to build an internal combustion engine and every moving part has three extra degrees of freedom, in case you wanted to pull a piston out in the middle of the freeway and use it to wipe your windshield! Just in case you want to do that, a piston can also function as a wheel, a cigarette lighter, or a potted plant. And of course it has the, ahem, armor (?) to do all of that. In fact, every nut and bolt and rod and wire has extra armor on it. They aren't bolted together, they are held with miniature six-axis robot arms, just in case. And everything has plating. Thick metal plating. Bullet-proof plating. Everything indestructible. Getters and setters everywhere! Can't you see how much better they make things! The design is so much better with getters and setterrrrss!

Needless to say, I don't write code like that anymore. I prefer to write in Virgil, to make my fields public and immutable, initialized in the constructor or the anonymous zone, and when I need six classes to cooperate to do a job, they do, and they don't put armor on for friends. Private applies to a file, everybody's friends there, no surprises.

But I don't expect anyone to follow me or worship it. I won't claim amazing design skills. Just trying to make things work without a lot of extra work or surprises down the line.

I have had the same experience, interestingly in the same time frame. I felt that when doing Java projects I needed to enter this special telepathic flow-state with my IDE and cut-paste shortcuts and change huge chunks of code every time the model changed a little bit. Transitioning to immutable data classes with functions operating on the classes, and abstractions focusing on what functions operated on them and in what context made life much easier.

Today this is the microservice. Why reverse a string when you can build a string-reverse-api microservice to handle it?

Microservices are a good way to turn a 15 person project requiring 3 subteams and a little project planning into a 150 person project with 50 teams and no planning.

Yeah. You know what the investors see when you do that? Growth

What I learned from my last manager is that if you are planning that means you are doing waterfall and waterfall was the bad old days much different from the agile good new days!

Way too many people substitute pattern matching for critical thinking.

One of the strangest ironies of my career is that the smartest developers often write the worst code. Their perfect memories enable them to effortlessly work with infinite layers of abstraction and write the most clever solutions imaginable to any problem.

And that's why my shitty memory is kind of a super power. Except for the way it makes everything else difficult.

Gotta dumb yourself down to write your best code sometimes.

In this vain this often posted article is fun https://grugbrain.dev/

Are you sure those "smartest developers" would be still smart if they have to maintain someone else abstract code? Easy to memorize layers of abstraction when it follow your own logic and you build it step by step.

Bingo. It's different when you grow up with it.

Still happens. I work in a codebase that is definitely on the upper end of complexity for the industry. ~18 months ago we hired a very smart developer who had one of the more rapid onboardings to our codebase I've seen, he was very productive in under a month. It turned out he was too productive and people were starting to find new layers being added to the code base. A few of the more senior folks had to come in and shut him down.

> shut him down.

Does this mean firing him?

No, it means helping him understand how certain types of contributions he makes are perceived by engineering leadership and giving him access to mentorship that would help him better channel his efforts in ways that would be seen as more productive and something to be rewarded.

If, over time, he continues to introduce more problems than he solves then he might be managed out but I'm hopeful that won't happen.

how do you define smartest developers?

ability to solve Hard Problems rather than create them

...and to explain the solution to humans with the same code that solves the problem on a computer. It rarely happens completely, but it's a good ideal.

I mean clearly the highest IQ people I’ve ever worked with. The smartest people I’ve worked with that are developers.

By outward appearance or some other metric?

Not to get on a soapbox, but I would imagine someone with great pattern recognition would be able to realize this way of doing things is deranged; and ultimately helps them more than the team- On second though: perhaps they are genuinely the smartest people in the room -- prioritizing their image and importance by making others believe they're too stupid to understand what's going on.

I really should go study Gang of Four -- job security demands it.

a lot of it is hubris as well, they just assume that others will be able to understand it because it's easy and intuitive to them, and if others cannot understand it they must not be good developers

If they work with developers of a similar intelligence its not a problem, and even has massive returns

Long ago I read something like...

A debugger needs to be smarter than the person who wrote the code being debugged.

If someone is at their smartest when writing a section of code, they are thus unable to successfully debug problems involving that code.

Indeed; this is called Kernighan's Law:

> Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?

https://en.wikiquote.org/wiki/Brian_Kernighan

There's a similar saying in cryptography: "Anyone can design a lock that's so complex they can't break it".

Both do seem a bit bullshit, though.

I hate to go against the Wisdom of the Ancients here, but there is no point at which you're too dumb to debug your own code. With increasing code cleverness, you may lose the ability to make quick mental jumps when debugging it - but this only forces you to be methodical. With a methodical approach, you will find the bug - the thing you might be missing is not smarts, it's patience.

As for cryptography: digital locks are broken by default. It takes care and smarts to design one that isn't trivially breakable. Trying to design a lock that's too clever for you to understand will almost always yield a lock that's broken just past the boundary of your understanding, so you should be able to break it once you realize it's possible and you put a little effort.

As for physical locks: same as above, plus you can always break it literally, by applying enough physical force.

Being a savant at creating and then solving a particular kind of puzzle is certainly a form of smartness. Figuring out whether it's a form that you want to have involved in your business is another form of smartness.

The principal-agent problem looms large.

Nope, those are not the smartest developers. Those are the cleverest developers.

The best developers solve the hard problems without writing the worst code.

The key is consistency across the organization. Even with a complex (perhaps even over-engineered) standard template, it's nice to be able to open an application/service you've never worked on and have a pretty good idea how to find the key bits.

The worst is a huge monolith that has dozens of styles and favorite patterns sprinkled all over from years of opinionated developers leaving their mark. For anything non-trivial, it might take an hour or two just to get your bearings and enter the mindset of the original developer. Then, you have to make the decision of whether your new function is going to follow that (broken) pattern or add yet another new one.

+1. One of the single best pieces of programming advice I ever received when I was young was my TL saying "I don't like this, it's too clever". I spent a brief period at a FAANG company and IMO the people there are too smart to be good developers.

My coding heuristics now pretty much boil down to:

1) Less code is better code. The only code with no bugs is no code. After you get something working, remove as much code as you can.

2) Simple code is better code. Don't waste time making your code complex for the sake of being efficient until you've used it and proven that it's a bottleneck with a profiler. Otherwise, just do the simplest thing you can think of that will work.

3) Don't think too far ahead unless you specifically have been tasked with it. Otherwise, do what will work now. Refactor when necessary.

Yep. 100% on board with this. The more code and the more complexity, the harder the system is to understand, maintain and extend.

Yeah. I summarize 1+2 as "optimize for reading" personally - none of us is as dumb as we are in the past/future, explain yourself and don't be clever without an explicit need and a way to recover from it later, etc.

This is partly true, I think. But it's more a problem of focus: the interesting part to many smart developers (and developers in general) is the initial solving of a problem. Very clever developers can go much longer without refactoring, since their cognitive capacity is higher. They can get through the initial problem solve in a single go, potentially, and have a complete mess on their hands. This is only bad if they stop there, though (many do). If they then put that cleverness to use _simplifying_ the code, they can produce some of the cleanest and best models for problems that you'll see. There's often no upside to it, though, so many just skip that step. Unfortunately.

I find these arguments going back in circles and concluding nothing.

All teams are different, and a good rule of thumb is to write code the least skilled or experienced person can understand on its own without guidance. Since teams vary so do these limits change. I had teams where it would've been odd to even try use any abstraction, let alone algebraic data types, and teams practicing lots of abstraction and architectural patterns all on top of functional effects. The teams widely different in proficiency of the best and weakest members and so did the codebase patterns.

It's like in sports, different teams demand different approaches because the qualities and proficiencies differ, there are no universal truths but solutions that fit better different teams.

> All teams are different, and a good rule of thumb is to write code the least skilled or experienced person can understand on its own without guidance.

This. Also, if your team develops on framework X and people are hired based on that, stick to the damn patterns used by your framework!

I had once worked with a developer that one day unilateraly decided that the abstractions provided by the framework we used were not good enough. Instead, we should use the abstractions from a "design pattern" described in a shitty blog post they dug up, and "everyone should read". There wasn't enough pushback, so we had ended up with some seriously unmaintainable code.

I write code that I can understand when I'll come back to it after one year or two. Sometimes I don't understand what my old code does and I realize that I failed. It wasn't simple enough.

OP, I think you’d really enjoy the book “A Philosophy of Software Design” by John Ousterhout. It discusses this topic at length. With respect to reducing complexity, you pretty much drew the same conclusion he did.

I came into this thread thinking of recommending this book. Are there any other books like it that discuss managing complexity, maybe not just for software engineering?

In C++ the over-abstraction starts in your first lessons. I know and understand all the arguments, but I will never type this:

    std::cout << "You have " << itemCount << " items of type " << itemType << "." << std::endl;

When I could do this:

    printf("You have %d items of type %s.\n", itemCount, itemType.c_str());

C++, OOP, and whatever crazy async shit is going on in JS ruined the conceptual model of “code” for going on 2 generations of programmers.

We do students a disservice teaching them how to “code” first. And not instead how to solve problems with code

Yeah, that async stuff is brutal.

Great for solving the typical problems encountered in a web browser.

But a real fucked up mental model to bake into an entire industry’s worth of developers

To me this seems overexaggerated. Most people I went to university with already struggled to understand async code, without having done web development before. And for those with web development background it was pretty easy to adjust.

I also don't know any school, college or university that starts their programming courses in a webbrowser. Typical languages in Germany for a first semester algorithms/data structures class are either Java or C++.

By entire industry I mean web developers, which is a cohort that usually has more non-traditional educations and tends to think in a async mental model.

What you explained happens in Germany is pretty par for the course in the US as well for CS degrees. But even then, why the hell are we throwing a book of data structures at students? You don’t give an apprentice framer a nail gun and say “go at it!”. They’ll destroy everything. You give them a hammer and educate them when they start to complain of smashed thumbs

Where do we use it where it isn’t needed? async/await syntax is much nicer than callbacks or promises. Maybe you’re talking about the old ways.

I would go a step further and teach them how to read code before teaching them how to write it

I will never type this:

    printf("You have %d items of type %s.\n", itemCount, itemType.c_str());

When I could do this:

    std::println("You have {} items of type {}.", itemCount, itemType); 

    https://en.cppreference.com/w/cpp/io/print

That's perfect -- as easy to write and to read as printf. I feel like an idiot for not using it all these years.

Edit: Oh, it's in C++23 standard. Only took them 38 years! ;-)

Agreed with you on both counts - cout was an abomination and that std::println looks great (I'd never seen it before, either)

I doubt it's yet implemented in any standard library. But it has been in the fmt namespace for close to a decade: https://fmt.dev/ . I do use it in a C++11-only codebase.

Many languages have string interpolation, which looks more similar to C++'s << than to printf

  puts "You have #{itemCount} items of type #{itemType}."

  print(f"You have {itemCount} items of type {itemType}.")

  console.log(`You have ${itemCount} items of type ${itemType}.`)

I’m just a guy, so take what I say with a grain of salt.

This problem is broadly solved by a combination of domain-driven design (DDD) and functional programming (FP).

Yes, I know you’re tired of hearing about FP and you think FP engineers are all white bearded monad salespeople. I don’t care.

Odds are, if you’re an engineer, you’re not doing purely abstract work: your code has some correspondence to “stuff in the real world”. FP is unencumbered by the class/object model and you can abstract over anything trivially using plain old functions: don’t “choose” your abstractions, use these, and your domain models, to figure out what the correct/real(est) abstractions are. It’s not so much about too much or too little abstractions, it’s about how suitable the abstractions are.

Speaking for myself I think FP is going to have its moment and it will be when average devs can ignore the syntax and theories in the abstract and just solve problems.

Pretty much everything is better with FP and walking a reasonable line when showing benefits _without_ immediately diving into all the esoteric parts wins people.

We need the MVC of FP type of trendy bandwagon. The FP rails.

I am optimistic about Roc and hope Rust also continues to help make FP style programming more commonplace even if I still dont know how to implement a monad.

Re: monad

For my part, the most confusing thing was that I was trying to understand monads by looking at both haskell and category theory.

However, a monad in computer code is a different beast than a monad in category theory (I believe this may be true in every programming language currently, but if you have a counter-example I'd love to hear about it). They look vaguely similar, but aren’t the same thing. Nobody told me this, but it greatly simplified comprehension once I had that epiphany.

A lot of monad explanations just confuse monads and functors too (functors are your burritos / wrapper data types; and functors should probably be understood before monads). IMO seeing monads as being primarily about function composition, kind of like the functions `compose` or `pipe` or `thread_first` or `thread_last`… but in a context that makes use of functors, is the best way to really see the core simplicity of the idea.

I like FP, but I don't think the situation is solved by FP. There's a lot more room for discussion of abstraction and complexity within FP.

My comment said it was “broadly solved” by a combination of FP and DDD, not by FP on its own, and not completely solved either (the problem described is multifaceted and you can find many ways to screw it up regardless of what tools or frameworks you pick up… but certain tools are a bit harder to point at your foot).

Functional is generally accepted as The Best Abstraction imo

Generally accepted… But by whom? If you mean “the software engineering community at large”, I’m not so sure… I’d say from my experience: [*]

- >98% of engineers are mostly familiar with OOP

- about 1% of engineers “write FP code imperatively” without knowing that they’re kind of missing the point

And among the >98%:

- 20% think FP is about using map/filter/reduce (not a bad start, but misses the main ideas)

- 35% are openly hostile to FP for a whole variety of reasons, some better than others but none great

- The rest don’t really know what FP is, and if they’d have to guess they’d say it’s when you use functions.

We no longer have the same performance constraints that computer scientists had to deal with in the very early days of computing, when the theory was being written and the von neumann style of programming began to take hold. It will still take a long time (I’d wager decades) before the findings of academics are accepted in the industry.

[*] These figures are all made up and are presented only as a rough way to communicate my perspective based on my experience… But again I am not particularly qualified to speak to this.

I fear you may be right. It always felt a struggle to use FP in perf constrained contexts but with modern compute making that largely irrelevant there's really no reason to avoid indulging one's self.

It is interesting though, the openly hostile subset you describe I don't really consider as working in SWE as it would be the equiv. of working in a kitchen and saying you reject using knives.

There are certainly specialized areas where FP is inappropriate, maybe even for fundamental reasons (although I am not 100% sure of this: compilers have been improving rapidly, I don't have a great sense of how much further we can theoretically/fundamentally push a c compiler versus a haskell compiler, and the variety of possible type systems adds enough complications to make my head spin).

But --and maybe there's a certain skew in my work experience?-- that's probably not really relevant to the vast majority of the programming that gets done. By my estimation, most professional software engineers spend most of their time working on nonrevolutionary, fairly mundane things, like basic crud apps, web apps, microservices, frontend, backend... While a minority (I think?) work on games, codecs, embedded systems, etc. (That's not to say that performance is never a concern in these areas, of course.)

But dwarfing all of the above, in numbers, are all the people who are not professional software engineers, and who are writing code as an extension of the specific work they are doing in their domain (that's all your python scripts, R scripts, ad hoc sql, lab work, etc). I'll suggest something controversial here: anything people typically think python is good for, [haskell|racket|...] is probably way better at it. That's kind of where I see the most obvious use cases.

In my view anyway, it's a pretty good bet that the step-change in complexity reduction you get with FP (big scary words notwithstanding; I am speaking here of the programs' complexity) would actually have a massive positive impact on the accessibility of programming/sw-engineering to newcomers, as well as on programming/sw-engineering's already-impressive propensity for enabling cross-talk between disciplines. There's soooooooo many low hanging fruits in programming literacy.

One of the best post on FP I read was John Carmack's realisation that FP had a strong place in any code base [0].

It's a really valuable lesson to keep most function as pure and stateless, but transformation of the existing state.

Build your entire codebase around this idea and frankly, abstraction will take care of itself.

[0] http://www.sevangelatos.com/john-carmack-on/

Carmack's ideas did a lot for my code. He has good insights around what to take from different problem domains in order to get a result that gets right to the point and solves a specific category of error. You should never settle for "squishy" code that looks pretty just because: the code must do this, it cannot do that. Write it at the top of the function, and the best code will just go right to the goal without grabbing on to any boilerplate.

That's something I was really missing in my earlier programming years, when all I saw were Java, Python and Ruby bloggers regurgitating OOP and Agile terms and I would try out their idea, marvel at the visible complexity of the abstraction, but then see it fail in a practical scenario. WTF.

No abstractions turns to spaghetti Too many abstractions turns to spaghetti

I don't think the answer is one or the other at all like the blog post is implying. I think his previous conclusion of a threshold makes more sense. Perhaps though they have chosen the wrong abstractions and hence the confusion

>No abstractions turns to spaghetti Too many abstractions turns to spaghetti

That's because abstraction is a myth perpetuated by the likes of Sussman (ie academics that never wrote real code) and in reality the only thing that matters is PSPACE ⊆ EXPTIME.

One could argue there are different degrees of ugly and beautiful spaghetti though.

The thing is, even if its spaghetti, I can follow the individual noodles. I can carefully pull them out of the pile and lay them out all neat and separated.

With "cleverely abstracted" code, I cannot do that. Because there are no noodles any more. There are small pieces of noodles, all in different sizes, spread out over 200 plates, that reside on 140 tables in 40 different rooms, some of which are in another building. And there are now 20 types of sauce, and 14 types of parmesan, but those are not neatly separated, but spread evenly all over everything, and so it all tastes awful. And each individual plate still manages to be spaghetti.

Okay, yes, I may have strechted out the food-analogy a bit.

But that's pretty much how I feel when digging through over-engineered codebases.

But hey, it's serverless and scales to zero.

And some guy banged it out over the weekend, which means he's a rock star.

Nobody else can match (understand) his velocity. He's the best.

"we can never let him go, we'd be lost without him!"

Sounds like we worked at the same place

I usually hear lasagna code if it's got too many codes, and ravioli code if it's too packetized.

First time Im hearing of ravioly but I like the analogy

Laying the spaghetti out neatly is, I think, the correct level of abstraction. Identify each noodle, separate it, put it in the right place and make sure they’re all in the correct order. You don’t want noodles zig zagging and crossing over each other everywhere. You want nice clean lines of noodles you can easily follow from start to end and as few of them as possible.

I loved the “I can follow the individual noodles”. Yes, sometimes even simple spaghetti is better than spagetti tower lassagnia but it’s still too slippery to reason through. But I do get the part about being able to at least grasp it, slippery as it is…

Ad absurdum then you can program in assembly just fine, right? Can I dump you chrome’s assembly output? It’s all trivial, just movs and whatnot.

Yes, it's just the new style of spaghetti, this time with lipstick and a bow tie.

One notable difference is that now the GOTOs are implicit and only in your head, so it's notably even harder to comprehend than the old style of spaghetti.

Also often adds in some code hidden in comments ('annotations'), some squirreled away in complex configuration files, and some other code where what actually gets executed depends on concatenated string variables, and maybe for good measure some places where which code gets executed (and how) implicitly depends on the filepath it is under, and now we have a mess even a debugger can't help with.

It's nice to see more people finally getting disillusioned with these trends.

He does not mention that "linear" code are much more grep-able than fancy pancy code. Even such simple things as error messages as string literals in the code makes your life so much easier.

A degenerate case of grep-able code is e.g. virtual methods for some class and derived types that share name with methods of another class and derived types.

I also distastes code with so much dynamicness that you have no clue where stuff happens. I really like long functions that do many things one thing after another.

This seems like a good place to share a horrific experience I'm currently enduring:

I recently took over a a Django API + React front end project.

What should should have been a basic CRUD application with some calendaring and video calling added turned into a nightmare and costs of over a million dollars for my client.

I'm a Rails and iOS dev mostly, so I have a bias, but here we go:

1) Using GraphQL for a business process CRUD app is a nightmare. Building on Django's ORM with Graphene, then on the frontend with Apollo Client, Queries/Mutations, and React with some Redux mixed in is a nightmare of hard to know database and business logic paths.

2) On the Django side, having their Views and Models (mostly Models) handle business logic, and then constructing a Graphene based GraphQL layer to be consumed by another layer GraphQL Queries/Mutations via React is a mess, for what is essentially a bunch of forms on the frontend

3) Splitting out React Components into Atoms/Molecules/Organisms/Pages/Templates and more has been a nightmare to work with. Seriously - do you really need to build an Atom for a button that you pass the title text too?

The whole project is impossibly fragile.

Say what you will about Rails, but you can build a robust and functioning CRUD app that solves business problems without the need for developers to hack through a jungle of abstractions!

Sorting this all out for client has almost killed me and burned me out!

> Using GraphQL for a business process CRUD app is a nightmare

Could not agree more. GQL is such a big mistake in so many use cases.

Care to elaborate?

In my experience, it’s unnecessary layers of abstraction for benefits that are not realised unless you are at some level of scale where you know it does.

Facebook invented React and GraphQL for good reasons. But many of those reasons don’t apply but the implementation does add development time and code complexity.

In a React and Django Graphene API situation, you have to write GraphQL queries and mutations on both sides.

The idea being you might write a GraphQL query that only retrieves exactly the data you need and nothing more.

Which is great when those optimisations improve customer experience, but for most of the SMEs I work with, they do not make any perceivable difference.

GraphQL is layering on complexity and additional libraries when a good old fashioned REST API does everything you need anyway!

I guess GraphQL is fashionable right now!

"Mit Kanonen auf Spatzen schießen" : shooting sparrows with a cannon.

This is the story of software development, nobody uses a shotgun to get the job done quickly, because afterwards they're out of job.

It's borderline criminal but all the nonsense is about scamming your employer to pay you hefty money for many years to build the Kaiser Wilhelm Gun.

And the "geniuses" disappear right when the project gets to be delivered. Like the past few projects I worked on, the original developers that shall remain unnamed would spit out code like there was no tomorrow, always latest fashion and of course throw one or two badly broken ad-hoc domain specific languages pulled out of their ass. Get paid for two years or so and in 6 months before scheduled release, suddenly "be assigned to an uttermost important project", in fact another two years of jacking off with no responsibility. While their piece of art gets transferred to suckers like me who only need to do the "easy" finishing touches and get it released. If it succeeds, praise is in them, if it fails, it's my fault.

Where do I get one of these jobs where you make every task take longer than it should but get promoted instead of fired?

DM me when you find out ;-)

If you're providing an API that's well defined and constrained in scope (i.e. a typical CRUD API where you're creating/modifying normalized objects in a non-changing object hierarchy), then graphql is usually just a shitload of unnecessary overhead.

Examples that come to mind:

1. An internal API where you own all the clients (you own every query; optimize them.)

2. An API that deals with objects where scopes aren't going to vary (i.e. if your clients are fetching the same 6 basic queries every time, you're not getting benefits out of GQL).

3. APIs that are write/update heavy. Every mutation is the same amount of work (or more) that a REST controller would have to deal with, but you have the overhead of the GQL library on top.

Basically: everything is just JSON on the wire anyway. Stop paying the overhead penalty of GQL unless you have a zillion clients and a massive object graph that's going to be queried in unpredictable ways. Typical REST libraries make backend code management and structure MUCH cleaner than GQL, with much more obvious points for caching, query optimization, etc.

What's the plan, full (parallel) rewrite or refactor?

First order of business:

1) Fix the issues with email notification logic

2) Get them off of the very expensive & complicated AWS stack We've migrated staging and testing to a DO Droplet with DO MySQl/Redis stack. Moving to DO with plenty of headroom is a 10th of the cost of AWS!

3) After that, not sure at this stage. Most likely we will refactor low hanging fruit.

A ground up rewrite would require knowing all of the business logic. We'll just begin cleaning it up and getting it into a manageable state. We don't know what we don't know yet! :-)

I'm in a similar situation to yours, and I've kind of settled on a similar approach: put out the most pressing fires and reassess 3-6 months down the line. At first I had thoughts to refactor but now I just don't want to deal with the complexity, not to mention I have end-users pressing me. I'll have to tolerate the spaghetti, but at least make it produce correct output.

Best of luck with the migration to DO :)

I've had the pleasure to work with an immensely talented developer who would spam unnecessary patterns all over the codebase, to the point where it required several coffees just to go through a single branch of domain logic. Even the IDs of his aggregates were classes.

At the same time, though, I feel very disheartening that at my current company devs ignore what even "aggregate" means, and don't have any kind of clue about patterns and higher concepts in general. I really miss smart and educated people in IT.

>Even the IDs of his aggregates were classes.

People were arguing for such a thing because e.g

if you have 2 guids then you can write code which handles two completely unrelated guids

`student.Id == car.Id`

meanwhile if you have types like StudentId and CarId then you'd have compilation error

While I'm not a fan of this, then the concept is sound.

Yes, this is a philosophical issue that lies within our attempts to model reality in our systems. What defines identity? A dog named Brian is not the same as a man named Brian.

For value objects, surely Leibniz's philosophy applies: if a theory describes two situations as being distinct, and yet also implies that there is no conceivable way, empirically, to tell them apart, then that theory contains some superfluous and arbitrary elements that ought to be removed.

For entities, it's much more complex.

I still feel some kind of way when seeing classes (with methods!) being used for IDs, though.

> still feel some kind of way when seeing classes (with methods!) being used for IDs, though.

Depends on the language. In many languages strings are classes in general, so it makes sense that a new type for a strong is is one as well

I've no clue what you just said :)

Modeling the business domain with types that match it is table stakes. Otherwise you end up with a "stringly-typed" codebase in which everything is just a primitive type.

This and the issues the author of the article is facing comes from the wrong mindset. Your job is not to create the purest, most perfect codebase, but to deliver value to your users. Code is a means to an end.

> I've no clue what you just said :)

This. Before you wax all philosophical, try to understand first what you are arguing for and against. Make sure to read what you wrote at least two times, and that it actually makes any sense.

I have done that in Go and Typescript ("branded types"), it's actually very useful to discriminate ids. It prevents stupid mistakes that are actually hard to spot. Making it a whole class is maybe going too far however, as the costs may outweight the benefits.

nim has distinct types, kotlin has inline classes, scala has type aliases. It is a good feature.

> an immensely talented developer who would spam unnecessary patterns all over the codebase

He may have been talented but unnecessary patterns are the hallmarks of a bad programmer

One thing I have to say about this is that the fact they did this with a bunch of Python code is probably a huge part of why it didn't go well. Abstracted Python code drives me crazy because, in part because of the dynamic and typed nature of Python, the support is so poor for then navigating around it afterwards.

Compare to Java and with single keystrokes I can navigate up the type hierarchy and bring up a full display of every class implementing an interface. With a couple more keystrokes I can rename a function in the base class and update all the implementors, globally across hundreds of thousands of lines of code. And in Eclipse this happens nearly instantly with 100% fidelity (I assume in other IDEs too). I can move around the codebase with so much ease. As soon as I hit Python code like this I'm completely lost.

I see people complaining all the time about IDEs (if you need an IDE then your code is bad) and then also about design patterns and abstraction and I do see a connection in that yes, if you need to scale complexity there is a fundamental aspect of that whereby you need a more structured approach to how you are building things and the underlying tools that are supporting that.

This is true. I recently worked on a large Python application and both discovery and refactoring were a NIGHTMARE. Things that were a complete breeze in Java (or really, any strongly typed language)

When I took a software engineering course, they talked about the benefits of "decoupled code," "dependency injection," and "abstractions," but when it came to an actual tactical explanation of these things, or method of recognizing highly coupled code my education fell woefully short.

Tactically speaking, if you don't abuse the global scope (class methods/singletons are abuses of the global scope), don't have circular dependencies, don't have "intialization" methods called after constructors, and most important of all, don't call non trivial functions in constructors, then your code is probably reasonably abstracted.

More importantly than being abstracted, it is probably absolutely boring and trivial to unit test.

It took us a long time as an industry to recognize that popularity of “design patterns” was due to problems with languages. And the best fix is to improve the language.

Mostly! Pattern matching is far superior to the visitor pattern. When I teach compilers using the Appel book, there's this blip of utter confusion that always shows up around the visitor pattern, which is used to implement either or both of semantic analysis or translation. Literally the visitor pattern is harder to understand than compilers.

Does the industry know?

I've been trying to tell people that for 20 years or so and Whenever I say it, I'm shouted down. Usually by someone who loves Java.

My pet peeve is Java programmers still coding like Java 8 just came out.

Most professionals do not have the power to pick the language, though. Even those who do have to weigh the quality of the language against competing concerns like hiring, tooling, libraries, etc.

"It's difficult to get a man to understand something, when his salary depends on his not understanding it"

Reading understanding here not only as "intellectually grasping" but also "standing under it", as in supporting it with your actions.

The cognitive load increases the furthest away you get from the problem you're trying to solve.

I've worked with some really smart people that couldn't program themselves out of a 24hrs challenge because they went deep in lalaland with patterns up the wazoo and didn't just consider what the program needs to do.

A program is a set of instruction for a computer to do stuff. It takes data in, does stuff to it, and pushes it out. Whether you're doing a UI or an automated landing system, it's all the same. In one case, you're waiting for user input to do stuff with it, on the other you're reading sensors at a high frequency and affecting control inputs as a feedback loop.

Patterns should only be used if they actually solve a problem. E.g. the command pattern in systems that require Undos.

The simplest way to manage complexity in any program is to manage state carefully. Avoid global state, and use as much pure functions as you can. The net effects of this has been evaluated ad-nauseam and the results are clear: easier code to reason about, easier to parallelise, easier to test and less cognitive load.

Over-using patterns and abstractions is throwing cash in a pit. The harder a system becomes to reason about or change, the more it will cost to maintain.

Over-abstraction will result in

   main() { application_here() }

which might look good in reviews, but the metrics won't add up.

>Over-using patterns and abstractions is throwing cash in a pit

It's more like digging a ditch and convincing the company to throw money into it. It's a very lucrative proposition. That's the problem.

> Now I introduce abstractions only if I decide that the future reader could confidently modify the code without looking at the implementations

Going full no true scotsman, but is it really an abstraction if that property does not hold true? Isn't the author essentially arguing against leaky abstractions, which should self-evidently be a bad idea? The problem of course is that developers do not always have the prescience to predict if an abstraction will be leaky or not.

Every abstraction leaks. That's why we need a lot of pruning and a care not to layer too many levels of abstraction.

Reminds me of the worst code-base that I ever worked on...

Start with Java, but don't work in Java directly. Add a layer of meta-programming on top. In this case it was an aspect-weaving system that allowed injecting methods into classes. By itself that would not have been too harmful, but this system had been used at scale, for more than a decade. The result was the emergent effect of aspect weaving on a code-base, which I have not seen before. The entire code-base had been splintered into thousands of small pieces that only existed to be injected into other contexts. It was very strange code.

The aspect-weaving system worked as an extension to the parser, so that all of the code written in it was essentially action rules for the parser. Data was constructed as synthetic or inherited attributes in the parse. The "hairy control flow" that is normally eliminated after the initial parsing phase existed in every pass in every part of the compiler. Essentially parsing never finished, as even late in the code-generation backend we were still tracing the emergent effect of grammar production rules.

Layered on-top of this was a pattern-heavy approach that tried to eliminate all remaining structure, shape and form. Days could be spent digging through scaffolding and layering code to try to find the pieces of functionality that went together to handle a particular construct or feature.

The layers of maintained- and generated- code meant that nobody actually knew how large the system was. I spent an afternoon trying to count it properly and I think my estimate of 1.5m lines was probably the closest that we had come.

I still have nightmares occasionally, but they are becoming less frequent over time.

Otherwise known as “your CS degree has shockingly little to do with this god forsaken login page we’re designing”

I’ve had so many long conversations with early-mid career developers that the stupid code being written will have a shelf life about as long as a head of lettuce. And that the “best” solution is typically the simplest, shortest, and can be written the quickest.

This article made me think of this blog post:

https://mikehadlow.blogspot.com/2012/05/configuration-comple...

I prefer the old version or the pre over-refactored code better too, I can at least understand it.

I've noticed that when control flow is abstracted away at runtime for flexibility of future changes just in case we need to handle other cases in the future, the code starts to become difficult to reason about.

When people extract everything out until you have hundreds of low signal-to-noise files.

I think it's a reaction to team based software delivery, where multiple people need to work on different layers independently and separately, but when you're working solo, a dense codebase you can fit in your head is a lot more enjoyable.

Maybe the size of the codebase determines this.

> I've noticed that when control flow is abstracted away at runtime for flexibility of future changes just in case we need to handle other cases in the future, the code starts to become difficult to reason about.

Control flow is a big thing to watch for. IME an abstraction that's defined in terms of plain functions and values, and lets you use your own control flow, is often a good one; one that requires you to supply a bunch of callbacks and let it take over the control flow is often a bad one.

Starting around the mid 2010s I think, I started seeing needless use of "Design Patterns" all over the place (mostly by CS graduates). Factories, adapters, observers in places where it was just plain overkill and needless extra lines of useless code.

I started referring to it as cargo-cult programming.

It's not cargo cult if you get results (highly paid jobs, long term employment)...

Simple code is like the Zen ideal. Direct seeing, direct apprehension. It beats any abstraction. It's right there.

Like, we could almost have our ideal UI be a dozen lines of code (in some kind of CLIish dealy) , exposed for twiddling.

It would be the ultimate in informativeness, control and flexibility.

Given a domain, code generally changes similarly. For example: You have an app that interfaces with a scale to weigh widgets. At first you support one scale and so you just write code to solve the problem of reading the weight and sending it to the weight processor. After some time, there is a new scale. You now add some abstraction so that they both report a generalized weight message to the weight processor. When you later have to add a third, because you have that generalization, you can tell a new Junior dev "just follow what the code is already doing" and they should have no problem doing just that and add support for the third scale.

That is enough abstraction.

I don't have the large code base experience, so I'm talking a bit out of line.

However, contra OO, I submit that keeping the data orthogonal to the methods is a win.

Restated, I haven't seen much win with objectifying everything and having a huge object inheritance tree. Probably useful in some domain.

Recently looking at botocore. There are no classes for AWS services; just a "database" of JSON documents enumerating "shapes" and "operations", and then some wrapper classes that dynamically instantiate classes where needed.

Feels as though AWS may be on to something, though doubtless their server-side code is non-trivial.

If you havent read it already it may be worth reading why the AWS libs are built that way. Definitely a problem at a certain scale.

And if you havent seen it already you would probably appreciate Project Cambria from Ink and Switch.

https://www.inkandswitch.com/cambria/

Editing to add: and "schema driven development" with tools like Buf

https://buf.build/

> Restated, I haven't seen much win with objectifying everything and having a huge object inheritance tree.

Those aren't very connected, at least in several languages. Using interfaces and composition you can have pretty shallow inheritance overall. Most of my objects are just one level "deep".

> At some point I worked on a Python product written by a person with some good business knowledge, but essentially not a trained developer. They handled our team a bunch of cron-scheduled Python scripts that somehow implemented a complex data processing pipeline and our job was to maintain those. I remember my horror of looking at the code which had zero abstractions and essentially written as a bunch of linear scripts with copy-pasted bits of logic all over the place. Of course, our team has decided to bring some order to that. We have rigorously created abstractions for individual business logic workflows, implementations of those abstractions, factories to instantiate them, and more, and even more on top of that. Fast-forward a couple of months, I wasn’t working on a product for some time and then came back on the team. I looked at the code and I came to realization: the older version better. I had no idea what new code did. Instead of linear logic I had to jump between interfaces, their implementations and a bunch of other abstractions to gather together a complete understanding of the overall implementation. The voice of an experienced developer in my head said: “chosen poorly abstractions are here”. But another one annoyingly replied: “what does it mean to be a poorly chosen abstraction?”.

Unfortunately this is a very commonly observed difference between "software engineers" and "business people who write code". The latter want to get a job done, while the former are often looking to implement engineering doctrine that may not be relevant to the problem at hand.

It cuts both ways. How much overhead is created over time by "just make it work and ship it" mentality? Neither stands as good or bad in itself, both need to be checked.

I'd rather fix the problems with a straightforward but flawed system than unwind a dizzyingly over-engineered one. The intent and implicit requirements are usually clearer with the former.

The fundamental problem, I think, is that software is by definition a way to create abstractions. And there are way too many incentives to complicate these abstractions. And on top of that, these complications have inertia. Once people learn they can gain prestige, job security and self importance by learning and multiplying abstractions, you have a runway process.

Complex systems evolve by brute forcing their problem space during a boom and pruning the non-viable inroads during a bust. That means a lot of agents will be stuck building dead-ends only to get wiped at some point. That's why meta cognition is so important. We are not blind agents, we are a complex system ourselves. We are capable of internalizing giant swats of symbolic representations of system archetypes. We are capable of teleological behavior. Do it people. You can learn computer science and preempt a life of being mired by abstractions that won't lead anywhere.

Sadly greed lead us astray too often.

In real world problems, there are many different constraints, mixed in with legacy code and tests along with many new requirements. It's hard to wrangle through all of those 1000s of lines of code no matter how you slice it, abstractions or not. I think sometimes people just want to vent that if it's hard that there MUST always be a better way. That's not always the case.

It's not gong to get fixed by just thinking about cognitive load. It's a very delicate balance: increasing cohesion and encapsulation, adding robust tests, also adding new features, not spending ungodly amount of time refactoring, being mindful of how easy it is to review etc. Even cognitive load is not the right way to look at it. If the abstractions give you very desirable guarantees in terms of performance and correctness, you need to consider it too.

Design patterns reduce cognitive load, to those who know and understand the design patterns. For those who don't, their cognitive load is increased. That's why so many people so there's more to learning how to program than learning a programming language and its most popular libraries. You need to learn the design patterns too. It'd do you well to also know some anti-patterns. After all, it's useful to know what not to do as well as it is what to do.

Once you've mastered the most useful design patterns applying to your problem space, you can then focus on learning the architectural patterns.

The idea that you don't need design patterns in smaller code bases is so strange to me. Design patterns are not some mechanisation designed by Big Code to get in your way. We call them patterns because they are similar structures that tend to reappear time after time in many codebases. Design patterns just gives them a vocabulary and a consistent(ish) style.

If you're working on smaller code bases and you're not seeing design patterns emerge I would strongly question what's going on (or whether you understand patterns).

There is one thing to have patterns emerging from code and being able to identify them and trying to shoehorn your code into patterns . Obviously the latter is a problem and I think it’s the culprit for the hate design patterns get. From my experince.. I’ve seen some twisted abominations in the name of design patterns in various codebases throughout my career, I think there’s some truth to it. However, I don’t disqualify the use of design patterns, they do have their own merits when used properly.

The problem with abstractions (in code) is in their purpose - "Hide the details". You can create an abstraction that hide the details really well but this also means that it makes it really hard/convoluted to find the details when required. Also, usually you don't have just one abstraction, instead you have layers of abstractions and that makes things more complicated.

With regards to abstractions, the perceived quality of code depends heavily on who will be expected to read it. "Principle of least surprise" isn't universal, it's subjective because you're hiding complexity not actually making things less complex. If an abstraction feels natural to the person reading it, they will find the code easy to read.

Weird take, design patterns take off cognitive load. It's easier to say. Use an observe or strategy pattern or adapter pattern. In most sufficiently large OO code bases, folk end up implementing design patterns sometimes and often poorly, without knowing the names for them.

I work on multiple small multiplayer games. We have shared modules across the front and back end. We split our code into what is game specific, what is library level and what is framework level.

We got the simple stuff right the first time, but it's not until project #3 where we are actually building the framework properly.

A library exposes utilities that an application chooses to use.

A framework constrains the application to a precise system and offers a lot of power within that system.

Frameworks are the most risky to write.

You should always prefer libraries over frameworks until you are certain not only of the problem domain, but the future of the company.

Our first project we really just prioritized features and time to market.

When I was designing the golang based backend for our newest project, I spent almost 2 weeks iterating on the design of the framework. The set of constraints around what an API call would do. I implemented a system for syncing JSON objects between the client and server. A system I had been thinking about for 4 years. It works really well and felt good to implement.

The constraints I created offer us a lot of power. Logs are automatic and consistent. We can run analytics on the diff of every single player interaction. The constraints guide developers into writing performant code.

However there are flaws in the framework. Those flaws could become landmines. Or there are theoretical designs that cannot be represented by the framework, and I have a strong opinion on how to extend the framework to support that. Another dev might implement that new feature in a spaghetti code way. But I can't really justify spending another week iterating on those flaws.

When I was pondering where the line is of how long to iterate on the framework level, I was reminiscing fondly of how simple the first products backend is. Each API call written as a unique snowflake, with all its reads and writes and errors in a straight line, and divided into simple functions with no rules. But then I remember the flaws, the database load is high, there are reads and writes going on inside deep function calls. Reading the same record multiple times in an API call. The lack of consistency in the response structure and how the client would deal with that. The lack of completeness of the logs. The inability for us to comprehensively analyze our players data.

Abstractions are useful. It's hard to get them right. You're better off dumping all the Legos on the floor before you try to sort them.

The cleverest code is, in my experience, the simplest. Anything else isn’t clever, it’s just showing off.

When everyone else says "clever" they mean complex and obscure, not simple. Clever is difficult to understand.

Redefining words to mean something other than standard usage is clever.

Hah! Nice one.

Should have written it in rust, it has ZERO cost abstractions.

Abstraction if necessary but not necessarily abstraction.

Author provides a good example. Linear scripts are a horror to maintain but a project with too many abstractions are a horror to understand.

I'd rather maintain the latter because I just need to understand the code once and figuring out abstraction layers is fun at the cost of my employer's dime.

> the older version better

Better in what aspect? Better to maintain? Better to debug? Better in solving the business needs? If the answer is all of them then congrats you have poorly done refactoring.

Reply on news.ycombinator.com