Software Dark Ages
threedots.tech
threedots.tech
Also, the claim "In the historical Dark Ages, religion put down science" is simplistic. If by "Dark Ages," the author means the time between the fall of Rome in the west and the Carolingian Renaissance in 800, there was no institution maintaining science, but there happened to be institutions maintaining religion. During this time, what scientific knowledge that was preserved in the west was preserved by monks copying manuscripts. If the author means to refer to the whole Medieval period, then many of the most famous scientists were clergy, and many scientific institutions grew out of religious institutions, so again the claim is confusing.
It's unfortunate that the author chose to distract from a good point by using a problematic metaphor.
I have literally almost never come across any metaphor, save for a very very small number of them, that are at all helpful even.
Most metaphors IMO only serve to obscure the reality of things and to draw false parallels between things, and to make it seem like knowledge is being imparted when it is not really.
On top of that there are all of the times that the metaphors themselves may consist of comparing against something where what they are comparing against is not even correct, such as may be the case here.
If I could legally change my middle name to “Enough With the Frickin’ Metaphors Already”, I would almost be inclined to do so, except I don’t want the word “Metaphor” to even be a part of my name because that’s how much I dislike the absolute majority of metaphors.
Yeah, but metaphors are like cockroaches, they spread like wildfire and are really tough to kill.
Here's my dismantling... The Carolingian Renaissance was not a renaissance. Frankia had never really been a part of the ancient civilization of Greece, Rome and "eastern lands." It was a roman outpost, for a time, but they never had urban civilization, widespread literacy, political unity or such. Same for the "Scottish Renaissance" or whatnot. The term makes sense for Italy, but that's about it. Before this period, it's all darkness... apart from an occasional roman flashlight.
In any case, the term "dark ages," at core, just means the absence of historical records.
Muslims in the East were still busy reading and interpreting Aristotle, practicing the most advanced medicine, mathematics, and religious tolerance while petty lords were raping and pillaging from peasants on and off their fiefdoms so much the Clergy had to make sermons and talks about cutting that out.
...seriously? It is that simple to you?
EDIT: Here's what seems to a good dissection of the impact of Greek philosophy in the Middle East: https://old.reddit.com/r/AskHistorians/comments/6stvd6/what_...
It's also true that the Muslim Golden Age and european dark ages overlap, to the extent that you want to think in such terms.
OTOH, I would point out that civilized or barbarous, Muslim or Christian, lords abused their subjects.
yikes!
https://slatestarcodex.com/2017/10/15/were-there-dark-ages/
Previous HN discussion of this article:
The author is just trading one set of hyped buzzwords (microservices/k8s/devops) for another (oop/solid/ddd).
It doesn't help when he claims that his approach is proved by science!!
There's no approach to software development that has been proven by science.
As far as I can tell, the search for the right methodology is part of the problem.
Instead of just writing the most sensible and simple code that would work, you have to adhere to some methodology.
Object Oriented Programming is not going to solve your maintenance problems and development speed. In my experience, it only makes it worse.
As far as I can tell, the author acknowledges that his methodology is not working for a lot of people, but he attributes that to "you're not doing it correctly" which is typical of advocates of OOP/SOLID/etc.
But this same excuse can be said about microservices. So what is the point?
The author spends a lot of time talking about how to talk to stake holders to understand the requirements.
OK. I'm totally behind the developers having complete understanding of what they are working on and what the expected results roughly should be like.
But as far as I can tell, this has nothing to do with domain driven design _per se_.
You can have complete understanding of the project, and implement the project successfully, without ever bothering with DDD.
It’s easy to say „just write simple code”, but how do you do it? It’s not an advice someone can follow. Complex domains have many challenges where having patterns helps.
I guess for some it is simply hard to grasp that one can have good understanding of software development without cramming endless design patterns and methodology acronyms.
It’s like saying „write good software” without any advice how to do it.
Design Patterns are not a catalog of recipes that you can use to put together a functioning system. They are instead an attempt at creating a "common language" of sorts for some common concepts. They don't really teach you anything about good software: they only allow experienced people to communicate well. The authors themselves have stated: "design patterns are descriptive, not prescriptive".
What one has to learn, instead, is the foundational knowledge that was used for building those "patterns and guidelines", and even for other things like SOLID. And dare I say and even functional programming, OOP and procedural programming itself. For example:
• Why you don't want tight coupling (foundation of lots of patterns and architectures)
• How to make program state predictable and why it's important (foundation of both OOP and FP, including encapsulation and immutability. And also of some patterns like CQRS)
• How to design good interfaces/APIs on all levels (functions, classes, modules, libraries, services, programs, companies)
• How to understand the cost of hidden dependencies (not talking about libraries, more like "functions that call other functions" or "classes that depend on other classes". This is Joe Armstrong's banana-gorilla-jungle problem)
• How to make functions/classes/modules that you can change without requiring cascading code changes in the rest of the program (again this has to do with coupling)
• How to make programs predictable in general (the most important business wise)That's quite distinct from design patterns, which are usually taken as gospel that needs to be followed for its own sake.
My karate sensei used to say that while all kyu ranks must follow the katas with extreme precision, the higher dan blackbelts have absorbed the knowledge and can ignore and innovate.
In that sense design patterns are like katas. Training wheels, but not an end-goal.
Write code in a way that everybody can understand it, even managers.
That's it. There's no magic. Code is read MUCH more often than it's written. You have to optimize for it being readable.
This does NOT mean the shortest possible code. I've seen people go to that extreme and it never ends well. But it also doesn't mean every variable has to be called "user_in_a_context_of_private_temporary_login" either.
Truth is, writing understandable and readable code as a skill goes well beyond programming. You will find the same skill in well-articulated presenters, book authors and lawyers.
So just this one advice should be your guiding star: write your code as if you will have a stroke tomorrow and will suffer amnesia and you would still be able to understand your own code very quickly afterwards.
DDD sadly often goes hand in hand with a ton of enterprise Java-like practices that never ended well for anyone applying them. It's one of the collective delusions that's apparently too persistent to disappear by itself.
Resisting the temptation to abstract everything behind factories / config providers / dependency injectors et. al. is a crucial skill in programming and project management. Most people fall to the temptation however.
DDD is absolutely not about factories or dependency injection. It seems like you mix up Java with OOP design patterns and DDD and treat them all like the same thing. They’re not.
I am not mixing those up. Even when I only had 3-4 years of experience I was very keenly aware they are separate things. But 90% of the people I worked with did mix them up and forced their decisions on me.
I know DDD can be applied sparingly and to produce common-sense code that can be easily read by (almost) anyone and it's what I strive to do for years every working day.
What I was saying is: it's an uphill battle against the mob rule of a lot of people who get easily hyped and lack analytical and critical thinking skills and just blindly apply every single enterprise pattern they've read in a book.
To that end I somewhat agree with the article -- but not completely.
They actually think they are doing everything right.
I've been trying to ask for some of the things this suggests, and been told that it's a 'low value use of my time' to have expensive developers talk to cheap service reps.
The entire point of my work is to multiply the value those low cost reps provide. I'm probably deluding myself to hope that will improve their working conditions, but even just eliminating the day to day pain points of their job is more motivating to me than whatever "move this widget 1px to the right cause the design isn't pixel perfect on the CEOs new phone with a strange resolution" type crap I wind up working on.
I see it all the time devs off-roading a relatively simple project to try out the latest tech. It always ends up taking much longer and leaves behind a mountain of technical debt.
That's ok though they'll leave in 6 months to go wreck havoc on another company's code base.
The microservices there were anything but. Not only tightly coupled, but coupled at build time, such that we had a mountain of custom Ruby libraries written by one person to parse and publish Apache Ivy files no matter the underlying build and packaging system, to construct dependency trees and build orders for this massive Jenkins infrastructure that rebuilt the world several times a day. It got even worse, because the development teams were building fat jars, then the pipeline team was putting those in fat containers, then we were shipping the containers, except across an air gap. So we're generating gigs of data every hour that some poor souls have to physically burn to DVDs and sit around waiting for hours while virus scanners approve it to go into the runtime system.
But I don't think the issue was so much a dark ages problem that nobody understood what we were doing. It was just myopia. Nobody understood the impact their decisions had on downstream and upstream teams. Architects were pitching great ideas to customers, managers were making tremendous promises, and neither had any idea that the as-built system came nowhere near matching the glorious vision they drew up on a whiteboard. And the developers didn't know the whiteboard vision even existed. They just saw tiny chunks of single-sentence Jira tickets with no context and no idea how they fit into the larger system.
It's the Buddhist koan thing about 10 people looking at an elephant but nobody seeing an elephant.
This is the answer. Most of us are in the weeds throwing darts.
I've experienced all of these strategic patterns mentioned, and it's still been a massive clusterf*ck of failure.
DDD attempts to solve the right problems, but so does everything else, and adding process when there's an incentive and cultural misalignment doesn't actually help.
Every success (including major ones, going from so risky the VP doesn't even want to attempt it to best thing ever delivered by the department style of things) I've seen has been due to two things. First, dev believing that their responsibility was to understand and solve a particular problem, and that they were empowered to actually do that. Second, product believing that what they'll be evaluated on at the end of the day is if a valuable solution is provided. Both of these are predicated on upper management creating incentives for doing them, rather than all the other BS that upper management can end up prioritizing instead (i.e., status reports, documentation, checklists, roadmaps, etc).
With those two things in place, you'll figure out a process that works. You want to use DDD? Fine. You don't want to? You don't need it. You can have all the same information you'll get via Event Storming and etc collected and shared verbally via tribal knowledge (ideally not just this, but I've done it successfully, if with some obvious risk), or written in wikis, or whatever, and be successful.
Without those two things? The devs will be bored as product talks at them rather than to them as they move stickies around, the stories will still reflect nothing of value, the actual work will be extremely low quality and will be constantly in need of rework (both due to quality and due to actual value), there will be constant asks for documentation that no one will read, and constant meetings to prepare for and explain status.
> Some engineers tell me they are “just engineers”. They don’t care too much about who uses their software or why. They are just implementing, say, JIRA tasks – building services with some bytes on the input and some bytes on the output.
I think this might be one reason why I didn't stay in my previous job as an engineer at a big tech company. I do care about who's using my software and why, especially since I work in an area (accessibility) that's all about the human factors.
But now that I'm a cofounder and one of only two developers at a tiny company, where I have the power that I wanted to shape the whole user experience, I find that I too often get side-tracked trying to make the technically best decision on some tactical thing, e.g. choosing the best distro for a container, as if I were specializing in that area, when I should just quickly choose something popular and good enough so I can stay focused on the big picture. As is so often the case, I guess I'm trying to have it both ways.
_Now_, we choose off-the-shelf X because it will get us started and we have 99 other things to bootstrap.
_Next_, we throw the original away and plug in better tech Y.
_Later_, when resources are not an issue, we’ll use Kubern... tech Z. :)
"Later" can turn out to be functionally equivalent to "No" and still be as valuable as "No" can be.
Moving carefully can be valuable too, of course.
Engineering isn't about coding, coding is simply a tool to achieve results. If your organization managed to silo it's engineers into ticket-coder orgs, there's something wrong with your company.
There is nothing a microservice does for maintainability that an interface and modularity won’t do.
It only makes sense if there are specific resource intensive things that need to scale separately, or if you have more than one language or runtime.
It’s probably something pushed by cloud providers to increase lock in and use more resources, since adding self hosting capability to a microservice based system that is dependent on Kubernetes is going to be that much harder.
Technically I agree. It doesn’t really do anything that decent interfaces and loose coupling do not.
Libraries help split your project for teams, as do SOA. Microservices are just extraneous splits added into it.
What microservices help with is making data-based native applications and those single page applications for the web. But the usual patterns people push around only make those tasks harder. At this point I believe the entire microservices knowledge base is bullshit.
Reliability. There is a chance that one microservice crash looping does not bring the whole system down. This would not be an advantage if languages were better at isolating failures but they aren't.
Independent release cycles. It is good to be able to upgrade only one service. For example if something goes wrong, you roll back only one service. You also have a smaller code base to debug.
Runaway resource consumption. A memory leak or a logs explosion only affects one service. This arguably is a version of "scales independently".
Above stuff is from my experience at Google. YMMV. I can imagine that a badly designed microservice architecture does not bring these benefits.
Google needs microservices, their codebase is many thousands times too big to deploy as a single unit. They also have the money to focus on reliability over velocity. The startup with 2 developers doesn't need microservices, and the things you are talking about aren't things that a startup should think about before they have any users. Yet they still often goes with microservices because that is what the big boys do. It is those cases we talk about here.
Also, microservices are much better at isolating faults than any traditional langauge is, even for pretty systemic faults such as memory leaks - if each service is running in some container, storing most state in a DB instead of memory, users may not even notice when it crashes with an OOM, something no runtime I know could realistically handle (unless you do gargantuan work to manage memory explicitly for this goal).
> It’s probably something pushed by cloud providers to increase lock in and use more resources, since adding self hosting capability to a microservice based system that is dependent on Kubernetes is going to be that much harder.
The whole point of Kubernetes is the ease of switching between clouds or cloud and self hosting. As long as your application only depends on Kubernetes abstractions, the cost of moving from EKS to AKS or to Kubernetes on bare metal is going to be relatively small - probably smaller than most deployment options that can handle a similar scale and reliability.
A good example - I bet modern graphics cards and simple what that does to neural nets are going to basically erase a lot of former truths about text search, image interpretation and when it is appropriate to use either. And another - rapid upticks in internet/mobile availability reshaped what was true about the web 10 years ago.
It is hard to build well & to last in such shifting environment. Low quality, fast solutions have an edge. In time the wheel will turn. There will be a sudden turning point where spending 5 years on really ironing out bugs and performance tuning starts to make a big difference.
In short, any problems have nothing to do with modelling techniques or the approach taken by software practitioners.
I rather don't want hardware innovation to slow down or stop.
It depends a lot on the domain on which you are working on. In places where I worked, such big changes could be encapsulated and separated from the domain logic (by using Clean Architecture, for example).
This is where modelling techniques are useful - with proper exploration you can create proper boundaries that will save you from such big changes. The only thing that is constant is change. It's all about being prepared for that.
Take for example hard drives. A filesystem optimized for a spin drive will take into account that the needle has to move a physical distance and arrange files to minimize the movement of the needle.
All that goes out the door when you have an ssd which has a O(1) access to any part of the dardrive
My one experience with DDD and microservices was more the reverse; DDD was applied through code patterns but had no organizational/proces adoption. This caused bloated over-engineered microservices where simple things took way too much code and time to figure out.
Sounds like DDD is just another set of buzzwords to learn but I like the warning that you should use common language when communicating across the developer-user membrane.
Ina nutshell, that’s the problem with almost all software development methodology — we don’t/can’t frame any of them in a falsifiable manner, so it’s difficult for the field to progress, even over decades. Bad ideas keep hanging around, and can’t be filtered from good ones. One can always keep playing “No true Scotsman” games on anybody’s experience, so most discussions end up devolving into froth.
Evans - Domain Driven Design
I'm actually a big fan of modular, properly implemented monoliths. In the first blog article I was even showing that it can be actually an implementation detail if an application is monolith or microservice: https://threedots.tech/post/microservices-or-monolith-its-de...
It isn't inherent to microservices, but it is common enough that many people will work on a distributed monolith.
> It’s a key for achieving proper services separation. If you need to touch half of the system to implement and test new functionality, your separation is wrong.
The advantage to micro services is that you can develop, test and deploy/release them independent of other components in the system. But there are plenty of other kinds of dependencies that can exist between components and if you don't don't manage them somehow, your system will drift toward a "ball of mud" where everything is dependent on everything else and any change is cross-cutting and difficult. That's true whether you have micro services or not.
If I have a monolith where each operation synchronously delegates to other modules within the same process, then this system may be tightly coupled. However, even if it is, API calls are fast and predictable.
If I rewrite the same monolith into microservices, where the same API calls are replaced with HTTP requests to the other services, my system is still tightly coupled. Each API call now incurs xx ms of latency and risk of transient failure. I could deploy each module independently, but doing so would cause other modules to fail. Arghh! Arguably the degree of coupling hasn't changed here, but the implications are that much more severe.
Loosely coupled microservices usually use some highly available message queue or streaming system so services can operate independently. You can still be tightly coupled in this architecture though. If you issue a message, then immediately wait for a response message, you're in pretty much exactly the same situation as above.
Actually loosely coupled services usually issue and consume messages separately. In this case, if one service goes down a backlog builds, but the other service can still push messages on to it.
Some loosely coupled microservice architectures (Starling Bank comes to mind) do use synchronous HTTP messaging. Sometimes you want the services themselves to ensure delivery of messages.
With separately versioned and deployed services, a developer can't overwrite some global variable because they are in a rush and it's the quick and dirty solution. They may need to ask the owning team for an OK to introduce a new endpoint, run the design by them, code review, etc. This tends to lead to better designs. Overall it can still lead to complicated system but I think the extra guardrails are a net benefit.
Also if you want to stop the other team from abusing your code then you can easily fix that. Most languages lets you deliver code in an interface that they can't easily break either, try that before microservices.
The defining characteristic of the dark ages is a lack of historical records, and a general idea of stagnation. A period we could nearly seamlessly cut out of history.
It is certainly not the case today, a lot is being recorded, we do a lot of things, so if we have to define a software dark age, when would it be?
I'd go with 2000 (dot com bubble burst) to 2007 (first iPhone). For the end date, the iPhone is not that important, but it marks the rise of "big data", smartphones, and mobile internet.
The problem with Blind is it attracts the former group almost exclusively. I'm happy at my job, downloaded Blind once to see the content, and deleted it almost immediately. I have no interest in reading toxic posts by anonymous folks struggling at my company.
The truth is most likely in the middle, as usual.
All else is noise.