Building Domain Driven Microservices
medium.com
medium.com
I've only seen bad implementations out there (tens of products), and have yet to encounter one that looks reasonable. If your average developer has trouble implementing something sane when the only problem is splitting data and functions locally into different components, why in the world would you assume that putting in a network layer, exponentially increasing the failure rates, loosing good stack traces, making refactorings that have to move data from one part of the architecture to another a pain in the butt (because requirements never change right?), etc... would be a good thing?
The horrors I've seen... oh boy, the horrors.
The people I've heard praising microservice architectures so far fall into two categories: (1) Ex developers that moved into management and read a few articles/bought a book on the subject and think it's a great idea (but have no actual practical experience) (2) developers that have just started working on a new one and are still in the honeymoon phase (mind you, not developers that have started working on an __existing__ one that is a few years old, those figure out the mess they are in pretty quickly).
You can screw up not-so-micro services too, but it's far easier to screw up microservice architectures and they are far, far harder to fix down the line when you figure out how you messed up (or re-architect when that thing that the PM told you was never going to be a use case is going to become a use case (after they tell you they are sorry of course)).
The issues with micro services are just so unbelievably large that it'd take too long to enumerate them here, but for me the largest issue is you are putting arbirary network partitions into your application and you CANNOT possibly know when these network partitions will suddenly make an operation you need to be atomic impossible (or at best eventually consistent).
In general it is excruciatingly difficult to iterate on micro services as well.
Then there is deployment, service mesh, k8s etc. etc. that you have to wrangle with, all because people refused to make bounded contexts in their application without making them into different services. It is much simpler to just design your monolith into separate contexts than it is to introduce all the error handling you need for network partitions.
I personally don't understand why Elixir isn't more loved by these micro services zealots - it allows you to get extremely far without adding network partitions and then allows you to switch to message passing when you need and then networked message passing all extremely smoothly as it makes sense for your app.
Even there, the "clear" separation of concerns means that it becomes difficult to do some things that seem like straightforward business goals. You'd like to bounce emails to a user's aliases if they are over quota but allow messages to that user from support. Unfortunately the milter and the alias resolution hooks are different services and don't get to communicate, unless you use some other functionality... I'm a bit hazy on the specifics, but I can remember needing to bend over backwards sometimes to satisfy things because you couldn't chain additional information through. This ordinarily comes up quite a bit when people compare it to exim, an SMTP monolith where you are more easily able to use information from one service in another.
I think postfix is well built, but the architectural choice has consequences and it isn't a silver bullet.
I only have a problem with the label "microservice". A service is an independent product, that's all. "micro" says absolutely nothing about the size of that product. It's like "serverless", it doesn't even explain what it actually is thus quickly become a marketing buzzword.
The architecture in the article here seems to be driven by a certain type of OO data/interface-modelling:
In school/courses/tutorials/books we are initially taught a type of object oriented data-modelling, which is at the level of: objects in the world are objects in our programs. A person is a Person class, a shopping cart is a ShoppingCart class. You can put stuff into the ShoppingCart, it belongs to a Person and so on.
I call this "Kindergarten-OO": It puts labels on things in the world and then models software interactions 1:1 based on that, as if software is a simulation of the world.
This conflates all kinds of things and the actual engineering and user-interaction requirements get smeared all over your codebase.
Now this kind of Microservice design looks kind of like that but lifted up to a network of computers.
You end up with suggestions like: "Design applications around eventual consistency" (article). Why? Because you didn't design your architecture around timing processes, data-structures/-flow and interactions, but around this made up simulation with ~N^2 communication channels.
A process that takes a longish time and maybe doesn't need to be real-time consistent: Analytics. Let that be eventually consistent, close the feedback loop each Friday and enrich your data which then informs the rendering (ordering, emphasis of products and so on) on your front-end.
An architecture around interactions, flow, data-processing, security/auth and so on looks much more like a decision tree, state-machine or something like that, rather than like a zettelkasten graph.
What is key is to have a scalable time series db to monitor the interactions of the components and services (and microservices) that comprise your platform. Measure the KPIs around hardware, OS, software and services interactions. Monitor and alert on them, via real-time dashboards.
1. You can't run join queries across domains
2. You can't migrate your schema across domains
3. You will need to sync and map a db with domain dbs for reporting
4. You need to write api/client server for every domain and maintain this with db schema changes
If you want to expose extra features or change the api design, you version your api.
Reporting is a problem, but most large business have some kind of datalake/datawarehouse where all of it comes together anyway these days.
But you are still coupled. If you have to version your API that means it is relied on by something else and in order to use that extra feature still requires both (or more) services to all be on the same page.
Microservices let you deploy and maintain at different times (if that is appealing to you) but in order to realize a new business feature you are still coupled.
Now you have x tests for x versions as well, make sure the old version remains old and the new feature works in the new version.
If you version it though, services that don't care can cary on regardless.
Microservices are actually often driven by business requirements. According to the Myhtical Man Month communication channels follow n(n − 1) / 2. So a team of 4 developers has 6 possible communication channels, a team of 5 already 10. This eincreases sharply as team sizes increase and is one of the reasons big enterprise projects suffer compare to startups.
The business benefit of Microservices (done right) is that you build islands of communication, that communicate using only one outside channel. Now communication increases less sharply as teams are separated by their boundaries.
According to the guardian the team size and structure is one of the secret for Amazons success. https://www.theguardian.com/technology/2018/apr/24/the-two-p...
For smaller teams the communication benefit will often not outweigh the additional complexity.
Also Microservices don't sync on the datastore, these are (in theory) not shared.
So yes Microservices can be emulated in Monolithic architectures, as can Monolithic Microservices be built. Usually you end up with the worst of both worlds. The complexity of Microservices with the speed of Monoliths.
The way to succeed is to do fewer things but make them count. Except that looks awful on your annual review, so the real way to succeed is to Defect. Be seen doing as much as you can and get out before they figure out you consumed all of the oxygen in the room.
https://web.stanford.edu/~ouster/cgi-bin/cs190-spring16/lect...
Obviating communication costs within teams is one thing, but what of the overhead and costs of communication for the whole system? Eg. how do you avoid the cost of deploying a new team that ends up deploying a duplicate microservice?
You may also explicitly establish an architecture team that has oversight and tries to ensure everyone's aligned on direction. But mostly if you at least have some shared place where you explicitly communicate the creation of new services or design development then it allows you to avoid some of the issues you're talking about. The other thing is having a way to discover APIs and services so you can go check if something exists before building it. In many cases teams do still build a variant themselves because they need something slightly different and thats ok if they're willing to support it.
It’s not perfect, but it’s a lot more efficient than the previous state of every individual developer needing awareness of what every other individual developer is working on at any given moment.
1) So which "team" then takes ownership of the entire katamari ball of sprawling microservices and understands the interconnectnedness and nuances of overlaps and competition between each microservice?
2) If the concept of technical debt at yesterday's monolithic architectures is biting us now, what does the future technical debt of microservices hold?
2. As you scale, sometimes its easier to create a new service than it is to go modify that old service. When a company scales fast or employees churn its either rewrite that service or leave it where it is and build something new. This can lead to something that looks a lot more organic than computational. I have only seen the likes of Google handle deprecation well, to the dislike of most people external but survival of the fittest is harsh at best.
That's the point. The overhead is high but the other options to employ extra developers bring even more overhead. And the company wants extra developers because the faster (or more parallel) development still has a good ROI.
If you want low overhead, a small team is the way to go.
For example a change in one library might cause an unexpected degradation in an unrelated application. The communication at that point is strained as there are now multiple teams firefighting and playing the blame game. Or a slight change in one library caused deployments to fail for everyone. In a perfect world with perfect engineering practices this wouldn't happen but as many have pointed out, development is a reflection of the business, and the business is not perfect.
Having mini-services, so even just taking a monolith and separating it out into manageable chunks, reduces the communication problem immensely. Doesn't do away with it completely, but goes a long long way!
It is specially "funny" if the microservices are developed by several contractors.
Recent fun; a bunch of microservices written by the same contractor were killing the shared db (the 'state' of all microservices); so while most were running fine, some were hitting the db hard and costing the company quite a lot of money until it was fixed. No-one felt it was their responsibility to test & fix because it was an external company and they 'delivered' according to spec. There are many ways to do this better, but it happens quite a lot (also with monoliths ofcourse but there I find it faster/easier to diagnose + fix (depending on the size; I guess there is a point somewhere in LoC where microservices start to shine).
Then you get into deployment orchestration. Not every org can handle pure CI/CD. Even if they could they'd have to all agree to it.
This is all theoretical to some degree, because I have seen teams deploy Microservices in banks during the deployment windows and I have had people explclaim they have ONE Microservice.
So you can definitely do it, but Microservices have other advantages, as in how easy can you then scale the components, how resource hungry are individual services and so on.
Team A can finish their part of joint effort before Team B and get deployed on Monday, then Team B finishes their effort and gets deployed on Tuesday: this looks the same in both microservices or monoliths. You can have blue green deployments with monoliths with zero downtime.
Why does it look the same? Because you have to be careful with how you modify the interfaces in any case. If team A is requiring a new field from Team B to be not null immediately, then the Monday deployment breaks in either case. If team A uses a feature toggle or just does the old thing when the new field is null, that has nothing to do with microservices vs monoliths. If you are using a monorepo in both cases the synchronization story is the same: master needs to build and run without error.
The real distinction is what the Monday and Tuesday deployments look like in either case. With monoliths, the Monday deployment and the Tuesday deployment look identical: the whole monolith goes up and down. With Microservices, team A can be responsible for their deployment on Monday and it can be completely distinct from what team B does on Tuesday. If team A is team B, then the gain was a minor allowance in the deployments being different which might be a net loss.
Actually, when team A and team B are in lock step, then the monolith allows changes to be deployed that would otherwise be considered breaking if deployed in a microservice architecture. However, a lot of monoliths will still be clustered and might talk to each other across the network, killing this advantage.
Since then we have seen plenty of microservices solutions come and go, what stays is the complexity of dealing with distributed programming.
Anyone that isn't able to write modular code using the language features is going to write network spaghetti code, with the added benefit to debug network packets and connection issues on top of everything else.
This is the most important bit. To me microservices is just another way to arrange your system into module, not much different in concept to modules and packages. Regardless of whether you’re dealing with monolithic or distributed system, if your abstractions are poor, your work and your system will suffer.
Agreeing somewhat, but not totally ; monoliths are much easier to test and stabilise. All bigger microservices projects I know (personally, not what I read from Uber etc) have tremendous overheads compared to their monolithic counterparts. Some of them are really nicely done, but there still is a lot of overhead in support (usually the services are written by different people (and possible different languages/environments, for instance .net core, .net framework (for Windows specific services) + Typescript/node), so other people need quite a large ramp up to get into them), networking, monitoring, etc. As far as I have seen, it needs a larger team.
And it is very different from having you modular built monolith in your IDE for a spin, than doing a little (local/dev) testrun of 25 microservices + the one you are working on. I can feel the benefits but I have yet to encounter them in real life; so far I have seen (smaller; ~100m/year rev) companies going back to monolith and most of the stories I read are from very large companies with massive development+deployment teams where you actually can have 3+ people per microservice.
If you are working with 3-4 people managing 20+ (changing, forming a complex system), I don't really seen it happen (but would love to see practical examples of that) while monoliths of the same complexity/functionality (literally; I worked on a few 'just because scaling!' rebuilds from monoliths->MS(->back)) have no issues at all with that.
Hypothetically, I should be able to take a well-designed monolith and “deconstruct” it into a distributed system by deploying modules individually and replacing function calls with network calls. Going the opposite direction, I should be able to do the reverse with a well-structured distributed system.
This pattern of modularisation exists across the software stack; it goes as far as the digital logic level with lumped-element abstraction. In general, we trade-off some benefits with a more complex messaging mechanism, akin to threads sharing the same memory space VS processes communicating through sockets or files. In general these are all technical details that help us accomplish what we need our system to do, but pinning the right abstractions helps us actually reason about our system.
Languages (libraries) should support this more because people are doing it. I don't want to be bothered by this complexity etc for most of the project; I will architect it properly but the language features should help me in this case.
Some new languages/envs are exploring this, like [0] and Erlang ofcourse is an example of naturally working (or having to work) like that.
In many other environments we work it's mostly annoying; very possible but a chore instead of the language/environment helping me. We roll out quite large deployments anyway; given the choice I rather would see a nicely architected monolith.
I could pick more recent examples, just wanted to make a point how old the concept goes.
You can have that with or without microservices. You can screw this up with or without microservices.
The main factors for going for microservices are technical but, as always, must be understood.
One advantage that I do see is the ability to deploy components individually, but I see more downsides than benefits overall. I hope that consensus is becoming "start with a (modular, well designed) monolith and scale out the parts as needed".
The main benefit is organizational and lies in having a small and very measurable interface between services.
Crucially, the team the develops the service runs it and provides 24/7 on-call for it. This also include taking decisions on how much to spend on hardware VS optimization as the team is responsible for the overall performance.
More importantly, the team is now responsible for service outages.
This is very different from having a team write many microservices. It's especially bad when a team writes a bunch of microservices (the so-called distributed monolith) and somebody else has to run them.
Sadly the article does not focus on this aspect.
As a side note, once developers realize they get paged out of bed by their own code... it's amusing to see how the cool framework of the month is not cool anymore.
There is a formal process to investigate and correct errors after a non-trivial incident that requires collecting evidence and then discuss what happen, when, why and who did what with the whole team.
Then people ask the question "why was this not prevented or forseen?" and keep going backward until you exit the technical realm and look into people's choices.
The root causes could be, for example, that it was a management decision to prioritize something else over availability concerns... and the engineers are off the hook.
Sometimes one engineer was overworked and tired and did an error in good faith that is too difficult to prevent and avoid, and that's also acceptable.
Sometimes that the whole team ignored an availability issue in the product architecture, and that's bad.
And so on... Needless to say the outcome can impact people's performance review.
The realities are that going from monolith to microservices is a huge investment, and only large engineering teams should do this migration.
The reason is that once you go microservices You instantly require an immediate investment in devops which is something you didn’t need as much before with a monolith. Oncall rotations and pager duty schedules become imperative because services will go down. Also monitoring is another investment that you immediately need otherwise you’re completely blind.
The article talks about things that are obvious, like designing the apis etc, but those are the same regardless of if it’s a module within a monolith or a microservice. All the characteristics they list at the beginning is no different than what a team that owns a module in a monolith would require as well. It feels like this article isn’t very useful because it regurgitates concepts from regular enterprise development and packages as new for microservices when it’s not.
What the article lacks is the difference in architecting for a real microservice environment.
One of the biggest missed issues is never mentioning dependencies. Sure it’s great to talk about how to model services with eventual consistency blah blah. But if you already have a large engineering team, which you must have otherwise why have microservices, then there will be a dependency between services and to believe otherwise is crazy. You will probably have one or two services that when they go down, your entire company is down. It’s imperative that every service is architected with that in mind and what happens when it’s down. Hand waving “eventual consistency” won’t cut it in the real world, it’s the biggest problem that each team needs to focus on early otherwise it’s a ticking time bomb.
I disagree. The post is a good summary of all the key features and main concepts of a microservices architecture developed with domain-driven design. It does what it says on the tin, and you don't have to read three books to get there.
> It sounds like they haven’t actually worked in the trenches supporting and coding real microservices at scale.
I have no idea where you got that idea. It certainly wasn't from the blog post, though.
> The realities are that going from monolith to microservices is a huge investment, and only large engineering teams should do this migration.
Your rant makes no sense at all. You're just mindlessly repeating cliches about issues that have absolutely nothing to do with the subject. I mean, the blog post is about microservices software architecture with domain-driven design and clean code, but here you are whining about how you believe refactoring projects designed following an entirely different architectural style is hard.
If they want to write something new then talk about the realities of how microservice development is different than regular enterprise software development. Not regurgitating the same old concepts and pretending it’s eye opening and new.
Again, your rant makes no sense at all. The blog post talks about a specific distributed system architecture style whose services are designed following a specific architecture style.
And here you are, angry at someone because... Because you believe the author has ulterior motivations?
> If they want to write something new then talk about
Sorry, but you have no authority to tell others what they should or should not write.
The author of this post contributed something to the public that's interesting and has value. Meanwhile what have you done?
Plenty of books to read, videos to watch. Learn/listen before you speak.
> You will probably have one or two services that when they go down, your entire company is down
If you have microservices like that then your macro architecture is bad because one of the key points of an SOA is to limit the impact of failures. Probably you are using a lot of synchronous communication like REST because it was "easier" and now you have to pay the price. You chose consistency (which can be a valid choice) and now you complain about lack of availability when any undergrad knows you can not possibly have both (at a scale that requires partitioning). Using synchronous communication everywhere between microservices leads to a "distributed monolith" where you have to pay most of the costs of a SOA but get few of the benefits.
The one correct thing you said is that a microservice architecture is indeed a huge investment that requires sophisticated devops processes in order not to drown in operational tasks. Microservices are a good choice when you need to scale either in amount of users or developers beyond the capabilities of a monolith (i.e. you need an application that is partition tolerant). At that point you can chose between consistency + easier development or availability. No matter what choice you make, it will be painful but it can't be avoided.
Except "REST APIs" (what microservices expose almost always) don't reveal anything most of the time, wanting to hide internal state behind a beautiful and uniform facade essentially representing a two-tier model. For example, orders and order line items can't be added/deleted at will, but depending on whether the order has been processed or not, should expose a "cancel" action (with partial refund/with penalty for cancelling), etc. Sure you can hide that behind a DELETE method, or "reify" a "cancelled" status, but why?
> Services are resilient to failures.
In the case of "REST APIs", this means manually and labouriously attempting to reconcile state across multiple services in the client.
Compare this with state changes though ordinary RPC, which can expose a meaningful state model, give contextual diagnostics/exceptions, and can have transaction context flowing with RPC calls.
It doesn't. You're confusing the persistence model with the domain model. Your persistence model is dictated by your need to persist your data while meeting the requirements of your domain model. Your domain model does not care about your persistence requirements. Your persistence model is bounded to your persistence service and should not leak out, while your domain model is ubiquitous to your system.
Let's put it this way: with a Clean Architecture you are able to add/replace any persistence service you wish, and that has no impact on any other service.
However I direct your attention to the fact that Django docs make it very clear that to them in general each model maps to a single database table. That's quite Spartan even for an ORM, and obviously the point of a domain model is to model a problem domain, not to model the database.
Main lesson is that you should not use it for everything (yes, the books say it, but it's easy to overdo it in the beginning). Don't do DDD for the CRUD parts of your app. It works extremely well for business use cases, but it's very bad at CRUD. A mix of both really hits the sweep spot though.
No microservices though, just a monolith. Imo most businesses shouldn't need microservices, only the ones that really need to scale. We definitely don't have anything even close to that.
Personally, even following all the guidelines laid out by the famous book and other ancillary documentation and articles, such as this one, the whole thing can fall apart quickly when you have a business requirement that doesn't match your technical architecture. You will at some point have to cross-contexts and communicate between domains. In Martin Fowler's article, this is literally hand-waved. I think there's maybe a few sentences on it?
Here, the suggestion is to use 'event storming' to make sure you design your services correctly from the start. Okay, what about if your business pivots your initial assumptions are wrong. Our team has adopted an action approach, but it goes by different names (channels, user actions, business actions) which model a single action and compose sub-actions from different domains, usually in a single transaction. This seems to work, but now we're operating in some non-aggregate space that couples all the domains.
So basically, the most difficult to manage and complex scenarios, where you model your cross-context flows, are never described beyond, use async (not always possible) and re-organize your services (also, not always possible). Would love to hear if anyone else has any experience here, or maybe I'm missing some fundamental part of DDD.
Most materials present hexagonal architecture in a very one-sided way, however. They don't discuss the downsides.
For example, every connection point between components requires an abstract API layer. This is a significant overhead compared to just having components call into eachother. In the long run, this is worth the effort. In the short term, there is no benefit.
Additionally, while it is true that the APIs can be flexibly hooked up to any implementation, there is still plenty of room for bad assumptions to creep into those APIs. When you have many adapters for a given port, those assumptions are entrenched, and are difficult to change.
In essence, DDD forces you to have conversations about separation of concerns, and then being careful to stick to them. Microservices force that, but you can get the same effect with well-thought out namespaces in a monolith.
In particular it now requires software engineers to understand principles and design patterns for distributed systems that when I was first starting out were relegated to theory and university papers. Can you guarantee ordering? Can you guarantee exactly once? How do you get transactional semantics? How do you handle cascading failures? It's all a bit overwhelming.
I have also seen however that monoliths at some point lock up an organization. The key measures of software delivery performance are deploy frequency, lead time from commit to running in production, mean time to recovery and change failure rate. Monoliths impact all of these when you engineering org starts scaling past a handful of teams. A monolith also becomes a single point of failure. One person in one team checks in bad code, and the whole system falls down. Microservices can give you resilience. You can also scale microservices independently - a CPU intensive service can get a bunch of pods, a memory intensive service gets a beefier memory allocation, and other services can run idle most of the time or even be deployed as FAAS functions that almost never run.
So I agree it's not a technical solution as much as it is about scaling an organization. At WeWork the team shrank so much that we began consolidating small services back into a single service because the operational overhead of having separate services didn't make sense any more when a single team was responsible for all of them.
I do think choosing to split out a new service is a big decision to make. You have to really think through the cost and benefits. You also need to make sure you have the right tools in place so you can detect and recover quickly.
Things like log aggregation, SLOs defined, measured and alerted on, canary deploys, progressive delivery, etc. Without that you're running completely blind.
I also think you want to be thoughtful about keeping your coupling low. If each service depends on twenty other services then in my mind you now just have a distributed monolith.
I don’t advocate DDD/microservices but if you absolutely must go down this road, Kafka Streams really neatly answers all these questions. Ben Stopford’s book[0] and blog posts[1] are really great if you want to know more.
[0] http://www.benstopford.com/2018/04/27/book-designing-event-d...
[1] https://www.confluent.io/blog/building-a-microservices-ecosy...
The biggest, most common mistake is when a bounded context is called "<noun> service". This should be a huge red flag. Unfortunately, most books (and this post) always use this bad naming starting everyone down the bad path. At the very minimum call it a "<noun> management service" if it is the primary/general create/update activity. e.g you might have a checkout service that creates most orders and a order management service that can make drafts or adjustments. Order management (being the decoupled source of truth) might even be downstream from checkout service. Other services that need this information should get it's own copy asynchronously, i.e. decoupled.
The best way to learn how to do bounded contexts right is to study ones that worked well. Most writing tries to describe this in an abstract way using placeholder widget service things that helps no one.
The BFF pattern is good but doesn't solve everything. If you're accessing multiple services for a synchronous mutation, it's likely not highly reliable but can get by for an interactive service that can tolerate some errors. A messaging service is not one that should, they should always accept, store and forward with the lowest possible error rates.
Similarly DDD promotes separation of concerns and test driven development. That is exactly what eXtreme Programming told us in late 90ies when OOP and unit testing was in. Before that it was structured programming and software design specs, 4GL etc.
Bottom line is I wish people would start calling a spade a spade and indulge in less fads. Let’s just call it structured programming over the network with clear separation of concerns and clean interfaces. That’s good engineering. Test early and often. Write good documentation.
I understand though the marketing needs. IT has this steady influx of beginners that don't have the insight yet. Since there are so many of them, at any given time, it's reasonable (or maybe it actually works) to use standard advertising techniques at them. "Secret sauce", "The missing puzzle piece", "10 things that make you a better dev", "Dev done correctly".
Developers are kind of a dream target group: lots of disposable income, technically adept (i.e. at the cost of social skills), excited to experience a latest unicorn.
I'm seeing this whole "hype cycle" thing as a competition among more senior, adept in marketing, developers. Since there is such a high demand, the supply is also rich.
A wiki-like reference guide that can assist architectural and technology selection recommendations.
Potentially with support for argument mapping[1] and decision trees[2] to deal with contentious or context-sensitive topics.
[1] - https://en.arguman.org/
For some reason, important information isn't making it there. There's no page for Chain Replication. Maybe nobody has tried to add it? Chain Replication has been around for so long that it seems unlikely nobody would have tried to add it.
Wikipedia also seems to fall apart at the seams. It's great at giving you the answer to questions for which there is an unambiguous answer, but it doesn't do a good job of answering all the other questions you might have. Why is Prolog not very popular, what seems to be holding it back? Any answer to this is mostly speculation, something which is not allowed on Wikipedia.
Yes I am being cynical as well, but have a quite filled up bookshelf to prove my point.
REST API's and microservices are a design that originated when software teams started growing in size and didn't want to lose the agility they observed when having control of their complete stack. It was enabled by a fundamental shift in infrastructure: previously, everything was managed by dedicated infra teams, but Containers and Container Orchestrators gave development teams control of their deployments. The old model of monolithic services stood in the way of developers who had access to this new power, and hence microservices were invented to address that.
I could give you similar reasons for the rise of Big Data, Infra as Code etc. Every one of these changes are driven by changes in fundamental technologies. They aren't fads.
It does not, nor does the blog post say so. The two are completely orthogonal, and while TDD is a rather questionable development method, DDD is a widely accepted design principle.
These claims are orthogonal, so the "while" construction is misleading.
TDD and DDD are both arguably questionable and both undoubtedly widely accepted.
Not cynical at all — this is well observed and I wish more people saw it. I think software engineering is more susceptible (permissive cause) to fads because so much of the work is more craft than science and the actual catalyst (proximate cause) for the faddishness is the economic incentives driving the posturing for "thought leadership".
Consultancies need to compensate for generally not having a long-term stake in the projects they work on in order to overcome skepticism from clients so they have to invest in content marketing via thought leadership. The principles of marketing don't change -- convince people they have a problem, sell them the solution.
In a similar fashion every developer has various incentives to either consume and propagate or create this kind of content as well.
The only check on faddishness is well-grounded technical leadership.
The advantage of having each team own a microservice is that each team can deploy their software independently from the rest of the teams. And it works as long as the APIs are honored. So Microservices are way to decouple the work of different teams allowing them to work independently from each other.
While this let's you scale up teams, it's a huge overhead. So, unless you have over, let's say, 500 devs, it's not worth it.
Actually, one assumption is that the microservices don't share data. Only API (Rest, SOAP, GraphQL, etc) calls are allowed. And the boundaries are created minimizing the coupling between microservices.
> While this let's you scale up teams, it's a huge overhead. So, unless you have over, let's say, 500 devs, it's not worth it.
As opposed to Monolith ? I think you will have problems collaborating in a Monolith at around 30 devs.
What I am opposing is the idea that you can have 30 devs and 50 microservices. Which I have seen in real life.
You trade the ability for teams to deploy independently for the ability to be able to make breaking changes. This is a good trade off in my experience and is better for team velocity.
The approach we have taken is to keep a single codebase and deploy. We have applied the ideas from DDD to segment the code into contexts and assign ownership of each of those contexts to a team. This allows us to get some of the benefits of Microservices without losing all the benefits of a monolith. Having these boundaries in place also gives us the flexibility to move to Microservices if and when we decide we need to.
When you have 10 teams working in 1 mono-service, who is responsible for which parts of the DB schema? Can I query a table in one part of the app that another part owns? What if another team changes a table that I relied on and introduces a bug? Who is responsible for fixing it?
Microservices solve the people problems more than they solve the tech problems, assigning organizational responsibility for functionality, infrastructure, strategic objectives, etc. It also provides clear interface points between these things that can make organizing work easier. You get to split your 1 org into 10 little vertical orgs. This has benefits beyond just code boundaries. Now you can have 10 different goals all being achieved at once
You can go the other way, of course, and split the stack horizontally (One team is responsible for infrastructure, one for DBA, one for Frontend, etc) but then no team is working towards any one goal. Who is responsible for making forward progress?
Exactly. This is the idea that I haven't been able to articulate. And it is rarely addressed in the literature.
> When you have 10 teams working on 1 mono-service, who is responsible for which parts of the DB schema?
You can still develop interfaces between teams without microservices. There are a number of ways to do this but the simplest is just a folder structure that reflects the boundaries and enforce them by convention. If the model/repository/[your favourite DB abstraction] lives in `context1/user` you can’t access it from `context2/account` without going through `context1/publicInterface`.
It’s surprisingly easy, depending on your language, to write tools to automatically verify boundary violations if you need to.
This still allows you to solve all of the problems you listed .
that’s very different from a team of 8 people managing two dozen microservices, which is just insanity.
It really couples nicely with DDD and I’m in the middle of my second project where these principles are used.
I would not do it unless there’s a dedicated platform team that will take care of all the infra pieces you’ll require. This kind of points to the scale where it makes sense, I guess!
> Now, we know that in any modern application, sacrificing availability is not a good idea either.
I feel like this will apply to less than 10% of applications. Normally you'd rather deal with the occasional downtime than dig through the fallout of broken attempts at distributed transactions. Ask yourself "What amount of $X,000 are we losing per minute of downtime" and see if it rounds down to zero.
> Another way to model this system is to separate, or group related models into separate microservices. In DDD, these models — Price, Priced Items, and Discounts — are called Aggregates.
And this must be the case for less than 1% of readers - it would take a massive system before I'd consider having prices and discounts in separate databases. You're taking on the overhead of writing an additional service for what could have been a database join.
Great, then a new US comes in that changes the intention and you enjoy the pleasure of changing the interface. Enter versioning. You start crying.
That's not a microservices problem, not even a software architecture problem. That's at best a project management problem. In fact, pretty much every single engineering project there ever was would struggle to meet any form of neck-breaking change in design requirements.
You don't. You have more services accessed over a network, and might not have control over all services. But that's already par for the course in nonoliths.
If you were to refactor your components by changing classes around (say, a major version update in a dependency) while not updating the components that used them then you would also have problems.
That's why versioning and service discovery and HATEOAS are a thing.
Each service having well scoped and defined RBAC or AuthZ for its focused set of features makes the whole architecture as a whole much easier to reason about from a security standpoint. I've done successful pen testing and auditing of some monoliths in my time where the critical security issues arose out of untested and unexpected execution paths that were only possible because the surface area is so large.
Maybe in theory a "well written" monolith would be superior but I'm only going but what I have seen in practice. I think the extra work is worth the trade off.
I’ve stated it previously in the thread - you need to enforce conventions through tooling - which is where “devops” (ugh, I know, but really...) comes in to play.
We try to view the tooling platform as just another DDD domain.
Nice idea but most of the times horribly implemented, with no care about monitoring and logging as someone has already said.
It requires a degree of attention to details during design and development that few are willing to put in. It's easy to go for a prepackaged solution/framework thinking that you are building something that could be considered state of the art, when instead you end up with something that takes multiple tries to even boot correctly or 10 minutes or so to "stabilize" a garbled mess of inter-dependant components.
Designing these systems require a whole new set of skills that were not required when designing monoliths.
Beyond technical concerns, I learned that you can architect a looming disaster and then leave for a higher position at a big company. Nobody watching your conference talk will know.
I’m in a team like this, supporting a bunch of developers - we build a CLI, an API (in our domain! It’s just another domain with the devs as consumers) and juggle approx. 16 different tools (monitoring, metrics, tracing, service discovery, etc etc).
Doing it through custom tooling that enforces conventions is, IMHO, the way to do it successfully.
We’re also responsible for wider reaching services within the platform such as an API gateway, authentication, and event ingestion from IoT devices. As the team that has an on-call rota we also give the final go/no-go on services going into production.
> [...] we intend to cover our experiences with Event Storming in a separate blog.
In case anyone is not familiar with this usage, these days "blog" is often used instead of "blog post." I was thoroughly confused by this when I first ran into it. I tried fighting against this usage by colleagues, but it seems to be widespread enough now that it cannot be fought... I guess language evolves, and I have to get with the times.
I suspect one or more of the following aspects might be at play: • general loss of identity (since lots of posts are on medium, and less on a personal and work blogs) • "blog" is short for "blog post" • desire to distance oneself from "post" which is more associated with facebook and instagram posts
Blog: web log. A collection of entries that are used to log something and is made available to the internet.
Blog post: a specific entry posted in a blog.