Meta’s Microservice Architecture [pdf]
usenix.org
usenix.org
The fact that one has seen the video doesn't make the link recognizable. (And that's a problem with Youtube.)
However, in my quest for the holy monolith I came across small teams who against all odds were adequately functional despite this heretical paradigm. They were definitely building a distributed monolith which was a monster to run locally. But, it worked. They were shipping and most importantly it matched their culture of small isolated islands of functional specialists with as little communication between them as possible.
I will refrain from commenting on the virtue of communicating as little as possible, but sometimes you just got to make the best of a situation.
It can be the cause or the consequence. If your team peels a microservice out of a monolith and assign a couple of developers to work on the newly-created service, those guys will be heads down on their work and focusing on their project.
If management does not go out of it's way to rotate people around projects, you'll end up with ad-hoc organizations around their service architecture.
Conway's Law is about how your organization's communication structures work, and how that inevitably leads to specific outcomes. That outcome is an eventuality for the team, because the team can't change the communication structure. They can design the best damn microservice in the world, and the resulting use of it could still be shit, because the other teams aren't necessarily up to snuff. And you still have all the baggage of dealing with those intra-team, intra-service issues.
It's great for cloud providers too. It's often cheaper to just run entire copies of monoliths than it is to run microservices. All that synchronization and API calls has a huge cost, even more so when you add support infrastructure (RMQ? Kubernetes?)
Most companies would work just fine with a monolith, with the occasional service split off from that when it makes sense.
It sure seems like a lot of the microservices plays come from technologists wanting to use it as OTJ training. There are lots of neat things to experiment with, and there are lots of ways to glue things together. There's a whole resume to build with all of the ancillary tech required just to run a process.
Your point about cloud is right on, too. That goes both ways. Some CEOs chase the deals. But some CTOs chase the greener-grass dragon. One thing leads to another, and suddenly you're migrating from k8s on GCP to nomad on DO.
> with the occasional service split off from that when it makes sense.
Exactly what meta does, according to the article. There it is.
This sounds like a terribly naive take.
It matters nothing if units talk to one another. Nothing. All it matters is ownership and accountability. What happens if the service managed by team A goes down and takes the whole org with it? Is team B going to take the blame because one of their developers posted a PR to tweak the project's README?
They can’t get their internal silos to work together on a single project, so they have each silo make services for the others to use in service of the project they’re trying to make that needs multiple teams.
But… all the same problems come up. Teams building to their requirements and not what the consumers need, refusal to cooperate in design/requirements/etc, scheduling issues or project priority issues.
“Microservices are cool and what Google does and will help us avoid our organizational issues” is very loosely paraphrased from what I heard as the exact pitch for why it was being used.
If only someone that mattered would have written on time an essay titled "Microservices Architecture Considered Harmful".
I believe what the original poster meant to say was microservices grow like O(klogn) and monoliths O(n), i.e. microservices have some upfront constant cost that is large enough that, for small projects, monoliths will still perform adequately and be cheaper to develop and deploy. Once you exceed n large enough to overcome that upfront constant cost, microservices become the superior option.
I'm not agreeing or disagreeing with this, by the way.
A monolith itself can be monolithic either at the design level (only the whole thing can build and work) or at the deployment level (multiple components build, test and – to a certain extent can work – independently of each other but are assembled into a single deployment unit).
As far as differing viewpoints are concerned, let's consider a hypothetical design where GET, POST, PUT, PATCH and DELETE are a «µ-service» design and deploy as 5x distinct deployment units but as the whole and at once, i.e. leaving the DELETE out of the deployment renders the solution inoperable from the business POV. So far so good, we have a conventional µ-service.
Let's now consider a hypothetical yet real scenario where a single POST invocation triggers a chain of 27x service invocations to 10x different systems to produce a result. The implementation is stateless and idempotent, but the processing logic is complex, and it has to be executed in a particular sequence, intermediate results have to collated or fed into the next processing step – either sequentially, or in parallel, or both. And the processing can't be refactored out into smaller components due to the complexities or the business logic or due to technical constraints that the 10x external systems impose. Is the implementation of POST a µ-service, or POST is a monolith, or POST is a monolith within a µ-service? It can be either, neither or both – depending on the point of view. Such hypothetical design are, in fact, real and are fairly common in large environments.
On the scalability point, there is no single answer, either. A design can have scalability constraints either at the design level or at the deployment level. An example of the design level constraint is the strict processing order requirement that imposes substantial restrictions on what and how it can be scaled. It can typically only scale up but not out with not much room to wiggle around in. If the processing order is not important (i.e. the eventuality – not the linearity – is the only requirement), then scalability (up and out) becomes a deployment level constraint which is easy to fix (i.e. a configuration time or an auto-scaling policy change).
Scalability can also be impaired in both, monoliths and µ-services, if at least one external system is slow to respond, or the response time varies. In such a case, both designs will equally suffer.
> […] and be cheaper to […] deploy.
I have deliberately omitted the «develop» part to emphasise the «deploy» part. The deployment cost has gone down significantly over the last decade alone. What used to be an arduous task requiring a coordination of multiple people has now become a few line deployment file change and a one person job in many cases. Even complex deployments are much easier now than they had ever been. Therefore, I would posit that the cost of monolith and µ-service deployments is roughly the same today compared to monoliths being cheaper to deploy 10+ or so years ago.
Except where it was rebranded as throw/catch. There, goto remains quite popular.
All while bridled (C-style) gotos were considered acceptable by Dijstrka, but are now looked upon with disdain. Amazing what a little marketing can do.
Go/Rust/Zig Errors as values is a much better system forcing you to explicitly deal with the error, crash or pass it on. Rather than hoping you handled all the correct exceptions/someone else will handle all of them.
There will always be naysayers. The above is starting to become generally accepted, though, and modern languages are moving away from the practice just as languages moved away from unbridled gotos.
Doesn't mean that Dijkstra was wrong, or that they are the same as goto.
Also, throw/catch do quite more than a goto, though. It's not only about stack unwinding, it's about the ergonomics of not needing code that uses throw to have any information about where catch is located.
Sure you can simulate some use cases of try/catch with goto, but implementing the general case is much harder.
throw/catch is exactly the kind of good goto-replacement provided by "higher level" programming languages that Dijkstra is talking about in his paper.
Not within the context of discussion, where goto refers to the harmful kind. Throw/catch exhibits the very problem Dijkstra was talking about, with execution jumping all over the place haphazardly. Functions, conditions, iteration, break/continue, even (C-style) goto does not exhibit the same problem. They are strictly bridled in the execution scope.
There is good reason why modern languages are moving away from throw/catch. However, there is little question that it is still widely used at this time.
That's like saying all maths is just the application of addition.
That’s what I have “rebranded goto” in quotes. And that’s what I address in the second paragraph.
Going back to the original discussion, my point was that the only statements that are real gotos still in usage (except for C cleanup gotos) is break and continue, which even support labels
Except I posit that throw/catch still suffers the same problem he speaks of. It becomes difficult to follow when and where the code will jump to an arbitrary spot. The harmful parts of goto live on. Granted, we are learning. Modern languages are abandoning the throw/catch concept.
virtual polymorphism for example is completely unexpected, as well as function pointers and other constructs that in my opinion are harder to track than exceptions
We did it with developers in the tens and we didn't have much issues with this approach. In fact, at one of my workplaces we had much more issues with a monolithic app than with the microservice based app we replaced it with.
So a rare bug in your mailing list signup workflow that hangs the process and causes it to be killed causes a random selection of inflight webpage requests, payment transactions, message handlers and business processes to fail. And if those failures aren’t all cleanly handled, your mailing list signup bug could propagate into a much wider issue.
Whereas if you have a ‘mailing list service’ that has its own processes that can be killed and respawned, that bug only takes our mailing list processing. Which is good because the bug was probably made by the team who owns mailing list processing. And they can roll back their code and be on their way, with nobody else needing to know or care.
Monoliths should be stateless (if achievable) and have no concept of partial success in cases where you would like atomicity unless everything is truly idempotent (easier said than achieved). If those criteria are met then callers just need to retry in the event of failure which can be set up for basically free in most frameworks.
If you're pushing fatal recurring bugs into production, then that is a separate problem wider than the scope of a monolith vs. micro.
For those of us in the real world who can’t afford perfection, the ability to isolate the impact of the inevitable bugs that do sneak through has some appeal.
As does the fact that exhaustively testing a microservice in a realistic timeframe is a much more tractable problem than exhaustively testing a monolith, which reduces the risk that such bugs will ship in the first place.
Bugs are less likely to ship. And when they do they will have a more limited blast radius. And when they’re detected they can be mitigated more quickly.
Those all sound like great benefits to me.
Bugs aside, the architecture does matter, and it matters a lot.
Whether it is a single coarse grained deployment (i.e. a monolith) or a fine grained deployment (modular services or microservices), a solution has a number of technical interfaces. The tehcnical interfaces broadly fall into low and high data volume (or transaction rate) categories. The high data volume interfaces might have a sustained high data flow rate, or they can have spikes in the processing load.
A coarse grained architecture that deploys all of the technical interfaces into a single process address space has a disadvantage of being difficult or costly (usually both) to scale. It does not make sense to scale the whole thing out when only a subset of the interfaces require an extra processing capacity, especially when the demand for it is irregular but intense when it happens. Most of the time, a sudden data volume increase comes at the expense of the low volume data interfaces being suffocated by virtue of high volume interfaces devouring all of the CPU time allotted to the solution as a whole. Low data volume interfaces might have lower processing rates, yet they might perform a critical business function nevertheless, an interruption to which causing either cascading or catastrophic failures that will severely impair the business mission.
The hardware (physical or virtual) resource utilisation is much more efficient (costs wise as well) when the architecture is more fine grained, and the scaling becomes a configuration time activity which is even more true for stateless system designs. Auto-«healing» is a bonus (a service instance has died, got killed off and a new instance has spun up – no-one cares and no-one should care).
For monoliths you cant be as specific. “Is the response a 500” doesn’t really cut it. “Average request latency” for scaling doesn’t cut it when some of your queries are reads and then some are completely unrelated mass joins.
The software is going to be deployed at different locations with different scaling concerns. In some places, it's fine to just run 1 instance where it does all 6 jobs continuously. At other places, I anticipate adding parameters or something so it can run multiple instances of a subset of the jobs, but not necessarily all the jobs on every instance.
- consider case where the task is CPU intensive but not so critical as to eat into other parts of the code
- consider the case where the task needs some data loaded for it to work. I don't think it is a good idea to have that data loaded into the monolith.
I somewhat fail to see how that saves much effort; routing setup sounds like a hassle.
What we‘re using at my work is just a mono repo with all services in it, which works pretty well, and we‘re like 7 BE devs
I see how it works, and I completely agree that to start out, so going from PoC to first business implementation, a monolith is the way to go (unless the goal from the start is 100 million concurrent users I guess).
But after that initial phase, does it really matter if you use one or the other? You can overengineer both and make them a timesink, or you can keep both simple. I do agree on things like network latency adding up, but being able to completely isolate business logic seems like a nice gain. But Im also not talking about real micro level (i.e. auth login and registration being different services), but more macro (i.e. security is one, printing another(pdf,csv,word etc), BI another one
Not saying it can't handle everything as well. Just saying the modularity of microservices makes it, in my pov, easier to handle large complex real time systems.
Maybe that's also something that comes with experience - as a rather "newish" guy (professional SE, so one level above Jr), it makes it easier to work on our project.
The routing of just load balancing is much simpler than the routing of exectution jumping between many microservices.
>You can overengineer both and make them a timesink
I agree, but a microservice architecture starts you out at a higher complexity.
>but being able to completely isolate business logic seems like a nice gain
That can also be done by having that business logic live in its own library.
Not necessarily at all, i.e. using GRPC it's all self discovered.
> I agree, but a microservice architecture starts you out at a higher complexity.
Definitely
> That can also be done by having that business logic live in its own library.
That's true, having it in it's own library is certainly a possibility -> but then it's also not that far off micro/macro services anyway, except you deploy it as one piece. And basically this is my argument: If you're having it all as libraries, and you all work in a mono repo anyway, the only real difference between micro/mono is the deployment, and that with micro you _could_ independently scale up whatever the current bottleneck is, which we've used plenty of times
Honestly I think the realistic advice should be to go monolith if you or part of your team aren't experienced with microservices or if your app is simple / you'd be overengineering it otherwise.
If you're starting a SaaS company, you can envision the moving pieces, and will be growing your team quickly microservices properly in the beginning can have a lot of benefits.
Just feels like another one of those dogmas people just mindlessly scream on the internet all day without considering all the cost/benefit analysis for each particular case.
In other words, different teams don't want to talk to one another. That's not really a good reason for having 'micro' services.
You can also self-heal monoliths. In fact, it's much easier to do that with a monolith.
> synchronizing dependencies, tools, and releases
Such a strong statement. Might a lack of industry experience be driving such strong convictions of yours?
Here are some reasons, off the top of my head, why a company would want to embrace microservice architecture, with all its benefits and complexities, with a developer count in the 10s:
1. You're building a product that touches multiple deep domains and the business has modelled a very aggressive headcount growth
2. You've outsourced a large portion of your development to a number of agencies
3. You have a team of individuals who know nothing but microservices
4. Your chief compliance officer is intimately familiar with the data protection benefits that microservices bring and is leaning heavily into it in their regulatory submissions as a way to compensate for some other gap in the business
5. You're building anything to do with image processing at scale
6. You are a subsidiary owned by a parent company with tons of experience and tooling for microservices
7. One of your VCs has offered up a dev team they own to speedboat your MVP, who specialize in microservices
8. You've received a buyout offer by a party interested in specific IP within your product, with the condition that the IP is isolated from other parts of the system
I would argue that microservices make it harder for different outsourced teams to work together.
I tend to agree with OP in that microservices really slow down a smaller team and more aligned with helping giant teams function.
That feels like premature optimization; split when you need to and not before
2. You've outsourced a large portion of your development to a number of agencies
Then your developer count is likely not in the 10s (you need to could the agency developers), plus if you've already outsourced your development in that manner, it suggests you already chose a micro-services architecture and tendered accordingly, this feels like a post hoc justification.
3. You have a team of individuals who know nothing but microservices
If a team can build a set of microservices, they can build a monolith, the skill sets are not that different, a microservice is after all just a really small monolith.
4. Your chief compliance officer is intimately familiar with the data protection benefits that microservices bring and is leaning heavily into it in their regulatory submissions as a way to compensate for some other gap in the business
That's an interesting one, you're trading technical complexity for compliance, and it may well be a use case, there is not enough data to comment here. But there are many ways to be compliant with <insert framework here>, micro-services might be one, but is it the optimal solution for all involved, well that depends...
5. You're building anything to do with image processing at scale
This doesn't require micro-services, it probably requires horizontal scalability, if its offline processing you might want a batch process you can turn on and off as required, but that doesn't have to mean microservices, at this point it becomes a semantic argument of what constitutes a microservice, but I would argue the idea of batch processes predates the idea of microservices. Also just because you might need to use microservices in a small part of your application stack, the rest of the solution can still be a monolith: a hybrid architecture if you will.
6. You are a subsidiary owned by a parent company with tons of experience and tooling for microservices
Again, if you have tons of experience with microservices that can be easily translated to monoliths, just build a microservice but bigger
7. One of your VCs has offered up a dev team they own to speedboat your MVP, who specialize in microservices
It its 10 devs or less, does it matter, build a monolith and optimize when it makes sense to do so
8. You've received a buyout offer by a party interested in specific IP within your product, with the condition that the IP is isolated from other parts of the system
Unless that is your goal from day 0, I'm not sure how you could anticipate this, again this feels like a post hoc justification. If, in the unlikely event that that situation occurs, that might be a good time to consider splitting out that functionality.
Just curious, how does microservices help here?
(/s, obviously, but I've heard variations on this plenty of times)
No. microservice vs monolith is not the deciding factor of the developer speed or bugs. It wouldn't even be top 5 factor for good developers. Difference between microservices and monolith technically is just network rpc rather than function call. In itself, it doesn't make much of a difference. If a developer finds themselves stuck because of microservice architecture, they are probably very bad developer in first place
- distributed systems problems - think consistency and ACID
- increased refactoring complexity - you now have to change the api three times instead of 1 to deploy the change with 0 downtime
- requirements for more generic CI/CD pipelines - you will develop many microservices, it's important to do it fast
it's not "just" a network rpc, even technically
Yes this causes overhead; every microservice will have a public API of sorts that will need to be documented and communicated, and any other service consuming it will need to eventually be updated to keep up with changes. If this overhead is not something your organization can afford, you should not be doing a microservices architecture.
Except is is. Microservice doesn't mean distributed setup. For the first problem, yes service running in multiple pods causes issues, but even monolith could face the same issue. Statefulset based microservices not only exists, but is a pretty common setup. Also I would even call multiple container on the same pod as microservice, which basically solves all of the problem.
There are many ways to solve second problem. Some folks create different API version. I prefer different deployment.
Last is a problem for monolith. I know of big companies where compile time is in order of an hour for monolith, even for small change.
A distributed monolith is simpler, but simultaneously more buggy than a centralized monolith.
- In case of errors, do we get a full backtrace of function calls across service boundaries?
- Is the function call as cheap as passing an argument? What if we're passing 1KB? 1MB? 1GB?
- Can we use a debugger to step in and out of those functions?
- Can we spin up a simple test runner to integration test a few levels of function calls together? Preferably something like Jest that has a watch mode so we can quickly run dozens of tests?
- Can all database modifications run in a single transaction so that we know that either all of them happened or none at all?
Unless you don't discard the error, then yes
> - Is the function call as cheap as passing an argument? What if we're passing 1KB? 1MB? 1GB?
No it is not. In general microservice costs are bit higher.
> Can we use a debugger to step in and out of those functions?
You could trace across services but passing the context. Also attaching debugger is possible afaik.
> Can we spin up a simple test runner to integration test a few levels of function calls together?
I use docker compose and it can be done
> - Can all database modifications run in a single transaction so that we know that either all of them happened or none at all?
Yes. I don't see how this is different than function call.
If you need atomicity _across services_ it's very different, hence why everyone resorts to eventual consistency and all the associated extra complexity. If there's a decent way to have true transactions that span multiple services, I've certainly never seen it.
Well, one interpretation is that Meta runs predominantly on a monolith but has factored out a few services.
Check out quite a few interesting papers from them in OSDI 2023 (top systems conference, co-located with ATC where the OP URL links to): https://www.usenix.org/conference/osdi23/technical-sessions
It seems to me the problem with the microservice concept does not come from the decomposition of a monolith into well defined stateful/stateless services but indeed the lack of further principles and tools for doing so in an optimal way.
Imagine a decision process that would ingest some quantitative details (including a topology of dependencies) and volumes of information flow and computation etc and spitting out an architecture. In many cases the outcome might be "monolith".
While fads and bandwagons promising utopia are prevalent in tech as in almost all other domain, there usually is a core truth that activates them.
Microservices are organized by business use-case in order to isolate a team so they can focus in deeply understanding the requirements of this specific domain. This is fundamentally a fuzzy and human division of concerns, and has little to do with the performance factors you state, which seems to be a secondary priority. Mantainability and code-quality are related concepts, but they don't seem to be the top priority either, for them it seems that it is more important to enable a team to attain deep knowledge of the problem-space assigned to them, rather than ease in implementing solutions.
Sure microservices are supposed to handle scale, but are we talking about computational scale or organizational scale? It is ambiguous, but I think it's more of the latter. That's probably why so many technical issues have surfaced surrounding this architecture after the initial hype, it was not about technical efficiency to begin with, and it only makes sense for very large teams of engineers, if at that.
Just my interpretation, I don't claim to be more qualified than any other professional here to have an opinion on it.
That’s too bad. One of the funnest thing is when a backend service calls into www, the monolith PHP code base. For example, callbacks when an item enters a queue or video call changes state. And by fun, I mean annoying to debug.
Language/runtimes like Erlang & Elixir on the BEAM/OTP that were built with this in mind work well. They are completely functional with all state contained with lightweight processes and can send messages with timeout handling transparently across nodes in a cluster. With OTP you get a supervision tree for those processes that can automatically restart nodes and child processes of nodes.
Assuming some simplifying assumptions, yes. But those simplifying assumptions tend not to hold in the real world.
Microservice is the atom.