Untangling microservices, or balancing complexity in distributed systems
vladikk.com
vladikk.com
1. I fervently believe, after having lots of experience with both microservices and monoliths, that microservices do not provide any technical advantage over monoliths. If anything, they can make a lot of technical details more difficult.
2. Microservices, however, can solve some particularly thorny organization problems for large teams with respect to how these large teams build complex software systems.
Thus, in my experience microservices can serve a useful purpose, but they will only be as successful as your organizational-wide communication is.
Furthermore, for the developers replying "I don't see why I'd ever use microservices for project xyz", well, if you're a team of 1, there is hardly ever a need to use microservices.
1) A monolith will have to consume a large amount of resources per request due to having all the models available at runtime. Lets say I have a system that handles 100 RPS but only 1 RPS uses that model, and that model is GB's of RAM and requires GPU, that means I will have to have this memory and GPU allocated for every request, making scaling much more costly. (Note: your mileage will vary depending on your complexity)
2) A monolith is a central point of failure, if I make a mistake in that service it can affect all of my requests, rather than just the request that calls out to that microservice, this is a difference between my entire service going down and just a feature.
These are just 2 I thought of off the top of my head, but I wonder why I seem to be out of step with HN consensus here.
Not all systems that use services is a microservice architecture.
#2 makes sense if you're taking advantage of that. In contrast, I've seen lots of microservices code that does little to no error handling. And if it's unicorn code that has error handling, they haven't really tested all the ramifications of their services going down in all the ways they can go down. Instead of having a system with a single point of failure, they have a system with N single points of failure. Their microservices are less fault tolerant than a monolith.
The backlash against microservices on HN is unsurprising. Over the years a ton of articles have been written vaguely about microservices and people cargo culted the architecture based upon these vague articles. Even reading comments people talk vaguely about "scaling" - what do people really mean? Before this backlash there had been non-ironic articles posted AND upvoted on HN about using microservices to scale to... a hundred requests per minute!
Lots of people seem to think that your architectural choices are either a single threaded rails app vs an eventually consistent distributed transaction CQRS nanoservice system that needs 99.999999% uptime. Meanwhile that eventually consistent distributed transaction CQRS nanoservice system that needs 99.999999% uptime is, for some reason, only running in one availability zone and falls down and corrupts your database as soon as any of your services fall over.
Oh, and it serves pictures of cats and it has 10 users and they're all family members. But someday that architecture will come in handy.
It seems that your problem has more to do with semantics than with what microservices are.
Arguably, the definition of a microservices architecture is "not a monolith", in the sense that it's a distributed system not only with regard to support and infrastructure services but also with regards to business logic.
Needing a heavy model per-request is very much the exception rather than the rule; even if you're doing ML, for most cases you'd do that with a single shared copy of your model. If you're somehow loading a model for each request, surely you can just not do that for requests that don't need it?
> 2) A monolith is a central point of failure, if I make a mistake in that service it can affect all of my requests, rather than just the request that calls out to that microservice, this is a difference between my entire service going down and just a feature.
I can understand people doing that in languages like Python or especially Ruby, where metaclasses / monkeypatching mean that it's easy for one part of the code to interfere with another. But I think it's a mistake to apply that to more structured languages where it's relatively easy to have an enforced separation between different modules within a single deployable.
1. Yes, if you have an application that works this way you will have to manage your deployment to compensate. But you can use configuration as feature flags to keep a monolith but when you deploy it to X servers you enable one feature set and to Y servers you enable another. So you get the benefits of developing in a monolith but the isolation of microservices.
2. See #1, but also use proper error handling. It's been a while since I've seen a single request take down the whole app.
- Microservices can be deployed as processes on the same machine and communicate over the loopback interface or domain sockets without much network-related risk. This is still more complicated than function calls, and I wouldn't quite call it a monolith.
- Monoliths can have traffic for different endpoints partitioned over different deployment groups. So the behavior of one endpoint can be changed independent of the others. But a module deep in the stack, potentially consumed by many endpoints, cannot be deployed on its own in a way that affects all its consumers. So I wouldn't quite call this microservices.
Another team writes and maintains a few more.
We also have some monoliths that provide customer facing UIs and aggregate data for our support teams and they are perfectly fine for that.
Right hammer for the right nail and all that.
Sure, monoliths are certainly easier to work with, and I think they should be the default starting point for most companies.
But regarding technical advantages, what if different parts of the program have different performance profiles and scale differently?
Sometimes there is a real benefit of having a service that can fail without impacting everything else. That is a technical upside, but it doesn't apply in most cases. But it do apply in some cases where you just want to limit any problems from impacting too many part of the platform at once. The service could of course still be both micro or not.
GOAL: Surprise and delight users by displaying their birthday on the setting page.
DELIVERABLES: Display User Birthday on Settings Page
"Surprise and delight" has definitely become one of my new bullshit bingo favorites.
There should be one of these videos for reactive programming as well. Misuse of observable pattern can quite quickly turn an app into a mess, making code navigation / minor change requests a nightmare.
But what you really want to understand, when building a distributed system (i.e. system to process large amounts of data), is how to partition the data, so it could be processed in parallel.
Essentially, there are two options, vectorization (single instruction multiple data) and pipelining (multiple instruction single data). They both have different uses in different scenarios, based on the critical path dependencies in the data processing.
Monoliths are easier to vectorize, microservices are easier to pipeline. But to choose which one you need before you understand the nature of data processing you need is a wrong way to do it.
If one part of your workload is suited to pipelining, and another part of your workload is suited to vectorizing, then that might be a reason to split the workload into different processes running on different clusters. Few but beefy nodes for the vectorized part. Many smaller nodes for the pipelined part.
That's not what I mean.
But you're correct, what you want to do with the data informs your decision of what should be separate processes, and you once you know, you might decide to split (modularize) the processing code accordingly.
Doing that other way around (i.e. to design the modules before you understand the data flow) is just going to cause more trouble.
> Back in the day, we’d spin up a microservice that did one, small thing just like that. We had a bunch of small services built and maintained by one person. This was great for autonomy, iteration speed, learning and making devops a no-brainer. You could spin up a service anytime: but you’d be oncall for it.
> Now, as my area is maturing and it’s easier to look ahead, as we create new platforms, we’re doing far more thoughtful planning on new services. These services don’t just do one thing: they serve one business function. They are built and maintained by a team (5-10 engineers). They are more resilient and get far more investment development and maintenance-wise than some of those early microservceis.
[1] https://lobste.rs/s/mc3k1c/at_uber_we_re_moving_many_our
That just seems like it'd be a disaster at any non-trivial scale.
Long time ago when computers were slow I did design some distributed applications myself - reliable IP multicast server, business process servers, map reduce type solutions like bill generation etc where it did make sense. That was long time ago and now most of those at their old scale could run on a single reasonable priced server.
For smaller teams or small projects it isn't really super useful. Also, my personal experience is that when you start making microservices, the monolith doesn't go away. It also isn't so cumbersome if each team is in charge of one or a few services and you only split things if there's a clear benefit.
I'm probably saying things you already know. My point is that I think for multiple cooperating teams, monoliths and microservices are complementary.
Microservices (where it makes sense) + Macrorepos.
Microservices are really handy when you're at scale and/or the problem you're solving works well with the paradigm. I'm thinking back to a gig I had where each of our vendor integration points (sending the same data, just formatted for that vendor) was a good use case even for a smaller company. We used Scheduled tasks to launch the application that did what was basically an ETL.
Oh, that vendor is down? Great, just have the sysadmin stop the thing.
No database muckery required. No need to build a fancy front end. Easy to delegate to/train on helpdesk type folks.
But any microservice is part of a larger ecosystem. It's critical to have a workflow that doesn't make refactoring the domain a huge chore/lift. IOW: Keep the related microservices in a single repo.
Really, unless you've got a use case like the above (processes that need granular control) or HA/Scaling concerns (since in that case you'll be able to better focus on the parts that need to scale), A Monolith or domain based services are the way to try to go.
Take one example: The single server idea is going to suffer outages. You could at the very least have two monoliths either load balanced or in some kind of failover setup.
But, what distributing some of the components into so-called microservices would give you is a graceful failure scenario where even if all the instances of a specific component fail, the rest of the application still works. Maybe only one function stops working temporarily but most work continues on.
This doesn't mean everything must be microservice koolaid all the time all the way, but I'm sure you can conceive of that being an advantage under certain use cases that would be worth other possible trade-offs.
As you point out, there is nothing inherent in a monolith that makes it less robust. I currently am building a monolith that is deployed on auto-scaling servers on a cloud provider, so if one server dies it just spins up other instances.
I've also found that most so-called monoliths still often end up breaking apart components anyway. I've only seen true monoliths as described above in the mom and pop type IT shops where there's only a couple support staff and they don't have the resources or experience to properly break things out into separate resilient components. And yes supportability is definitely a valid part of that design call.
Again that was just one example of a potential advantage to it while admitting there are trade offs. Others here have given better examples such as at larger orgs where multiple independent teams get involved with a single app.
For particular usecases that fall out of the wheelhouse of my preferred language, I'll typically use single OpenFaaS functions + endpoints for these things (think doing ML with Python, or really heavy computation with Go/Rust).
This combination of containerized Mostly-Monolith with single-purpose endpoints for particular usecases works really well and is very productive. If something is much easier in another language (due to tooling/ecosystem) for example, then I can just build that method or sub-system out in whatever language it's fastest in with this architecture.
The important thing to note here is that microservices give you one way to build a graceful failure scenario. You don't get it for free. Often that 500 will propagate all the way back! So you have all the same failure modes + network failure modes!
You could also build a monolith that could handle component level failures, for that matter. Tooling doesn't seem to be as common, though.
The point being you don't need to be a paying subscriber to microservices(tm) to recognize there can be positives to the approach, which you describe in your last paragraph
A monolith that can handle component level failures is very close to the same idea infrastructure-wise minus a lot of baggage implied by the buzzword microservices.
I'm not telling you that you must to do this. Others may have requirements that go beyond yours.
Just curious, but how do you take advantage of the multiple cores? Are you using a language that handles concurrency directly, with something like threads? If you use a language that has historically been single threaded (Ruby, PHP, partially Python) do you use multiple processes? If you use multiple processes, what manages the processes? What can spin up additional processes? And when you talk about a "multicore monster" what are you actually running there, the application or the web server or both? If the web server runs elsewhere, how does it connect to the app? Do you have something else to load balance the web server itself?
I do not thing upgrading microservices is any simpler. It could actually be way way worse.
What you should be doing is telling people what they need to hear and let the biggest layers figure out for themselves that they don’t work that way.
But we want to know what the big people are doing as if that will let us become big too.
But the bulk of what we do is reacting to situations, not creating them.
I disagree, the Unix philosophy proves otherwise. You can build very powerful applications, just by combining grep, sed, ls, ... and the like.
- Teams commit to individual repositories so that team membership, changeset review, and issue management are kept close to the relevant code, and checkout sizes are minimal
- Strong language support for inter-repository dependency management is introduced to enable effective rollback and rollforward of commits that affect other repositories
- The function call becomes the unit of deployment -- not a microservice, and not a monolith. This allows for low bandwidth deployments and for small amounts of highly-utilized code to migrate to the devices where they are needed for computation
- Function calls and return values are signed cryptographically so that inter-function calls have integrity, regardless whether within a single device or across the network
- Functions that perform I/O calls are tagged with the virtual storage and/or network interfaces that they require access to
- Kubernetes-like infrastructure manages the deployment, scaling, and resource allocation of functions to devices
This seems to me like it would provide the organizational benefits of microservices while reducing developer hand-wringing over where to define service boundaries. It just becomes a question of 'should this be a function?'.
Good tooling around binary-dependency-or-source-code retrieval could make it easy for developers to work on large applications built like this and inspect / debug / modify code as-needed without having to check out or download all the source code and binaries at once.
Ultimately everything about data flow, typing, abstraction, and so on has some effect on performance, encapsulation, complexity, organization. In a way all aspects of development from onboarding to builds and tests to release distribution need to be measured the same way we take into account of resource usage and basic performance criteria. If a piece of software masterfully meets all needs, but has internal interfaces that are simply too complex and brittle to be worth learning then that software will quickly decay and need to be replaced as the dynamic world alters the constraints of usage.