Comment by whack | Hacker News Reader

Comment by whack | Hacker News Reader

My intuition is that at a certain scale, microservices are preferable to a monolith. Two principled reasons:

- Autonomy over deployment rollbacks. If your team owns a microservice, and you noticed that you deployed buggy code, you can easily and autonomously do a rollback. Your rollback doesn't impact other teams. It isn't visible company-wide. This allows teams to take more risks with fewer downsides and more psychological safety

- Isolation of secrets, api-keys etc. If one module has access to an api-key, then it's effectively made visible to every single module in the monolith. Other teams can decide to bypass your API entirely and read directly from your database, making it impossible for you to abstract away your implementation details

Problems solved by microservices that could in theory be solved in a monolith:

- Build/test times. A big monolith will have a huge build time and a large number of CI/CD tests. In theory you could parallelize your build/test-suite to make it infinitely fast, but I'm not sure how practical this is

- IDE slowness. A monolith will make your average IDE run much more slowly. In theory also solvable, but would require major IDE changes and investments

- Code ownership by specific teams. In theory you can use things like regex-CODEOWNERs to enforce code-ownership of individual modules, but it's easier with microservices

- Migrating to better languages/frameworks. If your original monolith was written in perl or on a bad framework, it's very hard to move away from it. With microservices, each microservice can be independently and incrementally migrated, allowing the company's tech stack to evolve gracefully.

Any others I missed?

One that I don't think a monolith can solve is partial availability.

A monolith is either available or it isn't, which is fine as long as you can guarantee that nothing ever takes down the monolith, but we all know that that's not something you can guarantee.

If you split out services that live outside the critical path, then when those services go down you can still engineer your system to keep going and gracefully degrade the functionality provided by those currently-down services.

This makes incident response much less stressful because there's less urgency, which in turn leads to fewer mistakes during the response and faster recovery times.

(Yes, there are some pieces of functionality that will always be in the critical path, like authorization, but many things don't have to be.)

This is why I like having things split up. Even with zero service redundancy, it massively improves the reliability of the system overall. Often users don't even notice that a less important service had to restart because of an issue. If that service was just another component of a monolith, then it failing would guarantee that the whole monolith fails.

In my experience you’ll be fine with monolith instances and failover. Probably until you’re Google scale

Monolith plus failover gets you better availability, not partial availability. Code errors or network problems or many other types of errors can still take down both the main instance and failover unless you're perfect at preventing them, and when that happens you have zero availability until you fix it.

Not really. Even with a monolith, you get partial availability. Most often as a default.

Imagine a web server handling requests. It usually has a way to recover from a crashed request handler. On top of that, you have multiple web servers behind a load balancer. Some code paths may be failing, some may be available.

Then, you have some background workers, maybe ingesting data from third party services. They're completely different processes from your web servers, even though they still use the monolithic code base. If they're down, the web app can still work with stale data.

The partial availability can be compromised in microservices as well as in a monolith (e.g. auth as you mentioned).

> Imagine a web server handling requests. It usually has a way to recover from a crashed request handler. On top of that, you have multiple web servers behind a load balancer. Some code paths may be failing, some may be available.

This is a form of partial availability but not what I'm talking about. I'm talking about a scenario where a DoS attack or an initialization error or an external service timing out causes the entire service (all instances) to fail to respond to any requests. Past a certain scale of app these things do happen periodically, and if you've split out non-critical code into separate services and handled for graceful degradation that code can no longer bring down the critical paths.

> They're completely different processes from your web servers, even though they still use the monolithic code base.

I... wouldn't call these part of a monolith, though they're not exactly microservices either. They're separately deployed units of code that happen to exist in a monorepo. This gives them similar resilience characteristics to microservices, so they're not a counterexample, they're more of a subcategory of what I'm talking about.

> The partial availability can be compromised in microservices as well as in a monolith (e.g. auth as you mentioned).

This is the "but sometimes" fallacy. If solution B allows for graceful degradation 80% more often than solution A, it's still better at graceful degradation even if it doesn't reach 100% success.

Google famously still primarily maintains a monorepo, but they absolutely don't deploy a monolith. They published a paper last year that describes an approach that is quite close to what they have internally - https://dl.acm.org/doi/pdf/10.1145/3593856.3595909

Effectively, by setting strong boundaries between modules, they can develop on their application as if it was a monolith, but at runtime, separate those modules physically deploy them as micro-services. This solves a lot of the versioning issues (since the app is rolled out atomically) and correctness issues (easier to reason through a monorepo).

Modulith is the underrated happy middle ground imo.

Reply on news.ycombinator.com