When Should I Break My Application into Multiple Containers? | Hacker News Reader

When Should I Break My Application into Multiple Containers?

rhelblog.redhat.com

90 pointsfatherlinnux57 comments

Am I the only one that loves the monolith? Apart from the scaling mentioned in the article, this feels like accidental complexity. And, let's face it, 95% of apps are about serializing complex business problems into code.

Also, I don't agree with:

> Since these components are doing different things, there is little chance that there would be a performance benefit

Complementary workloads should not be a problem. It would actually make thrifty usage of the hardware.

The only reason I see for breaking the app this way would be to enforce low coupling.

Monoliths have their role in the world, certainly! I've heard Facebook have a mantra of sorts about their monolith, if anyone here could corroborate that would be great. It goes something like:

- build v1 in the monolith

- do users like v1?

- if so, does v1 use too much of the monolith?

- improve the implementation within the monolith

- does it still use too much of the monolith? Break it out into its own stack

I believe there are only two stacks:

- ad serving, where literally lower latency is more money and thus benefits from optimizations not realistic within the monolith,

- and "everything else".

Martin Fowler has argued for something similar. [1] I suspect that the "right" answer is complicated and based on things like where you're coming from, the size of team(s), and the degree to which there are well-defined shared services.

DHH of Basecamp also argued recently [2] that Monoliths can make sense although I noted at the time that, with a team size of about 12 engineers, that isn't much larger than the two-pizza team sizes you see promoted for microservices functions.

[1] http://martinfowler.com/bliki/MonolithFirst.html [2] https://m.signalvnoise.com/the-majestic-monolith-29166d02222...

Also known as "we'll fix it later".

The app in the article is an interesting case. If you've got an app split like that, working backwards from Conway's Law tells us there are likely two separate teams working on it. Cutting the deployment in half may well allow each team to become more independent. That might be the real win.

I actually regretted not mentioning the organizational advantage. Basically, scaling and organizational split are really the major two business drivers for splitting an application up!

>> Since these components are doing different things, there is little chance that there would be a performance benefit

> Complementary workloads should not be a problem. It would actually make thrifty usage of the hardware.

The argument continued "there is little chance that there would be a performance benefit from being in the same JVM". Splitting the two components out into separate containers doesn't preclude running them on the same hardware. But it does give you the option of running them on separate hardware, which may allow for even more efficient overall hardware usage.

The only performance reason I could imagine would be if both pieces had required large heaps and had difference garbage collection profiles. Putting them in separate containers (and by extension JVMs) would split the size of the heap for garbage collection and provide more deterministic garbage collection cycle times. However, to receive such a benefit, the heaps would need to be fairly large (e.g. greater than 4GB each).

> (e.g. greater than 4GB each).

That's an awful lot of RAM to be using. I doubt many people even have servers with that amount of RAM in them let alone enough for 2.

Have you seen a modern server? People have 1TB Ram in some of them.

I've seen 2TB already.

I thought that I didn't need to explicitly tag that as sarcasm. Apparently I was wrong...

the jvm is already a container, putting them in docker is additional overhead vs simply starting two java processes.

Splitting into separate JVMs two services that communicate through a Java interface would also add lose you:

* the L1, L2, L3 cache friendliness

* the inlinings that JITs can provide

* the temp variable eliminations that JITs can provide

* the ability to do zero copies

You would add:

* the overhead of the full http stack, if this is done in REST

* the network latency, if this is on separate hardware

If you care about latency, just forget splitting it. Worst cases could be going from nanoseconds to milliseconds.

If you care about throughput, it could help in offloading; but don't expect 2 to be split into 2 x 1. You would be closer to 2 x 1.5.

AWS is supposed to give us soon some new shiny X1 instance types. Before scaling forces you to split, there is plenty of room for pretty beefy monoliths with 2TB of ram and 100 vCPUs.

The article in the next point though says "These two applications communicate using a REST API (…instead of using sockets, shared memory or files, etc.)."

So it's already lost all the benefits that it could have from running both components in the same JVM and communicating via a Java interface.

> the overhead of the full http stack, if this is done in REST

Yeah, just don't do this.

In many cases, I consider microservices an optimization. It's usually just a case of time to market to getting enough paying customers to afford to spend time on the optimizations. Monoliths are great for time to market.

Here's a piece I wrote for Codeship breaking down the problem and how Heroku can seriously insulate you from the complexity of microservices. https://blog.codeship.com/exploring-microservices-architectu...

> Am I the only one that loves the monolith?

Discussions on this topic are typically mired by a lack of conceptual clarity†. Are we describing a monolith in terms of the process model, or the deployment model?

In context of scale-up/scale-out concerns, we would be considering the deployment model.

In context of loose-coupling and extensibility, we would be considering the process model. Postgres is a classic example. [1]

[edit: note above are not mutually exclusive.]

Naturally, an increase in the number of moving parts places additional orthogonal demands in terms of operational integrity. As with any upward clicks of complexity, the decisions should be informed by valid‡ requirements and facts.

Ok, done with my mini rant /g. As to your query, in certain cases a monolith is indeed worthy of love.

[1]: http://raghavt.blogspot.com/2011/04/postgresql-90-architectu...

† Not a surprise in a field celebrating its unique brand of "senior engineers with 3 years of experience"...

‡ No, hype cycles are not valid architectural and/or business requirements.

For further clarity, I suggest these blog posts by Uncle Bob Martin:

http://blog.cleancoder.com/uncle-bob/2014/09/19/MicroService...

http://blog.cleancoder.com/uncle-bob/2014/10/01/CleanMicrose...

> ‡ No, hype cycles are not valid architectural and/or business requirements.

I'm stealing this. Fantastic way to highlight motivation for certain techs.

I don't have a lot of skin in the microservices architecture game, but my gut intuition tells me that in most cases, it's better to build your business on a monolith for simplicity (it's easier to conceptualize your entire application when it's one thing) and then break it down when it's "proven."

Do people tend to love "THE" monolith, or do they tend to love "THEIR" monolith? By that I mean, do people like some of the simplicity & efficiencies of the monolith pattern as a general pattern, or only when they're able to strictly control the evolution of the architecture in there?

What about the deployment benefit? Surely if you want to scale out your API you don't necessarily want to scale out your webservers - that's ludicrous! Surely the applications should be already decoupled enough that there's pretty much zero reason to run the backend on and serve the frontend from the same box? And it's not like an SPA really needs any power - if you're just serving up bundled JavaScript and some HTML, anyway.

This is exactly what I do. My product consists of 4 base docker containers. 1 is a nginx reverse proxy to 2 of the 3 remaining containers. 1 is a database container. The 2 other containers hold the API which is "monolithic" app. The other container is just bundled html/css/javascript.

This allows me to scale the API to handle load independent of the app itself, since that's just static assets, it doesn't need it as much.

I have basically a docker image that basically is just an nginx reverse proxy that forwards to a local socket. Then I can use it with whatever app I want, (usually node) and it provides a slightly "harder" (request buffering etc) http interface to the outside world.

This was exactly why I chose the workload I did. I think independent scaling and possibly enforcing decoupled behavior in the development teams are the only real reason TO decouple.... The rest are all descriptions of things that "might" make decoupling valid :-)

> The only reason I see for breaking the app this way would be to enforce low coupling.

Which is a big deal...

Microservices enable easier code re-use and division of labor across teams, especially when persistent data is involved. It is a technical solution to a people/economic problem and not the other way around. At a high level, I think of all the popular APIs powering the web (Stripe, Google Analytics, Twilio, Intercom, etc.) and how they enable developers to do their work more efficiently.

"Buy vs build" was mentioned as a promising attack on accidental complexity in "No Silver Bullet". If you design your microservices in a way that they're usable in different contexts and don't build at all when a paid or open source solution is available, you will boost your productivity.

This is again, getting to the right answer for the wrong reason. You could keep these code bases separate whether they run in the same JVM in one container or in separate JVMs in different containers. As long as you enforce communication over something like REST, you will never have a problem.

In fact, the application was already written this way and it was never enforced. This makes no sense to enforce if it wasn't already broken.

Fair point, but it's an existing application. I sort of assume it has low coupling already. But, if it already has low coupling, one might fairly ask, why do I need to enforce it?

For some future optimization?

Because applications that don't structurally enforce low-coupling are touched by many hands over time. All it takes is the occasional blurring of the line to end up with extreme-coupling down the road.

My argument is, if two different processes are doing completely different things in the same JVM, I am skeptical that they will share memory pages well, etc. I simply don't know without testing, but I don't have any reason to believe they would.

On the other had, if I was running multiple threads that all cached data from the same data store, it is quite likely that scaling up threads in the same JVM would have a massive advantage over splitting them into multiple JVMs.

Also, in the comments I admit that over 10K applications on 1K machines, it's probably better 99% of the time not to use 10K JVMs. There would be a huge CPU/Memory overhead....

When we first started containerizing back end components a couple of years ago I spent a lot of time thinking about containers this way: how many pieces of my app should go into each container? After doing it for awhile and paying attention to evolving best practices I realized that I was looking at it wrong. Containers are not a large-granularity box into which servers are packed. They're sandboxes for processes. Individual containers have very little overhead. They are not analogous to VMs, instances, or anything at that level of abstraction. They are absolutely more analogous to chroot. So for example, in the beginning I might have created a "logging service" with logstash, redis, and kibana running in a single container. Now I would have one container for each of those processes, assembled into a pod or cluster. This becomes nearly a mandatory design pattern when you start using orchestration systems like kubernetes, because they work best and most flexibly when the lifecycle of a given container matches the lifecycle of the single process it hosts. If the process dies the container transitions out of the running state and is restarted. This gets a lot more complex and harder to reason about if the "readiness" state of the container relies on multiple processes running in its sandbox.

Exactly, every container developer configuring their own init process/process watcher is a recipe for zombie systems that are sort of running but totally broken.

Funny. Someone please let Redhat know that Struts is framework [1] built on top of Sun's JEE Web Profile [2].

[1]: http://stackoverflow.com/questions/1636238/difference-betwee...

[2]: https://jaxenter.com/introducing-the-java-ee-web-profile-103...

aside/ Sun Microsystems really blew it on the pedagogical front. Hopefully future tech leaders will reflect on SMI's failure to take advantage of (imo) 20+ years visionary head start in design & architecture.

I'm honestly more amazed it was mentioned. I think most developers who are using some kind of framework for server side rendering are using Spring MVC.

I'm not really seeing the issue. Struts is built on Java EE (which was implied if not explicitly stated). The backend in his scenario was built explicitly on JAX-RS (a Java EE component). He could have been a little more explicit, but I'm pretty sure that wasn't the point of his article.

Posts like this are interesting, I felt it started out about "I'm going to teach you about docker" and then shifted into a piece about general technical decision making; and being pragmatic.

Docker has fast become my world: from local development and detailing of dev setups through to deployment of apps using ElasticBeanstalk's single-container Docker pattern.

The dev setup piece is seeming the most valuable currently. That you can land in a project that I've been in and issue `docker-compose build`, `docker-compose up` and have a fully functioning development server without the need of installing a half dozen dependencies locally really does feel like a positive.

As a devopsy-leaning person the "single binary" deployment it lends toward is beautiful too. Pragmatism always though: for many of our services the network IO impact is undesirable, so we use go (statically linked binaries) on bare metal for those.

Is this network IO impact still an issue?

http://stackoverflow.com/questions/21889053/what-is-the-runt...

Great question, not so great answer. I think the answer boils down to: "depends on your deployment needs". If you want to scale application across different machines, break it in pieces. Ditto if you want to replace containers on upgrade (hot swap). But if you only use Docker to simplify deployment of interconnected parts which never function one without the other - don't break it in pieces.

I know the last advice goes against the common wisdom, but in my experience this is so.

Around 4-6 months before you'll need to run multiple copies of your services in different servers/VMs for scaling or HA reasons.

"Multiple copies" meaning "more than 2", excluding master-slave configuration.

But at this stage, containerization is practically begging to be done, becoming a pressing need. If you have to ask the question, it is probably too early for that. :)

I am a bit surprised there is no mention of privilege separation and security.

…yes, if your application / service has good separation of code, configuration, and data, installs cleanly (as installer scripts can make this whole process difficult), and features a clean communication paradigm – it does make sense to break the application up / allocate one service per container.

Amen. Treat your software as the sum of it's parts rather than a bunch of parts that will do.

Splitting application into containers greatly increases isolation between components minimizing a chance of turning a bug into exploit.

For example, if a component can communicate over Unix sockets, then one can put it into own container with networking disabled. Similarly one can selectively choose which part of filesystem is accessible with read or read-write access.

As I read it, this blog isn't about microservices vs monoliths. Microservice doesn't mean "more than one process" or even "very small processes".

From the microservice perspective a single service is supposed to be a vertically integrated set of functionality all the way from data to the UI, instead of horizontally segmenting API services and UI. Horizontal segmentation doesn't give the same team-scaling benefits since it still takes significant inter-team coordination to deliver functionality.

So the example is really a single "service" that just happens to include multiple processes.

Actually Microservices is also about team size. It makes no fucking sense to break a Application into Microservices when your team consists of five members. Actually Microservices have their place but Monoliths, too.

Also Containers will some day have their place but at the moment they are just not ready for 70 % of the workload. it put's just too much complexity on your system. Also Microservices add a lot of complexity, too, which is easy too handle if your team has a certain size, but without prober tooling, management services it's really really hard to do it.

Embrace madness and be honest with yourself :

Never!

Use Go and build a single static binary that does everything and fck this fancy container technology , that split this into that, sandboxes it and and other security sht.

Disclaimer: I don't work at Google, RedHat, DHL or anything remotely associated with Go, Containers , shipping.

Only when you need to because it will save you time managing things. So... this is probably never for 98% of us that aren't writing Google sized applications.

If you have multiple applications, I'd definitely put each one in a container to beef security up.

I tend to agree :-) Hopefully, you guys read my comments in the comments section of the article. Quite honestly, I am skeptical how often it makes sense to "force" splitting up an application. I hope that was clear in my article...

For me, the answer for the given scenario was so straightforward and obviously "yes" that I wondered why a whole blog post was necessary?

This now has me wondering if I am a bit more drunk on the container koolaid than I may have thought!

Read the email thread that inspired me to write it and it will become obvious. In the original email, a user asked if they should split an application that was almost exactly what I describe in the blog post, into multiple containers because you "should" only run one process per container. That is a ridiculous reason. That provides zero business value in and of it's self.

The two main business values I have found from splitting something up is 1. independent scaling and 2. independent development teams. Other than that most is preference and philosophy. You can get to the right answer for the wrong reasons, it doesn't mean you were right.... :-(

How about... whenever your application has multiple processes? ;-)

Exactly this. Sometimes I will have multiple entry points (app, celery, &c.), but each process is always launched in a separate container.

I would argue, service, not process :-)

You could certainly make that argument, but I've struggled to think of cases where processes aren't the right point of container separation.

Nginx, Apache Prefork, Apache MPM, and PHP FPM all have multiple processes, and I would NEVER try to split up subprocesses of any of these into multiple containers.

It's quite simple if something communicates through shared memory, it should probably be in the same container. If it doesn't then that's NOT a reason to keep it in the the same container, but I don't see how it justifies the work of splitting it up....

Yeah, worker processes are the only thing I can think of. Even then, It is often more flexible to just run one process containers and have more of them. Then you can allocate containers more flexibly.

Absolutely for anything that uses shared memory for IPC, but then you basically have threads pretending to be processes. ;-)

Another thing that you should consider is that You will become the maintainer for several of the dependencies.

Reply on news.ycombinator.com