Hacker News Reader

Comment by tynpeddler | Hacker News Reader

There are two big reasons to go to microservices (note that the exact definition of microservice can vary a lot).

1. Organizational streamlining. If the team working on the monolith becomes to large, then coordinating and pushing out changes quickly can become incredibly difficult. One rule of thumb I've heard is the two pizzas rule. If two pizzas can't feed the team working on a system, it's time to break up the system.

2. Horizontal scaling. If some components of your workflow require much more computing power than others, then it makes sense to break up your system to move computationally intensive tasks to their own services.

While there are lots of other decent reasons to break up a system, if you can't invoke at least one of the two above reasons, you may be shooting yourself in the foot. I think he's dead on when he points out that if you don't have engineering discipline in the monolith, then you won't have it in the microservices.

View on HN

I have this idea for a new framework/language. I'm sure if it either already exists or it's a dumb idea in practice but anyways.

You build a monolithic application. Everyone works on the same code base. Things are broken up into modules/classes/packages. From the programmers point of view it's just like working on a standard Java project or something similar.

The magic happens at the method and module boundaries. When the application is first started everything works normally. Methods call other methods using addresses. As the application runs some parts of it become hotter than other parts. At some trigger point an included process starts that spins up 1+ cloud instances. Only the hot code is deployed to the instances. If necessary the instance is load balanced on multiple nodes. You configure the triggers and whatnot as part of the applications config. The framework/language would either come with support for popular cloud services or allow you to create whatever system you need to create the instances.

My hypothetical language/framework would proxy all method calls and remap object instances to the new instance(s). If the extracted code cools down enough it is integrated back into the main monolith. At that point proxying is turned off and the methods use address again.

Using this approach you get the all the advantages of a monolith (interface compatibility checked by compiler, not needed EVERY service writing their own http code, etc). Of course you can't optimize latency as easily and merging is harder with monoliths. There's undoubtedly a hundred other reasons why this is a terrible idea.

Its not a terrible idea but introducing a network boundary adds all sorts of constraints and issues that a normal program flow can safely ignore. Network partitions dont happen within a local system. I’d read up on CAP and Fallacies of Distributed Computing. They’ll more or less explain the challenges.

>a normal program flow can safely ignore.

This isn't really true other than network errors are more likely than a machine getting shut down but you should really be writing your code as if something could go wrong at any moment.

It's a different class of error though. You can have code that ensures transactional integrity everywhere in the case of power failure, but that alone doesn't mean it handles network partitions, 429 responses, timeouts, token expirations, corrupt responses, etc efficiently or correctly.

This reminds me of something I find myself constantly explaining to colleagues again and again: You can scale a monolith in its entirety to handle elevated traffic in one of its "subsystems"; having code paths that aren't receiving traffic doesn't cost anything (at least in the architecture of the systems I look after).

I don't see the value in separating a monolith to allow independent scaling, unless there are wildly different performance demands across its components which makes reliably autoscaling difficult.

I've gone through this thought process too, but where I arrived is that it's kind of the tail wagging the dog. The whole point of microservices (or at least one major one) is so that teams can operate independently without having to use the same framework. Each service provides a fairly static API and other services write code against it, but under the covers each service is relatively free to implement that contract however they want, whatever language, whatever database, whatever scaling strategy, whatever version control, whatever deployment cadence, whatever hardware SKUs, all specific to the service.

So with a hypothetical framework like this, yeah it would make microservices easier (in theory, though there are lots of technical problems too) in terms of "look ma, I made microservices", but it wouldn't actually address any of the problem microservice architectures actually try to solve. So, tail wagging the dog.

Some of the technical problems stem from memory working different from API calls: they can fail, the overhead is much higher (shouldn't call in a big loop), can't pass pointers, global state may differ on remote machine. So an application model that tries to abstract away those differences is bound to have problems. Also deployment: remotes will be temporarily broken if the API changes, until the deployment has fully propagated; a team running a microservice takes a different mindset with respect to API versioning than does a compilation unit. And that's really the crux of the whole thing: microservicing requires an entirely different mindset than monolithing, and approaching one with the mindset of another will cause problems.

Why exactly do you want to independently scale a hot code path? If its hot, its already using most of the resources in your monolith. Take your monolith, distribute it to more servers, and it will reduce the load of your other servers regardless of which parts of your code are causing the load.

Sure you can do that, but it's not as efficient as deploying only the code that is getting hit. Also, it might be taking up most of your resources, but it might not be as well.

For example, let's say in an eCommerce application that the shipping calculator is getting hit a lot. You'd like to be able to scale this independently as a service, so you can handle all the requests without also having to replicate all of the other resources, such as the cart persistence, user sessions, etc. that are a lot more memory intensive.

> For example, let's say in an eCommerce application that the shipping calculator is getting hit a lot. You'd like to be able to scale this independently as a service, so you can handle all the requests without also having to replicate all of the other resources, such as the cart persistence, user sessions, etc. that are a lot more memory intensive.

Assuming you allocate different resources for it. If you're using the same instance type for all your microservices, you aren't benefiting from this. In fact, you're paying for resources you aren't using.

You might even be paying more - allocating high memory instances to high memory services, and regular instances to low memory services that don't use that memory. You might be able to get by with only regular instances if you distributed your services in a monolithic fashion.

In my experience most small teams I've seen aren't that specific with their resources. Unless something is obviously super high memory, like a cache, they tend to just use default instances.

I have worked with a service where we saved some money by splitting them up and specialized the VM SKUs like that. But it's far from as trivial as GP implies. You have to plan out the interface, shake out any shared data (shared in-memory cache etc), design it to not be so chatty, and as with everything perf-related, do lots of testing. So it's not like a GGGP's solution that automatically spins up microservices could "just work".

This was a pretty high volume API at Azure, and the savings after all was said and done was sadly only around 15K/mo, so it'll take a while to pay for itself. So, I'd say for the average website this should not even factor into consideration.

(The other thing I forgot to mention was having a good consistent hashing strategy in the load balancer, so that the instances of each service consistently speak to the same instances of the other service, even as machines are spun up and down, but still maintains an even spread of load. This helps greatly in terms of allowing each instance to cache only the data for requests that target that instance. With a round-robin load balancer you end up with a lot more variation in your incoming requests, and thus require either more memory or expect more cache misses. That was probably the hardest part of the project, since the built-in load balancers don't have the specific functionality we needed).

I don't personally have much experience on this area, but something as valuable as this holy grail of automatic parallelization (be it threading or distribution), i.e. "just write the same good old code and the system will be smart and parallelize it" is bound to be something that people have been trying forever and failing, at least on the general case.

To begin with, anything with side effects is mostly a non-starter. You would need some way to annotate the boundaries where side effects can not happen, and this already requires considerable refactoring effort.

Even if you ignore this and suppose you are working with a purely functional language (this is probably what the "You just invented Erlang and OTP " meant), the overhead of the "smartness" can be pretty big. If you see a "map" over a list, you know you can parallelize it, but which one of the 100 "map"s should you really parallelize? If you try to be too smart, you will burn a lot of effort and may not get a payoff. If you try to be just a little big smart and you choose the wrong one you shuffle a lot of data for nothing.

In some way it seems to me a lot of today's big data systems like (surprise) MapReduce and Spark do exactly this, they offer ways to explicitly mark those boundaries where you want your program to be parallel (and require that you obey their rules about side-effects there) and contain some of this "smartness" for how to distribute your data.

Even in OpenMP you also have something like this, you can easily say "this piece of code is a task and the runtime can decide to run it in parallel", but you need to also tell it all the inputs, outputs, etc. (since figuring this out automatically is hard if not impossible) so it doesn't fit very well in a "big ball of mud" project.

In the (near?) future as computing power is more abundant and cheaper but the gap between sequential and parallel computing continues growing ever wider I can see those general "smart" approaches paying off, even if they are much worse than hand-optimized code. It only needs to be better than the average programmer.

So I've never actually done this, but I think this makes sense:

You can just structure your code in this way -- divide hard boundaries in your monolith. Segment things apart the same as they would be in microservices. Have a collection of methods for accessing each segment of code, and don't allow calling anything but that collection (API) from other parts of the codebase.

Set up monitoring/logging. If a segment of your code is using a huge amount of resources, it'll now be trivial to pull that segment out into its own microservice because you already have a defined API and hard boundaries.

You just invented Erlang and OTP

I would also add cost savings and ops excellence. Having individual business processes broken up into separate services can often lead to being able to tune individual services and allocate just the resources it needs. Especially when used with containers. It's also easier to spot offending commits.

A monolith is hard to tune and often ends up being a money pit.

Regarding point 1, why is coordination required? I think that continuous deployment, where you're integrating dozens of times per day solves this problem much better.

In large, distributed teams working on very large monoliths, it's pretty easy to end up with conflicting changes. In such monoliths, the testing process also tends to be long. So you run your tests, get a pass, but someone else merges in before you and there's a conflict. You resolve the conflict quickly (if you're lucky. Some conflicts are not easy to resolve), you rerun your test suite, only to find out that someone else has merged ahead of you again and you have to go through the loop one more time. And all of this assumes that no one breaks the pipeline.

At this point, many teams institute a merge queue. Which works only until more devs are added to the teams, which makes the merge queue very long and it can take several days in the ideal case to get things merged.

Reply on news.ycombinator.com