The costs of microservices (2020)

https://docs.undo.io/java/Microservices.html

[Edit: for which, by the way, any feedback is welcome - it's technically quite promising but real world experience is worth a lot!]

You just invented microservices, lol.

I think in these discussions, a lot of times people are taking past one another. If I start putting a JRE-targeted application together, I know I can eventually reach for isolated modules if I follow good internal practices, whereas if I'm in Python land it's pretty unattainable.

OSGi is the one that's deprecated and Java Modules is the one that can't actually provide that functionality yet, right? Or is it the other way round? Either way you get the point.

If you have to do X,Y,Z to get A done, then you'll do X,Y,Z.

If your language gives you a shortcut and now you can get away with X,Y then you'll drop Z.

And even if your language doesn't give you those shortcuts, if Cool Language lets you do just X or X,Y, you'll naturally want to use Cool Language. So, its a losing game...

KISS is great when applied correctly, but you have to be able to know the level of complexity you'll actually need for the problem at hand, and it is so so very easy to over or under engineer something...

Also JNI won't be isolated which occasionally might be a factor (more likely for stability).

On the other hand one could make colocated multi-process services to work around this while avoiding all the other complexities of microservices.

Hold on - the way you are phrasing this implies that Java Modules allow you to solve the version problem mentioned above. As in, your comment implies that Java Modules have a concept of version, and thus, allow you to pick the correct version of a dependency when defining relationships.

Java Modules explicitly DO NOT have that ability. In fact, the creators of modules went out of their way to discourage the idea of using modules this way. They literally introduced a warning that flags your module if you add a number at the end of it - specifically so that they can discourage the concept of introducing versions into the module system.

The way Java Modules (and Java Classpath before it) works is like this - if you have a module ABC, then that is the only version of ABC as far as Java is concerned. The correct version should be decided upon and provided, but once you hand that version over to Java and press compile/link/run/etc, the concept of version is not existent anymore, as far as Java is concerned.

To better explain this, every Java class has a unique identifier -- the module name + the package name + the class name. That's it. So, if I have version 1 of ModuleName.PackageName.ABC and version 2 of ModuleName.PackageName.ABC, there is no way for Java to disambiguate them, and thus, will throw a compilation/linking/runtime error, saying that you have 2 versions of ABC.

And to further expand on the warning point above, some clever developers tried to work around this by putting the version number in the name somehow (for example, module name = ModuleName1). To firmly discourage this behaviour, the Java developers who made modules released the abovementioned warning, so that this problem could be nipped in the bud.

To summarize, dependency versioning is a problem that Java (currently) does not attempt to solve. It's considered an extra-linguistic concern that is left to the ecosystem to solve (which you should interpret it to mean, they're letting Maven/Gradle/etc deal with this problem (for now)).

Finally, a few members of the Java team are giving some thought to maybe dealing with this problem in the language, maybe with a build tool. There is absolutely 0 confirmation that this will even be given serious effort, let alone released as a feature/tool. But the problem is definitely being considered by some members of the Java Team. In fact, a user on this site (pron) may be able to give some helpful context on this. This is my first comment on this site, and I don't know how to use it, so someone else can try and link him.

We just go from 'one' to 'too many' as equally unworkable solutions to all of our problems, and each side (and each person currently subscribed to that 'side') knows the other side is wrong. The assumption is that this means their side is right instead of reality, which is nobody is right.

The moderates are right, but their answers are wiggly and they're too busy getting stuff done to argue with the purists. But the optics are bad so here we go again on another swing of the swingset.

'Some Services' is probably right, but it varies with domain, company size, and company structure (Conway's Law). And honestly developer maturity, so while 7 may be right for me and my peers today, in a year it might be 6, or 9. There's no clever soundbite to parrot.

The best company I worked at had about 5-10 deployable per engineers on average and it worked really well. They were small, deployed almost instantly, dependencies were straightforward, etc.

Monoliths work fine too, it's just different tradeoffs.

I find this hard to relate to, the idea you have the discipline and culture to do microservices well if you can't do it with a monolith.

More likely is you migrate away from the monolith you never invested in fixing, and once you get to microservices you either call it a mistake and migrate back, or you have to eventually finally invest in fixing things.

Perhaps your microservice rewrite goes well because you now know your domain after building the monolith, that is another option.

I didn’t know how to troubleshoot Node, but I did know how to read docs. Suddenly I can troubleshoot Node. Hooray.

I concur with your sentiment, and would also be an absolute dictator for tech stack and workload management. We’re not using Node, we’re avoiding React if at all possible, every dev touching the DB in any way will know SQL, and no one is using Scrum.

How small is small? Even within this comment section there are people talking about a single developer being the sole maintainer of multiple microservices. I'm a strong advocate of (micro?)service architecture but I would never recommend doing the "all behavior is 100-line lambda functions" approach.

A horrible monolith vs horrible microservices is subjective, of course, but IMO having everything self-contained to one repository, one collection of app servers, etc. at least gives you some hope of salvation, often by building new functionality in separate services, ironically. Horrible microservices that violate data boundaries, i.e. multiple services sharing a database which is a sadly common mistake, is a much harder problem to solve. (both are bad, of course!)

There was a recent thread here where someone mentioned he personally was responsible for about four of their microservices...

It's absolutely true that postponing a decision lets you take advantage of more data and experience. But making a decision now also generates new data - and often does it better and faster: you're directly observing how your choice plays out in the real world. For cheaply reversible decisions it means that, when things go bad, you can go back and make a new decision, this time informed by knowledge of what went wrong with your previous choice. If you squint, it almost looks like time travel :).

Not every decision is going to be like that, but e.g. in software design, I would often delay deciding between several possible approaches to seek out more information (and/or wait for it to come from customer side), only to later realize that in that time, I could've prototyped most or all of the options, or I could've picked one and made progress and undo it fast when more information came - and that either of those two approaches would've let me to gain the missing data faster, while also keeping forward momentum.

There's a reason many programmers (myself included) need to repeatedly hear the mantra that goes: "make it work, then make it right, then make it fast". In my case, postponing decisions is often a kind of analysis paralysis, and I'm slowly learning that in many cases, it's better to just pick any option, "make it work" in the dumbest, most straightforward way possible, and then reevaluate.

But as I said, I am learning this slowly. My mind knows better, but my heart still wants to delay by default :).

In my case however, I ended up applying the "LRM" strategy to my own work, where I'm both the decision maker and the sole implementer. This is where I see my mistake. In my defense, the software development books that argued for delaying decisions did not warn that this applies to larger decisions in projects developed by teams, and may not apply to small-scale design decisions made by an individual contributor or a small team in the scope of a small piece of work. It took me way too long to realize that, for most of my day-to-day choices, the cost function is pretty much flat.

Why? Because it's a "good practice"? They have 12 people and no customers, they can almost certainly adopt a very aggressive developer cycle that optimizes almost exclusively for happy-path velocity. You'd never do that at 50+ engineers with customers but for 12 engineers who have no customers? It's fine, in fact it's ideal.

> with massive downtime as a result.

They have no customers, downtime literally does not exist for them. You are following a dogmatic practice that is optimizing for a situation that literally does not exist within their company.

I’m not convinced bad code would get merged more often if we didn’t require approvals. I am convinced we’d deliver code faster though, and that’s what I’m trying to optimize for. Your company and engineering problems are not the same as mine.

I agree with everything you say here, but honestly it’s quite ineffective. I wish we had found something better than unit tests (often misleading and fragile) and the ability of a qualified reviewer to maintain attention and context that a real review entails.

Not concisely in the general case, but in this case the difference is fairly straightforward - CI/CD doesn't affect the structure of your executing application at all, only the surrounding context. I don't want to spend the hours it would take to characterize architecture as distinct from implementation, but the vast number of textbooks on the topic all generally agree that there is one, though they draw the lines in slightly different places.

> I think what you're saying amounts to,

Very much no - my point is about the process of implementation. The services.. _do_ enforce boundaries, but the boundaries they enforce may not be good ones.

In order to successfully extract services from a monolith, you have to go through a process that includes finding and creating those domain boundaries for the domain being extracted. If it's your first time, you might be doing that implicitly and without realizing it's what you're doing, but under the hood it's the bulk of the mental effort.

The part where you actually _introduce a service_ can be anywhere from a tenth to half of the work (that fraction varies a lot by technical stack and depending on how coupled the domain in question is to the central behavioral tangle in the monolith), but by the time you've gotten the domain boundary created you've _already solved_ the original problem. Now you're facing a trade of "extract this well-factored domain out to a separate service application, to prevent its now-clear boundaries from being violated in the future". And I contend that that's a trade that should rarely be made.

Well, that's different.

My experience is that "leadership" often finds quality to be expensive and unnecessary overhead.

That's one reason that I stayed at a company that took Quality seriously. It introduced many issues that folks, hereabouts would find unbearable, but they consistently shipped some of the highest-Quality (and expensive) kit in the world.

Quality is not cheap, and it is not easy. It is also not really the path to riches, so it is often actively discouraged by managers.

Presumably you would still need tooling to enforce teams not breaking their API, and also to prevent people just modifying services to expose private internals that should not be part of the public API?

1) engineering needs a reason to embrace abstraction. Because the only thing that stops a new cowboy engineer is their peers explaining to them in what ways this isn't the Wild Wesst. One can assume if rules would benefit the team, they'd be doing them already, so why don't they? Maybe they perceive feature output velocity to be too high to risk changing method. Maybe decisionmaking power rests in the hands of one system architect who is holding the whole machine in their head so things that look complex to others seem simple to them (in that case, the team needs to spread around peer review signoff responsibilities on purpose, so one engineer can't be the decisionmaker and the architecture itself must self-describe). Maybe (this is how I see it usually happen) they were a three-person startup and complexity crept up on them like a boiling frog. Whatever the reason, if you're gonna convince them otherwise, someone's gonna have to generate hard data on how changing the abstraction could make their jobs easier.

2) If management has no idea what "prioritize quality" means (meaning no metrics by which to measure it and no real grasp of the art of software engineering), the engineers will interpret buzzwords as noise and route around them. Management needs to give actionable goals other than "release feature X by Y date" if they want to change engineering culture. That can take many forms (I've seen rewarded fixit weeks and externally-reported issue burndown charts as two good examples).

3) Engineering leadership needs time and bandwidth to do training so they can see outside their prairie-dog hole over to how other teams solve problems. Otherwise, they get locked into the solutions they know, and the only way new approaches ever enter the team is by hires.

And the key thing is: microservices may not be the tool for the job when all is said and done. This is an approach to give your engineering team ways to discover the right patterns, not to trick them into doing microservices. Your engineering team, at the end of the day, is still the best-equipped people to know the machinery of the product and how to shape it.

Really, you can't? Then I struggle to see how'll get anything else right. I've done it by using separate build scripts. That way only the interfaces and domain objects are exposed in the libraries. Now you lock down at the repository level access to each sub-project to the team working on it. There you go: modularity, boundaries, all without network hops.

I mean take your pick, anything open source be it desktop or web has a huge and deep dependency tree all the way down to libc.

Just wondering.

EDIT: Typos and desktop.

The team that maintains the service is also responsible for how that service is represented in the data warehouse.

The data warehouse tables - effectively denormalized copies of the data that the service stores - are treated as another API contract - they are clearly documented and tested as such.

If the team refactors, they also update the scripts that populate the data warehouse.

If that results in specific columns etc becoming invalid they document that in their release notes, and ideally notify other affected teams.

So, here are some concerns that affect performance, they mostly come from the need of various translations done by either iptables, eBPF analogues, arpatables, DNS server(s). If you read about benchmarks of Calico and Cilius, you'll see that they concentrate on performance of eBPF code necessary to do all these translations. My claim about performance tax is based on the idea that w/o Kubernetes you wouldn't need such translations (but, of course, you could build your own version of Kubernetes networking with a lot of software-defined virtualization, in which case you'd be in the same boat).

> Kafka

Is as horrid as it sounds. It's awful performance-wise on all counts. I'm not sure what point are you trying to make. Can you choose a better example? I mean, Kafka performs very poorly in all configurations, so, to think that it can be a good example of software that wants to achieve good resource utilization is just bound to give you bad results.

> You get isolation at the service layer. I don't see why that would be contentious, it's obvious. If you're saying you want more isolation, ok, you can write your code to do that if you'd like.

You either genuinely didn't understand what this is about, or pretend to not understand something really simple. "Micro" in microservices means that your services are small. There aren't any meaningful isolation tools or approaches when it comes to microservices, because isolation at the level of service doesn't matter / is trivial to achieve by many other means / is not a problem in real-world programs.

> Yep, the actor model is very low level.

This is simply a nonsense statement. Low on what scale? Your answer reads as if it was generated by a chatbot. I.e. words from the same general domain strung together, but make no sense.

> Microservice architecture has been far more successful than actor based systems

How did you count? How do you even tell if something is a microservice-based? This is just as absurd of a claim as saying "78% of enterprises use Kubernetes" (I believe I saw this unfettered inanity on cloud-native foundation's Web site). How do you tell if it's successful? What if actor model is a more generic description which beside other things, also captures microservices?

I mean, in more simple language, this is talking out of your rear. It's not a real argument anyone should pay attention to.

One is much simpler than the other. Artifact management is surprisingly complex, it only looks like it works great while you aren't managing many versions and acting more like a monolith, just spread across repos/deploy points.

I'm gonna have to disagree with this. Often when languages add features to their core, it allows for a very different (and often better) approach to writing code. Think Lisp with first class functions, or Rust with the borrow checker. Why should concurrency be any different? This feels like a Blub moment and I would highly recommend you give erlang or elixir a try, just to see what it is about.

Been free in my IDE since I compiled the code...

"E2E integration tests"

I can mostly run these on my machine, no need to stand up a cluster to get it running (bonus points if you virt multiple machines on one and still don't need the cluster for complex distributed scenarios).

"you will need some other kind of persistent message queue"

var queue = new Queue(); // then sometime later... queue.Save(); // or queue.EmplaceAndSave(); ... queue.Pop();

Trivial and didn't need a server to set it up.

Lots of tech stack stuff simply disappears without server boundaries and the like to get in the way. There is other tech you have to deal with, but at the small scales it mostly doesn't apply. Which, you as a dev are usually only working at small, testing scales, so you don't usually need to support it.

If you want to enforce a clean interface between an in-process client and server (i.e. a piece of code calling a library interface), then the best model is to think of it as a message passing system, where once a payload is passed from the client to the server and vice versa, the other side should not be able to witness changes to the payload. An immutable payload in this context is the same as the json that goes over the wire between a client and server.

If you really wanted an in-process application to look like a microservice you could take the added step of forcing a serialization / deserialization step on both client and server. I've seen frameworks that do this. But I think immutability, if it can be enforced, is a practical way of solving this problem and far less complex.

Inheritance is a hazier point I was making, in hindsight, because you could use inheritance for data modeling, which is quite fine in some situations... so I think it's effectively subsumed under the "separating data from logic" argument: which is that if you're passing behavioral inheritance from a "server" to a "client", then it gets harder and harder to predict how that client is going to use it and thus it's harder to reason about the functional boundary between two pieces of code. But it's a hazy point because I think the larger and more important point to make, as you point out, is that you simply shouldn't (in most cases) pass any kind of behavior between client and server--just data.

Another approach to look at (besides taking inspiration from Smalltalk) is the actor model, where it's incredibly clear that any communication between modules in an ecosystem is immutable messages. In fact a side effect of the actor model is that it gets pretty easy to move things from in-process to cross-process because most activity in the system is already performed through message passing, so you can just start channeling messages from Actor A to Actor B through a serialization/networking layer rather than in-process if you want to deploy them separately for reasons of convenience or computational needs.

Though I'm seeing a lot of convergence between environments over time, which makes me think we're all headed towards a nicer future.

For example in Scala, which does a lot of type inferencing, it's typical to tell the linter to require that public methods have explicit types, even though the compiler will reify them at compile time anyhow, because that makes the interface much more robust to refactoring. Meanwhile in a more dynamic environment, in Python it's getting more typical to use type annotations, and, similarly, to especially use them on functions that define a reusable interface.

I figure that the ideal languages in the future have module-level systems where they can define strict interfaces across modules, but then get as crazy and dynamic as they want inside of the modules.

Say you have Libraries A, B, and C, where B and C depend on A and not one another. You can have B call into C via a service, just as you would in a microservice. Now B and C can be versioned and deployed independency. You can also deploy updates to Library A incrementally, tying it to the B and C deployments.

If you are literally pushing different features that are in fact unrelated, you don't even need to worry about B calling into C, you can just partition your app into different modules, deploy them separately, and use a load balancer with routing rules to arbitrate between the deployments.

I like having this type of environment because you can make fairly quick and easily-resersible decisions about whether or not different parts of the codebase are deployed differently: sometimes it's compute and hardware requirements, sometimes it's because you want parts to be stable and other parts more experimental and volatile.

The microservice argument isn't addressing this type of deployment scenario: it's suggesting a shared-nothing or shared-little architecture across services.

git revert <bad commit>

I suppose in the end it comes down to the classic issue of needing to pay down of tech debt, which is true in any software ecosystem.

Both have boxes of stuff and the stuff talks to other stuff.

Logistically its a bit easier to scale the one where the boxes are married to closer to the hardware abstraction (microservices on instances) versus the one where boxes are married to the software abstraction (threads with memory), for the same reason one (monolith) is a lot faster (dev/process/latency) (at small scales) than the other (microservice).

You can scale both, really its mostly about fights about the tooling, process, and where your sec-ops decided to screw you over the most (did they lock down your environments or did they make it impossible to debug ports/get logs).

Practically, AWS is expensive, and they're bloated. Cluster environments that let you merge 1000 computers into 1 big supercomputer and have 1 million cores/terabytes of ram, come with different technical challenges that not as many people know how to overcome, or expensive hardware bills.

So I'd say if someone tells you it "has to be" one or the other they are blowing smoke. Micro-services were recently the hip-new-thing so it makes sense some really really bad nonsense has been written in them so people are rediscovering monoliths (and realizing the microservice people were snake-oil salesmen). In 10 years we'll realize again that some monoliths are really badly written and some people without a clue will re-write them as microservices...

> The monolith crowd claims it's easier to change/rewrite code in a monolith because it's all one big pile of yarn and if you want to change out one strand of it, you just need to know each juncture where that strand weaves into other strands.

I'm saying, in the monolith situation, with proper static analysis tooling - which even languages like python and ruby have nowadays - you don't "need to know" how all the strands weave into the other strands, you rely on the tooling to know for you.

And in my experience, static analysis tooling for navigating across service boundaries is, at the very least, far less mature, if not just entirely non-existent.

A proper branching strategy is not a technical solution but a social alignment of people contributing to common product. And it doesn't matter if they are contributing on different repositories or on the different branches. The technical complexity of aligning interfaces is the same in both cases.

"It's often not as easy as you make it sound."

No, it's of course not easy. But the costs in splitting teams up into different repositories and aligning their interfaces is not easy either. My point is they are both the same - just expressed differently.