Scaling Your Org with Microservices [slides] | Hacker News Reader

Scaling Your Org with Microservices [slides]

bridgetkromhout.com

68 pointskiyanwang50 comments

If anything is trendier than microservices, that is bashing microservices.

Before this comment pool spirals into that, I'll just leave this: It all depends on your project.

Did you have a bad time with microservices? Maybe the implementation was wrong. Maybe the project doesn't fit the model. Heck, maybe the project would STILL be screwed under a monolithic approach. There are so many variables to what classifies a success/failure in the microservice world (like most other things in software development, actually).

What's important is to not have a polarising opinion: "microservices suck", "microservices should be used for everything". Analyse your requirements. Imagine your professional project a few years down the line. Educate yourself on the subject (there are books on the subject!). Make. Sure. It. Fits.

I've personally had successes and losses with microservices, just like I had successes and losses with other architectural styles.

I've taken a middle of the road approach often I've often seen recommended. First build a monolith then split it apart logically when it makes sense. Not really 'micro' but my application with 6 medium components is easier than with one monolith, and easier than 20 micro.

Of course if I go to an interview I'll say I built microservices because it's a buzzword/checkbox.

This. Build monolith or SOA then shard off the pieces that are bogging down the application, then node them out.

The thing with microservices is that each one comes with a real cost, an amount of friction that gets contributed to your overall process flow. At some point, this frictive energy becomes intense enough that it jeopardizes the project.

I think "microservices" is a sane architectural pattern for some applications. The problem is that people polarize into one extreme or the other, and many attempts at a microservice implementation that I see would more rightly be classified "nanonservices"; things are broken up, almost at random, into tiny, disjunct, co-dependent pieces. This is a total nightmare scenario, and a monolith would undoubtedly be better.

I recently completed a "next generation" architecture prototype for a client using microservices (that part was specified by them).

I developed a dozen microservices to mimic aspects of their existing applications and used Kafka as a message bus. The services rarely spoke directly to each other, but instead listened to events fired into the message bus and updated accordingly.

My biggest takeaway - the client should not use microservices, but could learn some other lessons from the prototype. Microservices make many things harder and the only benefit to them, in my opinion, is they can scale well. But if you're not working at Netflix scale, they're more trouble than they're worth.

What I did recommend: - Build new applications API-first. By that, I mean your front end UI (assuming you have one) communicates to your backend via an API that can be exposed to the world. And assume it WILL be exposed to the world.

- Avoid bloat in your applications. Stop adding every possible feature to your existing app, and pay attention to when it's getting too big. This is mostly a gut/intuition feel, but you can tell it's a problem when your developers start to look like a deer in headlights when you discuss a new feature.

- (optional) Do create an enterprise message bus, and each application can fire events into the bus. This will help future-proof your organization, as new applications can be built to listen for events without having to understand the other systems.

I have yet to see a good technical implementation of microservices. It always seem to get messy, inefficient and hard to organize.

Everything seems so hard. To follow and debug requests for example. How do you do that in a good way? Running a local microservice application is another.

Maybe microservices is good for very large corporations, but even then I am not really convinced since it seems easier just to learn one bigger codebase that is structured well rather than digging in to many different ones that may be writter in different languages etc.

The hype is so big and the benefits are very unclear. At least to me it seems like there are only negative aspects of doing microservices.

I always tell people: Microservices sacrifice everything at the alter of scalability. Everything but scalability gets harder, not easier. You point out some good examples of places where this is true.

You should not be doing microservices unless you need that scalability. If you don't, regular SOA or even modular monoliths work fine. Microservices live in the place where those break down in scaling. That's a very extreme place, and so microservices are an extreme response.

As an example, let's take scaling services over a hybrid cloud. You have your own metal for the baseline request volume, with the capability to provision more capacity from a public cloud. A modular monolith is obviously out of the picture at this point. You would have two separate monolith instances, and would have to consider how and when to pass work between them... Possible, but at that point you're basically inventing a service architecture within a monolith, so why not just move to SOA? If you already have a SOA system, you still have work: Which services to deploy to the cloud, when to deploy them, how to recover and load balance to them, etc. Specific answers to these decisions are what led to what we consider modern microservices: What to deploy is the smallest unit possible, so that you can keep scaling as fine-grained as possible. Every request is routed through a discovery and load-balancer to any and all available instances.

I disagree, microservices can offer many distinct benefits over a monolith. Yes, fine-grained scalability is a huge benefit, but there are many other advantages as well. I was going to list a bunch but honestly they can be found on google quite easily. More importantly though, is what I think is the key to making microservices deliver on their promises: Well defined and designed API's.

Designing good API's is important for all software, but is critical to making microservices work well; and it is not always obvious how to do this well. I can understand why so many people say you should start with a monolith and then break it up into microservices: With the monolith you have already accumulated a ton of domain knowledge so you can break apart the service-boundaries much easier. However, it can still be tricky to do this because not ever module in your monolith should be a service, infact it is likely that most will not. Microservices are a different level of encapsulation than a module. Sometimes microservices should done one thing well and with authority, sometimes they should do many things within the same domain/context, sometimes they should do an entire class of things to their data source, etc. Designing good API's forces you to understand how execution and data will flow through your system and how you can effectively and cleanly delineate between them.

IMO, the most important thing for a microservice, regardless of its size, complexity or role within your system, is it's API. When you have a well defined API you can encapsulate and abstract that entire service from the rest of your system, which is a wonderful thing. This microservice can now be deployed, updated, scaled, refactored or even rewritten entirely without any of the other services in the system even noticing. It brings a tremendous amount of freedom to managing your software platform. You no longer need to worry about things that don't concern your service, just fulfill your APIs and life is good!

I don't disagree with you, but I do disagree that you need microservices for those benefits. Just regular SOA gets you well defined APIs and contract-based interactions. Pretty much every touted benefit except granular scalability is a property of the underlying service architecture, not microservices.

After reading this thread I had to read this [1], which I found helpful in understanding the discussion. I think in some cases it is just more natural to write microservices but at the end of the day if done right the end user won't really know the difference, it's all just SOA(?)

[1] https://stackoverflow.com/questions/25501098/difference-betw...

Here's some things off the top of my head:

* Single business function. Microservices don't have to be small, but all their calls should facilitate s single business function.

* Automated deployment. Services are deployed and undeployed live based on load.

* Unknown targets. Services are allocated and deployed to machines at runtime. The machine itself may even be allocated on demand. So service discovery is absolutely required.

* Stateless. Services instances can be spun down at any moment, so they can't be holding transaction state.

Particularly the comment on that SO question that begins "There is no difference..."

So many names for the same thing (that hardly anyone can agree on what exactly it is).

No wonder the doctors and lawyers and accountants take all the money.

I have worked on a project that had millions of users which began to implement a microservice architechture over a monolith. It was completely unnecessary since the monolith could easily support a load of a lot more (we did test that).

I stopped working there (partly because they wouldn't hire anyone to work on the monolith) and afaik the monolith is still in production (or at least the majority of it) since the microservices took so long time to develop I guess someone pulled funding.

I have to say my experience with microservices is a bit limited, but every time I have seen someone do it I just think it is too complicated for very little gain. It takes massive resources to develop it and I am not convinced it's the design that makes it scale better.

As a rule, if you will need your transactions spread around different datacenters over the world (AKA, you are Google), you do need microservices. If you don't have those multiple active datacenters, or don't need coherence, you don't need microservices.

If your cluster of 7 database servers is still too slow when you increase it to 9, you probably need to take a look into database optimization, not microservices. If any of those numbers look laughable to you, you should laugh about microservices too.

What do you mean by scalability?

That's a great question. I mean ability to serve concurrent requests, or throughput. Microservices will typically worsen other metrics like individual response time.

I am not sure microservices are good for scalability in that sense. Microservices divide your application vertically, but for scalability you want to divide it horizontally. You don't want a separate service for each feature of your app, you want an application server that you can replicate as many times as necessary.

Microservices can scale both directions, same as other systems. You can allocate more resources to individual service instances (vertical), and you can allocate more service instances (horizontal). The later scaling should be done in an automated fashion.

If you look at something like Spring Boot, you will find that an individual service instance is a stand-alone application server. That is what is duplicated when a new service instance is created.

A system is scallable if when you add more hardware, you get more performance.

yes, but both latency and throughput are "performance", and microservices worsen latency.

Well... Yes.

Performance is highly application specific. It may be any combination of [throughput, latency, responsivity] x [maximum, minimum, average, some cut over a distribution, instantly measure on some important time]. And there are probably more possibilities, those are just the usual ones.

Microservices can make any one of them better, very likely at the cost of making other ones worse.

We had to do it, but it had nothing to do performance. Massive team dysfunction and differing skill sets made it so 3 or 4 people were working on our monolith and not very effective in getting things done. We had another 5 that were very effective on other projects that didn't know the monolith stack. After some discussion we built up a design that broke the monolith into 4 parts. This let the teams work more effectively, even if it's not as efficient.

Docker compose running some integration tests with the builds helps some of the integration burden.

> The hype is so big and the benefits are very unclear. At least to me it seems like there are only negative aspects of doing microservices.

I think there are two groups or organizations that benefit from microservices:

- actual scaling is required

- "we are too stupid or politically entrenched to maintain a modular code base without them, so we try (and most likely fail) to enforce modularity with microservices"

Of course, group 2 will only further entrench themselves.

There's a group 3 also that's roughly "clearly I need microservices on my resume, so we're using it".

The funny thing is that this is probably the best strategy for the devs. The people who have chosen to stick with monolith for good reasons will get no respect from the next employer. Whereas the Microservices guys will be hotshots.

Same for Hadoop over SQL or node over Java.

Hadoop and SQL aren't even in the same use case category.

Hadoop+Hive are in the same category for our IT department. Instead of putting stuff into a SQL DB they are pushing to put all company diagnostics data into Hadoop and run SQL queries against Hive. Now we have minimum query latency of of over 1 second even for the most trivial of queries.

In my experience, this is by far the largest group. Then they build a distributed monolith and cause a giant mess, like the OP was describing. I really wish people would jump off of the microservices bandwagon, because so very few on it actually understand microservices at all, or have the skill set to make them work.

Well, one issue is that everyone seems to have a different picture about what a microservice is and everyone also thinks they have the correct understanding.

Your comment reflects that perfectly.

And the fourth which is haphazardly ending up on them due to the way the organisation is laid out and not invented here syndrome. Its mostly an organisation pattern not a technical one.

> At least to me it seems like there are only negative aspects of doing microservices

You (and others) have cleanly expressed the downsides, so here's my version of why microservices are valuable:

- Code is forced to have well defined APIs; you can't reach into a library's internals and rip out what you need when that library is implemented as a separate process.

- You can scale only the parts that need to be scaled, when they need to be scaled.

- You aren't tied to any single language for your application.

- Horizontal scaling is easier than vertical scaling, beyond a certain point.

- You gain a lot of extra fault tolerance.

Of course, all of these benefits only come when the microservices are implemented well; however when they are done well, scaling and reliability are easy. Since both scaling and reliability directly impact the end user experience, they will be given more weight than the microservice tech debt.

> Code is forced to have well defined APIs

That is incorrect. Microservices do not force you to write good code. I have seen microservices done poorly where each microservice duplicates code from the others, and APIs are not well defined. For example project A has folders "api" "api2", project B has folder called "api", there's another project just called "api" which also has logic mixed in unrelated to APIs. Files named "server" that actually implement a client, etc.

It doesn't matter if your API is an http endpoint or a local function call, if you suck at naming things, your API will suck.

> You gain a lot of extra fault tolerance.

Also not true as a "fact of matter". You have to account for API requests between your own infrastructure failing. Adding more system boundaries reduces fault tolerance.

> Since both scaling and reliability directly impact the end user experience, they will be given more weight than the microservice tech debt.

But your competitor's monolith might run on one server. They might implement reliability with a passive failover server, and they might forgo the tech debt & focus on adding new features, which also directly impact the end user experience. Those features will be harder to shoe-horn into your micro-services if you get the boundaries wrong. All of the sudden you have a scalable "reliable" system no one uses, because your competitor has more features.

> all of these benefits only come when the microservices are implemented well

All of those benefits apply to writing good code in general (microservices or not). The only valid benefit you listed specific to microservices is scaling is easier. But you can still scale a monolith. Just deploy all code to all servers, but only run certain things on each tier of services. So again, you don't really need microservices to do that.

I think AWS X-Ray deserves to be mentioned here - it tracks requests through different services using tagging and provides visualization. AWS also has other useful tools like SNS and SQS for building loosely coupled microservice systems.

In the AWS case microservices are combined with other paradigms like Serverless and event driven computing. I would judge all these together and consider the overall benefits you gain from fully managed services, built-in autoscaling, independent parallel development and deployment of microservices, etc. Personally I would not go back to the old ways.

> To follow and debug requests for example. How do you do that in a good way?

I'm assuming you mean "requests that involve multiple microservices"? And the answer there is simple for following - the first one involved tags the request and passes that tag on to all the subrequests. Easy to correlate across your logs. Debugging can be done in a similar way with a header saying "dump debug for this request" that's passed between services.

Well yes, but when I use a debugger for example that would require me to first debug application 1, then 2, then 3 in order to fully go through the cycle. This seems quite... time consuming.

You should not need to do this with microservices. Part of creating them is implementing well-defined API's, which create concrete borders between your services. It is extremely easy to capture/log all traffic passing between your services, which you should be able to use to determine which service is misbehaving, and them aim your debugger there. If you cannot do this, then your API's are not doing what they are supposed to be doing.

> It is extremely easy to capture/log all traffic passing between your services

But you would have to create a development environment running the complete micro-services environment on a single machine. Or setup distributed/cloud logging & debug against production. Which is generally easier to do if you use a monolith. Admittedly tools like docker compose and vagrant can help with this.

If you rely on a stepping debugger, you're fscked anyway when you step outside of desktop applications and websites. Logs and trace dumps (tcpdump/strace like) are what works almost everywhere.

"correlation ID" is the common term. Has value outside of microservices as well. Monoliths don't always have a single log file.

Excellent point - and that's where I've always used it to date, as well (monoliths).

I have yet to see a good technical implementation of microservices. It always seem to get messy, inefficient and hard to organize.

One word: CORBA

Those who forget history are doomed to repeat it

Yes, CORBA is better than the HTTP services we see everywhere. Yet, it does not fix the problems the GP was talking about. In fact, SOAP is almost as good (it's just harder to use, more ambiguous and less standardized, but not by much), and people still easily create messy APIs on SOAP.

It's more a problem that developers that try to scale orders of magnitude too early normally not knowing what they are doing; and developers that do not know what they are doing normally do not design clean interfaces.

Maintaining structure as the codebase grows is a challenge on it's own, especially when it's so big that you have multiple teams and way too many developers working on it.

At work we have a huge monolith and several smaller services. Working on the former it's never nice and my productivity is a fraction of what I can do when working on the smaller and saner projects.

I wouldn't always use a microseconds architecture. I think the best approach is to start with a modular monolith till it reaches a critical size.

> ... that is structured well ...

Is there a way to organize a large program (let's say in Java) that allows one to easily extract specific components and move them to their own service as needed? I'm thinking along the lines of local message passing between Java "components" (not packages). With a message system in place, you could easily move a service wherever so long as it's able to communicate.

Have you got a look on Elixir/Erlang/OTP?

From TFA: "I don't think anyone should approach management as a thing they move in to permanently. It's psychologically disfiguring."

Such a great quote. The longer I only do management-only work, the more anxiety creeps in silently about not being able to do these things on my own, not being competent enough, being a fake etc. The moment I decide to solve some problem for myself all is cool again - and I'm more trustworthy for the team as well. The problem is that such an approach tends to create conflict of interest - as a manager you shouldn't interfere too much and allow the team to solve the problems. Maybe the notion of manager is a problem in itself. It's a tempting perspective, but as far as I remember Google thought so as well and at some point they had to step back and reintroduce classic managers again ( https://hbr.org/2013/12/how-google-sold-its-engineers-on-man... ).

Incredibly cringey presentation...

"Hard things are HARD", etc. plus a bunch of memes and my little pony pictures.

Micoroservices are great for _RUNNING_ your app, monolithic are great for _DEVELOPING_ your app. I wish there was a way of developing like a monolithic and deploying like microservices.

Monolithics are good because you get a "stack trace" of an error! That stack trace is so much better than all that Ziplin stuff that barely work and takes so much longer to just see what caused the error!

Someone eventually is going to build an app framework that offers clean reusable abstractions for micro services in the context of a further-abstracted orchestrated and containerized provider-agnostic automatically highly available and scalable infrastructure, complete with backup, security, log analysis, and all open source and installable through a single remotely executable shell script.

to me this slide deck is "how to not completely fail at getting a minimally viable microservices arch going in your organization" which is good because apparently these lessons are not self-evident. But when are we going to get to "now that you're a healthy microservice company, how do you make it better? For you and your customers?".

I guess I would take the recent honeycomb outage as an example since charity majors in on the slides. When reading through the post mortem (which I liked a lot) the one question I never had answered was what are they doing to make sure 1 customer can't take down the entirety of their service? Assuming that failure is going to happen (via code bugs or bad actor customers)how do we better isolate individual customers from one another while still enjoying the benefits of mulitenancy? As we as an industry become more comfortable with microservices we must also learn how to better serve customers and mitigate failures by isolating and limiting blast radius, as well as deploying software in safer and saner ways.

Reply on news.ycombinator.com