Service mesh use cases (2020)

So, in my opinion, the questions are introspective:

- “Do I have enough context to know what problem those solutions are solving, and to at least appreciate the problem space to understand why someone may solve it like this?”

- “Do I have or perceive those problem to impact my infrastructure/applications?”

- “Does the solution offered by the use cases described appeal to me?”

If yes at the end, then one potential implementation is a service mesh.

A lot of these are solved out-of-the-box with Hashicorp’s Nomad/Consul/Vault pairing, for example!

We want to automate everything not related to the code we want to write. Service meshes sound like a good way to do that.

To me, it's a fairly big decision to layer something that's complex in it's own right on top of something else that's also complex.

As a company grows sooner or later most of these features become pretty desirable from an operations perspective. Feature developers likely don't and shouldn't need to care. It probably starts with things like Auth and basic load balancing. As the company grows to dozens of teams and services then you'll start feeling pain around service discovery and wish you didn't need to implement yet another custom auth scheme to integrate with another department's service.

After a few retry storm outages people will start paying more attention to load shedding, autoscaling, circuit breakers, rate limiting.

More mature companies or ones with compliance obligations start thinking about zero-trust, TLS everywhere, auditing, and centralized telemetry.

Is there complexity? Absolutely. Is it worth it? That depends where your company is in its lifecycle. Sometimes yes, other times you're probably better off just building things and living with the fact that your load shedding strategy is "just tip over".

Just as it’s foolish to create dozens of services if you have a 10-person team, you don’t really get much out of a service mesh if you only have a handful of services and not feeling the pain with your traditional tooling.

But once you get to large scale with convoluted business logic that is hard to reason about because so many teams are involved, the search for scalable abstractions begin. Service mesh then becomes useful because it is completely orthogonal to biz logic and you can now add engineers 100% focused on tooling and operations, and product engineers can think a lot less about certain classes of reliability and security concerns.

Of course in todays era of resume driven development, and the huge comp paid by FAANGs, you are going to get a ton of young devs pushing for service mesh way before it makes sense. I can’t say I blame them, but keep your wits about you!

I'm not a big fan of this approach; the two teams need to have a meeting and need to have a shared goal to implement the business's selected security requirements together. But sometimes fixing the org is too hard, so there is a Plan B.

If you’re running server less you already have 90% of what you’d get from a service mesh.

I will tell you that having seen what happens in big companies, baking distributed concerns into libraries always ends in disaster long after you’re gone.

When you have a piece of code deployed in 200 separate apps, every change requires tons of project management.

If there were 3 ways to categorise scaling (there’s more than this in reality) then they might be vertical, horizontal then distributed.

You’re describing an architecture that’s in the horizontal scaling world view.

You’re not in vertical because you’re using higher powered (but slower) strategies like active routing for comms between components where in vertical you’d have configurable queues but no routing layer.

You’re not in distributed scaling mode because your routing is assuming consistent latency and consistent bandwidth behaviours.

I don’t think one architecture to rule them all is a solvable problem. I’d heartily and very gratefully welcome being proven wrong on this.

Service Mesh is nothing new. People tend to call it differently back then. The key features it bring are: - Traffic shaping: mirroring, canary, blue-green... - Cross service observability - End to end encryption - Service discovery