Hacker News Reader

Comment by FridgeSeal | Hacker News Reader

> When you start using Kubernetes one of the first suggestions you'll get is to install a service mesh.

Who on earth is giving you this advice? Service-messages are squarely in the “you’ll know when you need it, and it’s not day 1” bucket.

Is this the experience people are having with K8s? Welcome to Kubernetes, here’s a service mesh gl;hf. If this advice is common, no wonder some people think K8s is over complicated.

To anyone not aware: it _does not_ have to be this complicated. Default ingress controller and a normal Deployment will carry you really, really far.

View on HN

I'm sad this is the top comment on an incredibly detailed and well researched article.

The context of the content is obviously for Kubernetes enterprise platform teams and not deploying your first nginx pod. The author also gives specific examples of why a service mesh is useful and in what scenarios it shines.

> The author also gives specific examples of why a service mesh is useful and in what scenarios it shines.

The author's entire point was money which is weird as it is peanuts compared to the DevOps salary of people needed to work with this complex setup. Unless you are in a startup and good in tech, in which case service mesh doesn't make a lot of sense. Why do you need encryption or metrics collection above ingress. Nginx does a good enough job for metrics.

It might come as a shocker for you but many medium-sized orgs that prefer to pay salaries for people who can do N things and just augment their capabilities with some Salas/paid service. But if the service is too costly, they will not allocate budget for it. Also, unlike many Salas/software vendors would try to make you believe, those solutions usually need tweaking & knowledge to be operated. And if any issues happen, swift support comes at a hefty premium. Also, not all salaries are SV-level, while these softwares usually have the same cost wherever you are in the world.

> It might come as a shocker for you

Thanks for the unneeded tone. But the solution that author likely uses is linkerd which is free for 50 users. And even then very likely it would be less than $1000/month. If it is too much for the org to pay, I don't think they need it.

Service mesh is not something a small or medium sized company needs.

> those solutions usually need tweaking & knowledge to be operated. And if any issues happen, swift support comes at a hefty premium.

Exactly my point. Base price is not something that is gonna cost them the most when using advanced tools like this. Even if it was free it would costed pretty much the same for any medium sized org.

Man, big corporations are often weird and way more complex than first (or tenth) look can tell.

You can have a guy banking 200k a year in base salary begging his superiors to have a fast laptop for work and compilation, and request is not even rejected, just hangs indefinitely. It would pay for itself within few days with increased efficiency. Or say Java dev fruitlessly begging for intellij idea license that cost less than 1 MD.

Been there, done that (in some way), now I just accept whatever boundaries are given, and let everybody know that due to this my work will take X amount of time. If they don't like it, escalate this so that management structure is aware of unreasonable expectations within given situation. If corp is so rotten that this all doesn't matter and its a long term situation, just leave such toxic place, you may feel scared of uncertainty but later you will thank yourself.

Health is a finite resource and leaking fast, you often don't notice it since warning signals come way too late.

> Thanks for the unneeded tone.

Sorry for that, it was unnecessary indeed.

> But the solution that author likely uses is linkerd which is free for 50 users. And even then very likely it would be less than $1000/month. If it is too much for the org to pay, I don't think they need it.

Didn't check the Linkerd plans in detail but the author says 2 grands per cluster per month. Now, even a medium org (like where I work), can have easily 10 clusters between different environments or different workloads. It's not that weird. And that would make the price already going to 20k a month which needs to be justified. Especially if you were using the same thing but not paying anything before.

Salas?

Autocorrected SaaS?

It was SaaS indeed, sorry!

K8s, service meshes, all this infrastructure complexity to me seems like more of a band-aid to bad backend practices and lack of standardization.

This kind of world arises if an org has, say 1000 engineers, or ~100-200 teams, and every team wants to do their own thing with accountability improperly mapped etc.

And infrastructure engineers turn into police with an obvious overemphasis on observability.

I guess a large amount of complexity in infra and wastage is done to hide human inefficiencies.

The flip side is that some of this standardisation lets smaller teams very easily get this functionality without the man-hours required to implement it in your application layer.

< Usual caveats apply, don’t bother until you need it, etc etc. Personally though, I’ve found dropping in linkerd, or Cilium CNI about an afternoon’s work, with no application code changes necessary>

Or there is a company mandate to encrypt everything due to the customer and legal needs. Then instead of rolling your own solutions you use an established mesh and move to other problems. The complexity is not there but your lack of understanding why someone may need a mesh. The last sentence is because you pre-judge the need and the platform it needs.

Yes I do this for living snd know what I'm talking about.

To answer your question, yes this is the majority of the experience and is almost a given decision in the initial setup, in part because of the security association. Especially in enterprises it's easier to just say service mesh than have to think the decision through.

Regardless of service mesh though, although it's a part of it, the complicated reputation is not undeserved and not solely due to meshes.

I have small cluster and the only reason I’d want service mesh is to inspect req/resp between services for better debug. And I’m not even sure they can do that.

Retries for free are good I guess, but not something essential. MTLS is something I don’t want at all.

Couldn't you accomplish that with tracing? You may not even need any changes in your application as eBPF maybe able to automatically instrument the application. Alternatively, If you use Cilium as a CNI, this functionality can come out the box. https://docs.cilium.io/en/latest/observability/visibility/

Exactly if you don't need the functionality provided don't use a mesh, but consequently if you then use it. Honestly not very complicated, but mTLS is very complicated done correctly.

If you don't hear this advice, you are simply not in the audience that routinely uses service meshes. They are quite useful in enterprise environments.

> If this advice is common, no wonder some people think K8s is over complicated

I mean, if even Google who were behind it in the first place, are saying it's complicated and have 3 levels of managed Kubernetes services, I'd say it's pretty clear to everyone it is indeed complicated. Some of the complexity is simply due to the complex problems it solves, some is footguns, some is layers upon layers of abstractions to glue around design deficiencies.

> To anyone not aware: it _does not_ have to be this complicated. Default ingress controller and a normal Deployment will carry you really, really far

You're absolutely right. However, Kubernetes is rarely chosen after a careful evaluation of requirements; it's more often than not because it's considered necessary or for resume driven development. In that case, service mesh is easy to tack on, regardless of need.

and default gateway API is going to let your ops people help you go far without adding a bunch of coordination over ingress.

its yet another tradeoff, it adds more security for the apps too, now your apps dont need to have tls stuff, you let the sidecar envoy-proxy handle that ect

Yeah I’m aware of how useful they can be, my comment was about for someone first introduction to K8s, they’re probably overkill.

yes a bit overkill for first use; deff want to tinker around with it without the extra moving parts at first; but i'd say if somebody is venturing into that space they should also consider security as well and should certainly try to get the added layers. a lot of companies miss out on security and implement it as an afterthought and run into more pain for everybody involved

Reply on news.ycombinator.com