Istio – An open platform to connect, manage, and secure microservices | Hacker News Reader

Istio – An open platform to connect, manage, and secure microservices

github.com

163 pointskumaranvpl67 comments

Open articleView on HN

Istio adds to Kubernetes:

- Automatic load balancing for HTTP, gRPC, WebSockets, and TCP traffic.

- Fine-grained control of traffic behaviour with rich routing rules, retries, fail-overs, and fault injection.

- A pluggable policy layer and configuration API supporting access controls, rate limits and quotas.

- Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress.

- Secure service-to-service communication in a cluster with strong identity-based authentication and authorization.

Istio on its own is powerful and flexible. It can also be hard to understand, setup, and manage, plus can be brittle.

(disclaimer: I'm co-founder of Vamp.io, a canary test & release system for DevOps teams that works with both HAProxy and Istio)

What Istio imho misses is a user-friendly way of setting up, managing and automating Istio configurations for microservices topologies to quickly achieve automated canary-releasing and A/B testing pipelines. Call Vamp a canary-releasing and A/B testing “control plane” for Istio/K8s if you will.

Basically the potential of service-meshes like Istio lies in "driving" it from higher-level metric/KPI-based automation systems.

We wrote several blog-posts on how to make use of Istio, starting here: https://medium.com/vamp-io/taming-istio-76fab339f685 (more blogs on Istio can be found here https://medium.com/vamp-io/tagged/istio)

Hopefully this gives some more insights in what Istio is, what it isn't, and how you can leverage it in your pipelines and architectures.

Thank you. Your comment is exemplary, just super-helpful.

I like the concepts behind istio. I want to be able to use istio, but I had it running in a test/dev environment, and it would just randomly start saying 503 for my virtual services. Removing and adding the virtual service would fix it for a couple of days. I didn't have time to really spend debugging it, and I didn't see any obvious errors in the logs. :-/

Yeah I'm always worried about using a product in the middle of an architecture that has 800+ open issues on GitHub.

To be fair, Kubernetes has 2137 issues open right now :)

I don't think there's any defense to this. If you look through the open issues you find stale issues, feature requests with no planned work, already resolved issues left open, "we'll get to it in a couple of releases", etc. Nobody is managing the queue.

I know it's a big corporate product, but this is a sign of bad project management, imho. At the very least they could offer to let a community member go through and clean it up if they won't do it themselves.

They have a bot which cleans up inactive stuff, marking it as stale, eventually closing issues stale with no activity. Anything worth keeping open is re-opened or a maintainer will tell the bot to un-mark it as stale.

It takes four months to close an issue with the bot. But before it closes, someone comments or removes the tag.

There are 195 issues marked stale going back two years.

It's only bad project management if you believe every issue from every rando on the Internet should be given equal attention. An open issue tracker is a poor metric for project health.

IOW, it's popular open source, the Github issue tracker is going to have a lot of noise. What would a community member do that the stale bot isn't doing? What open source project comparable to Kubernetes' scope and interest has an open tracker that exemplifies what you seek?

That's a useful thing to note. Kubernetes has a huge surface area, and you will definitely run into issues using it. That complexity is not the right choice for every org

To be fair, issue in kubernetes github are closed after 90+30d of inactivity, so even more "closed" issues were never actually resolved.

Check this, +4600 issues that were never addressed, just ignored https://github.com/kubernetes/kubernetes/issues?page=5&q=lab...

I've been monitoring istio since it was around v0.5ish and it is notorious for release regressions. Even in 1.0.x you're better off with 1.0.1 than later versions.

I had the exact same experience on my dev cluster. It's a bit of a scary black box if you don't have the time/experience necessary to debug it.

Istio also turned out to be way too complex for my use case, but I have found [ambassador](https://www.getambassador.io/) to fit my needs well. It is not quite the same, of course, but it covers the subset of Istio's features I need.

Can somebody explain what istio does in simple words?

I see that it hooks into micro/services communication, but why do we need that? isnt policies enforced by cni plugins? and can microservices communicate already with k8s and mesos default setups?

is istio an network abstraction layer for k8s and mesos? why would anyone want that, is there anybody who changes scheduling runtimes often?

istio can/will inject a network proxy container into your pods. The proxy intercepts traffic to and from the pod. It also ships with a set of telemetry and metrics services that the proxies send data to. Coupled with that, it has a whole suite of custom resource definitions for fine grained configuration.

I'd highly recommend reading the docs if you haven't already to get a better overview.

Kubernetes already gives you basic service discovery network policy and a network abstraction layer, but Istio is more about abstracting and enriching communication patterns on (sometimes) Layer 4 and (more often) Layer 7.

It allows you to use kubernetes manifest to centrally control policies such as: - Canary load balancing (pushing a certain percentage of the traffic to a canary release) - TLS Mutual Authentication - Retries with exponential backoff - Circuit breakers - Instrumenting your traffic for distributed tracing (e.g. with Jaeger) - Adding your own custom HTTP headers

All of that is transparent to your microservice. This comes instead of having to repeat the service mesh logic that does everything above in each microservice. From the microservice 's perspective they're just sending simple traffic without implementing retries, circuit breakers, HTTPS or instrumentation. Istio takes care of all the rest using sidecars and and ingress controllers.

Is the Kubernetes native (for lack of a better word) service discovery used in many real world use cases? The sense I've gotten talking to people who use kubernetes is that they fall into one of two camps:

- they're moving "legacy" infrastructure off EC2 or similar onto Kubernetes and their service discovery stack is already in place. They end up bypassing cluster IPs and dealing directly with pod IPs. - they're starting with Istio or something similar from the get go because the native service discovery is hard to grok and leaves out a lot of the nice things that Istio gets you for free (once you've set it up)

There's lots of overlap, but I've not heard of anyone making a strong argument for the native service discovery.

The native service discovery can be enough for small deployments with a handful of services where the complexity of operating a service mesh is not really worth it.

I've seen plenty of those from smaller startups.

Unless I've misunderstood you, if you're dealing with pod IPs instead of service DNS addresses you're not fault tolerant and you might as well not even bother with K8s.

Well you're not necessarily dealing with a single pod IP. Two versions of skipping cluster IPs are either having smart client libraries that are capable of multiplexing between different IPs, or using something like consul DNS to resolve a URL into multiple pod IPs and rely on client DNS to do the right thing.

That is untrue. Client-side load balancing is a common thing. The service construct also doesn't necessarily map to what external clients need. Kafka or Cassandra, for example, prefer client direct access to pods via headless services.

> There's lots of overlap, but I've not heard of anyone making a strong argument for the native service discovery.

Native service discovery works a lot better than using pod IPs, which are intended to be transient.

A standard ClusterIp service includes DNS records by default and it's good enough for simple use cases.

> They end up bypassing cluster IPs and dealing directly with pod IPs

Why the heck are they even using Kubernetes then? Just deploy a bunch of docker containers with the autorestart flag.

This is terrible design.

No, it isn't terrible design, it's a requirement. Try running any distributed data services without direct pod accessibility. This is why StatefulSets and headless services exist.

Lots of real world use cases involve load balanced TCP or UDP services, the service discovery is optimized for that.

If you need pod-direct accessibility, the StatefulSet headless service feature will give each pod a unique DNS name and auto-update it when the pod IPs change.

That said, legacy migration into Kubernetes was WAY Oversold by some.

It's not ELI5, but the best explanation I've seen comes from Envoy-creator Matt Klein: https://blog.envoyproxy.io/service-mesh-data-plane-vs-contro...

"Istio is the control plane and Envoy is the data plane"

"Ultimately, the goal of a control plane is to set policy that will eventually be enacted by the data plane."

Without Istio - 4 K8s pods each one gets 25% of traffic and that is the only option

With Istio - 1st pod takes 60% of traffic, second takes 30%, and last two take 5% each

With Istio - 1st pod takes users from /foo, second from /baz, third with user-agent forby and fourth with user agent kirby

Just some very simple examples. Istio has many more capabilities

Fundamentally, it’s about programmable application networking. Like, make my network change the protocol, or data, or re-route the flow the way I want, without having to change underlying network wiring, equipment, or router config.

It does this by using little out-of-process agents (Envoy) to sit next to your application as a sidecar. All application network flows go through the sidecars (which are localhost).

Istio is the config engine for all these sidecars, and for the overall gateway to your clusters. They call this a service mesh.

Istio in theory has little to do with Kubernetes or Mesos, except that it intitially assumed everyone will be running apps in Kubernetes (because Istio is from google). But It is designed to be independent of k8s as it can run with Mesos or Cloud Foundry.

Some example use cases:

1. Configuring (mutual) TLS for apps so each app doesn’t have to. This is huge, as keypair handling is a major point of risk and annoyance.

2. Auto-renewing/installing (mutual) TLS certificates across an entire set of apps

3. Intercepting and re-routing network flows for A/B testing , traffic shedding, or failure tolerance (circuit breaking)

4. Global load balancing with more ability to tweak the algorithm (beyond just DNS)

5. Tracing / visibility of application network flows across a WAN for debugging

Is it necessary? No. Is it useful? Yeah, it can be.

It abstracts communication-layer logic out of your containers. You don't have to code stuff like TLS, retries, failovers, circuit breakers, load balancing, etc. You code business logic in your microservices and declare all this communication stuff in a shared config file. This configuration is picked up by Istio and deployed into more containers that serve as transparent proxies implementing communication-layer logic.

Good summary.

It's a kubernetes service mesh. https://www.nginx.com/blog/what-is-a-service-mesh/

NGINX have since discontinued work on their service mesh called Nginmesh [0]

0. https://github.com/nginxinc/nginmesh#important-project-notic...

anybody has more insights on what happened here?

Because in modern-day cloud systems there is a lot of service-to-service communication, L7 becomes really important (i.e., you're building a distributed system). Istio (and other meshes) try to abstract away the nuances of distributed computing away from developers, by magically adding timeouts, retries, circuit breakers, observability, etc.

In practice, Istio is pretty complex and not for the faint of heart. You can think of Istio and its cousins (Linkerd, Consul Connect, etc) as the spiritual successors of the original Netflix OSS stack and Finagle.

Istio manages east-west traffic (between programs in your cluster) vs north-south traffic (in and out of your cluster). Kubernetes only provides basic service discovery with "service.namespace.cluster" DNS entries to communicate between apps in the cluster. The standard routing is just ip tables configured on each node. It works well but may be limiting for complex deployments.

What Istio does is replace all that with several components. The foundation is the Envoy proxy which runs as a sidecar to all of your pods and handles all the network traffic, providing much better performance, more load-balancing algorithms, advanced routing, retries, rate limiting, observability and tracing (at protocol level), grpc/http2 in both directions, TLS management, traffic shadowing, and custom filters that you can use to do whatever you want with the traffic.

There are also several "control plane" components which actually do the wiring and config management and work in the background. There are several alternatives in this space like Linkerd, Consul Connect, Kong, and even Nginx and Haproxy getting into it. Some are focused only on Kubernetes while others let you communicate across clusters and standalone VMs.

CNCF is tracking 10 service proxy options, 8 API gateways, and 4 service mesh alternatives: https://landscape.cncf.io/category=service-proxy,api-gateway...

The elephant in the room with Istio is that the more functionality you move into sidecars, the harder issues become to to test and reproduce. This is because Istio is transparent to applications and runs out of band so it is much harder to write tests to verify your application works with the latest Istio manifest that is deployed. In theory great for operators as they get more control over network comms & security but bad for developer workflow.

It’s hard to imagine something easier and more reproducible for developers than connecting to localhost and having your request routed to the correct endpoint automatically, whether you’re on a disconnected laptop or in production. That’s the power of the ambassador pattern, which Envoy enables.

When the request is routed correctly, yes, but the question is when it's routed incorrectly (perhaps because information you're providing to Istio is incorrect but you haven't realized it yet, perhaps because of bugs in Istio).

Running >100 Kuberenetes clusters with a team is already quite complex. Teams already struggle with Kubernetes core concepts and we have to make sure the time spent does not grow, because of some nice to have features that are not business relevant. Adding more complexity adds a lot of costs and most of the people looking into features like istio might see that it provides less value as promised by the advertisements. We use instead of service mesh is simple and flexible ingress controller, which can be used as api gateway https://opensource.zalando.com/skipper/. With simple annotations (core Kubernetes concept) we can do blue/green deployments, A/B tests, shadow traffic, you can build complex http routing as you wish and change everything in http in the request or response path. IMHO there is no much value left that service meshes provide: mtls and network policy maybe, everything else skipper provides. Blue/green with a simple kubectl plugin: https://github.com/szuecs/kubectl-plugins More advanced automated blue green deployments via https://github.com/zalando-incubator/stackset-controller. Skipper also supports several openapi providers and is used by several production users.

We add dns names for cluster internal communication through coredns templates to this api gateway. https://opensource.zalando.com/skipper/kubernetes/east-west-... You can adapt it step by step and stop without a bid all in approach as service mesh vendors promote.

If you have not enough you can get even more advanced and add hpa by requests per second.

We use all of these in >50 production clusters with regular shop and order traffic far beyond 10k requests a second. We are interested in features, but even more in stability. We are interested in the community and will fix most of the issues you will show us to make errors less likely happen for us, too. We don’t sell software nor support, that’s why we have less advertisements and don’t show up on every kubecon.

I hope to read you in our community channels: https://github.com/zalando/skipper/blob/master/readme.md#com...

How does the K8s / Envoy / Istio stack compare to Nomad / Consul from Hashi?

Consul is just Service discovery. Istio adds a service mesh in which you can introduce all sorts of routing like 1% traffic to a canary, dark deploy on another route, etc. also mTLS authentication and authorization. I think consul came out with Consul Connect to compete but I’m not too familiar with that. Also Istio just runs in your cluster effortlessly instead of needing some VMs for Consul.

Once you install Istio, all your traffic goes through Envoy sidecar proxies automatically injected to your pods. Consul just provides service discovery (like DNS, which Kubernetes offers by default). By proxying traffic, Istio lets you program how your traffic works (traffic splitting, policies, monitoring, ...).

Consul 1.2 supports service mesh deployments with a new feature called Consul Connect. It includes a sidecar proxy.

Good podcast episode about Hashicorp today: https://softwareengineeringdaily.com/2019/02/04/scaling-hash...

Oh lordy, I guess everyone’s making service meshes now.

The cool thing about Istio is that you do not have to modify your current K8s application. You simply need to reroute who your microservices call. So instead of calling another service or message queue; you configure all of your calls to call the envoy proxy. Then the proxy takes over and gets configured according to the control plane definitions.

One of the great things about Istio is that it is a super interesting and informative way to better understand the Kubernetes, Docker, Helm stack.

When I first started learning Kubernetes it wasn't clear to me how a small startup or company could leverage the power of Kubernetes because in reality you need some scale to make it worthwhile.

However, once you can play with, observe and better understand some application infrastructure which sits on top of Kubernetes like Istio the light goes on in your head and things get much more interesting.

Here is a really nice short simple introduction to Istio

https://www.youtube.com/channel/UC-zVlo1F3mUbExQ96fABWcQ

And here are the playlists...

https://www.youtube.com/channel/UC-zVlo1F3mUbExQ96fABWcQ/pla...

Thou idea is cool it product in itself. As you need a k8s person you would need istio person. It has lots of interconnected components, it own configuration primitives (lots of them) and if any of them misconfugured or wonky (looking at you mixer) you will have to spend days debugging it (mixer produce 10k of unreadable logs a second).

It’s great if you know it’s codebase. Not for general case.

The complexity of our world today in the software space is truly astounding. Agreed you would need an istio person, but that can be the same person who is the k8s person. If you have a company with large enough scale to be contemplating istio and service meshes in general; then if Istio makes your life easier in terms of dev ops in general, it might actually be worth it.

> Citadel - A centralized component responsible for certificate issuance and rotation.

That interests me, while I know nothing about istio, I'd love to have something like his for me personally, if I understand correctly this could centrally manage certificates used as password and their management?

Vault from Hashicorp (and its competitors) is probably better suited for what you ask

thank you for sharing, I will look into this, much appreciated!

Citadel does as you say specifically for secure communication between services (i.e., mTLS).

It basically automatically issues certificates to pods so they can use mTLS seamlessly when they communicate with each other. More of authentication/authorization piece.

Cert manager [1] also automatically provisions and rotates TLS certificates.

[1] http://cert-manager.readthedocs.io/

Citadel does this only for internal certa. It does not provision lets encrypt (at least didn’t several months ago)

thank you for sharing, I will look into this, much appreciated!

Does anyone else get intermittent 503 "upstream error or disconnect" when running istio? We get this fairly regularly and have to patch services to get it to run again. Would love to share ideas if anyone faces similar issues.

The promise of ISTIO is great, but the complexity it adds are not worth it IMHO. If you want to run the BookInfo demo app, ISTIO will work great for you. If you want to run anything else, buckle up. I'm sure there are people running ISTIO successfully, but there is little information out there about how they do it, at least not with gRPC.

We fell for the hype and tried running ISTIO for a production workload and spent months trying to get it working with little success. Our use case was to load balance gRPC traffic amongst a small number of services on K8S. Regardless of which load balancing algorithm we used or the amount of traffic that was pushed at it we could not get ISTIO to actually spread the traffic in any sort of reasonable pattern. For example, if we had 8 containers running 4 would get a ton of traffic, 2 would get a small amount, and 2 would just have heartbeats.

We tried to get help, but the RocketChat channel was a ghost town and got little response on the Google group. Lots of people asking questions but no one answering. Some friends at Google told us there's an invite-only Slack channel, but we could never figure out what secret incantation we would need to do to get access there. Evidently only people at Google, IBM and Lyft are allowed into that one.

I'll contrast this to the LinkerD2 (linkerd.io) project. We ripped out ISTIO and installed LinkerD2 and it just worked. We didn't have to spend days hand writing YAML files to configure our VirtualServices, DestinationRules, Gateways, and ServiceEntries. One command and the whole thing was installed and running. We injected the sidecar with their cool CLI utility and boom we had a service mesh that actually load balanced gRPC properly. The LinkerD2 dashboard is a very powerful tool to visualize how your traffic is being routed, response times, success rates, etc. When we had questions, we could pop into the LinkerD2 Slack and someone from the team got right back to us with suggestions and sincere offers to help. The CEO of Buoyant even reached out personally to us, a fledgling Miami startup.

We also had issues where ISTIO would route requests to the wrong micro-service at random. We would run the same request multiple times and get different results every time. After moving to LinkerD2, those issues completely disappeared.

LinkerD2 is still a very new project, but as far as we have experienced, it is far more production ready than ISTIO. I can only imagine how far along LinkerD2 would be if it was given a tenth of the resources that are pushing ISTIO.

Our experience was back in the Fall of last year, but we tried ISTIO again last week and had the exact same results on a new installation. Once again, we had to pull ISTIO out and put LinkerD in and were able to get the system functioning immediately.

ISTIO did do a great job of load balancing HTTP/HTTPS traffic, it's just gRPC that is a long way off from being ready for prime time. Maybe if you have an army of consultants at your disposal it will work for you, but as a startup I'd stay away.

Another alternate for Istio would be to use a custom xds-server (control plane) and Envoy as sidecars.

With few caveats

- Can become a configuration nightmare if not managed properly

- Not as sophisticated as Istio

On the positive side

- There won't be a lot of moving parts as in Istio.

- Envoy has excellent documentation for debugging

- Get things done with all the perks of Envoy

I hope someone builds this out at some point, a no fuss alternative control plane for envoy would be great.

I wrote one https://github.com/tak2siva/Envoy-Pilot

Although my team is evaluating Istio, currently we are using this to keep things minimal.

So this is like Kubernetes is to Docker, just to Kubernetes ;)

Not really, Kubernetes is a way to manage Docker containers. Istio just supercharges the network of a Kubernetes cluster. It doesn't really manage K8s clusters.

One way of interpreting this simile is to say that as just as one technology has subsumed (become the focal point) another technology, so does a service mesh as a layer on top of a container orchestrator. A fifth layer as described in The Enterprise Path to Service Mesh Architectures --> https://layer5.io/books/the-enterprise-path-to-service-mesh

This seems very interesting but I'm also getting HTTP response codes. Also, how does this compare to something like Nodered?

TBH node red always came across as a toy / prototyping tool.

Istio seems to be used in production a number of places.

Edit: also node red seems to be more about coding and istio seems to be more about configuration.

Reply on news.ycombinator.com