Introducing Conduit – open-source service mesh for Kubernetes | Hacker News Reader

Introducing Conduit – open-source service mesh for Kubernetes

buoyant.io

184 pointssteveklabnik51 comments

I'm happy for the guys at Bouyant, they've been great to work with in the past.

There's a very large complexity payment inherent in the new "cloud native" architectures that fundamentally isn't worth it for a large swath of use cases. Conduit, at first glance, appears like it might solve a large portion of it. If there's anything the last 10 years have shown us as an industry, it's that convention and programmer ease of use are the most important qualities of a new framework or way of working.

Rails / Go / K8S dominating Mesos / Containers over Jails / Maven over ANT / on and on. ActiveRecord is great, and Rails routing is great. Together they are magical for what they provide a user. Envoy is great. Istio is great. But there isn't a simplistic way to unify them even though for 95% of the industry it appears their functionality should go hand in hand.

We'll see if Conduit can fill that gap.

Istio actually uses Envoy as the traffic proxying layer so what do you mean by unify them?

> Conduit's Rust-based data plane proxies run in less than 10mb RSS and have sub-millisecond p99 latencies.

I love seeing things move from Java/PHP/Ruby to Go (or better) Rust. Memory usage plummets.

It's becoming something we're increasingly hearing from production users. To some of them, low, stable memory usage is almost more important than some of Rust's other features.

Thank you for your work on Rust. It's needed.

For components you run alongside your applications, it's really great. I once worked on an infrastructure where we deployed a lightweight but JVM-based proxy in front of every app, and that was basically a tariff of 256 MB on everything we deployed.

At first glance, this looks like a direct competitor to Envoy[1].

I'm sure the documentation will improve as the product matures, but right now much is left to the imagination.

Could you perhaps go in to more detail on the differences between the two?

[1] https://www.envoyproxy.io/

Like Linkerd, Envoy is a proxy that can be used used to build a service mesh. Conduit is a service mesh that includes a proxy.

So, yes, Conduit uses something that competes with Envoy ;) But I would not say that Conduit and Envoy are equivalent things.

A service mesh needs a data plane (e.g. Envoy) and a control plane (e.g. Istio). Conduit has both in one product.

Istio runs on Envoy; I'm not sure how Conduit is different from Istio.

https://istio.io/docs/concepts/traffic-management/load-balan...

But how is Conduit better than the combined products? Doing two things poorly is worse than doing one thing well. (I'm not suggesting that Conduit does anything poorly, but it's not clear how it's superior in any way to any competing implementation.)

What they are presumably going for is ease of use, Istio uses a pluggable service mesh with Envoy being the most common one, and Linkerd being one of the alternatives.

I'm guessing they think Conduit can bring value by being an intergated solution out of the box, and I'm excited to see if they can deliver on that.

It’s more of a competitor to Istio, which uses Envoy to proxy traffic.

I immediately thought of https://github.com/snoyberg/conduit

I'm genuinely surprised they don't make a comparison to envoy or istio given the users they're attempting to target.

How does it work with ingress? Does it ship with its own ingress like istio?

We tried istio a while back and it was too slow and hungry. Reverted back to no service mesh and it worked fine. I'm gonna be trying this next!

The first release doesn't include Ingress support. This is something we plan on folding into our Routing API in an upcoming release. If you have opinions on how you'd like to see Ingress implemented, now is a great time to share your thoughts (in a GitHub issue, is probably best)!

Have you tried using envoy directly? That's what we run, and it works well. I'd be interested to hear about the kinds of problems you ran into (as in, "yikes, is there something we should be watching out for", not "marketing").

If you just want Ingress with Envoy, Heptio now has the Contour project:

https://github.com/heptio/contour

> One thing we’ve learned is that there are deployment models where Linkerd’s resource footprint is simply too high.

It does seem like JVM based products standard answer "Buy More hardware" is becoming untenable.

Hence the ongoing efforts for value types ,some kind of reiffed generics and AOT compilation, or do you think Oracle would sponsor those changes out of charity?

However from where I am standing, it is still business as usual, regarding Java and .NET projects.

This is something of a special case. Running Linkerd as a sidecar container — which is the pattern that's increasingly adopted in the Kubernetes world for service meshes and proxies — means you're going to run a large, memory-hungry JVM in every single container. Just guessing, but that's likely 300-500MB per container. It just doesn't mesh (heh) well with how Kubernetes is supposed to be run.

It's not surprising that Istio settled on Envoy, which is written in C++.

It is kind of sad that writing resource efficient software is considered a special case.

Not really what I meant. It's a "special case" because the sidecar container is a niche where the JVM doesn't work well. That doesn't mean you can't run JVM successfully elsewhere, which of course people do.

For example, you may be deploying a bunch of lean Go microservices that, at maximum, consume perhaps 20MB of memory each (peak). You can run 100 instances of said app and only consume 2,000MB, but add a big JVM that weighs, say, 200MB a piece, you now need 22,000MB of memory instead.

JVM startup time might be another factor, but I've not measured this.

Spot on. FWIW with some effort we can get Linkerd down to ~110mb. For some companies that's acceptable. Still, it's a far cry from the ~3mb footprint that the Conduit proxies current have.

At Pivotal there's a lot of interest in Istio. You might swing past the NYC (still in Brooklyn?) or SF offices and try to talk people into looking at Conduit.

How does this compare to Istio or Envoy?

Conduit will probably have quite a bit of feature-overlap with Istio, especially, as it matures.

However, I think you'll find that the projects will be quite different in the way that they prioritize features and operate.

Our goals for Conduit are to provide a minimal, production-ready, commercially-supportable service mesh in the near-term. Time will tell how each projects goals and constraints lead to different user experiences.

Istio runs on Envoy; it sounds like Conduit is an Istio alternative.

https://istio.io/docs/concepts/traffic-management/load-balan...

They should have chosen name that doesn`t immediately make users want to uninstall it.

Why? Conduit seems as good a name as any for a project like this, if you ask me. Am I missing something here?

"Conduit" is often refered to "Conduit toolbar" [1], which was known as a malware.

[1]: https://en.wikipedia.org/wiki/Conduit_toolbar

It’s been a long,’long time since I’ve seen it that I completely forgot, hah! Guess it’s not as bad as a name still then.

There is also a haskell streaming library called conduit: https://hackage.haskell.org/package/conduit

I came here to say exactly this. A Google of "Conduit" gets every competing dictionary's take on the word. A search on "conduit software" gets you nothing but frustrated posts of people trying to remove conduit malware.

If you think Conduit is bad, you should see the list of names we _didn't_ use! ;)

Number 17 will shock you.

The link to the tower library in the article is not working (https://github.com/BuoyantIO/tower)

Updated to http://github.com/tower-rs/tower -- thanks for the heads up.

Nice, checkout Ambassador from Datawire also.

What is service mesh? Seems that I need some more info to understand what is this for...

There's a company named Conduit. Possibly incurring into trademark violations.

But what in the hell is it? If your product is so revolutionary it defines a new market segment, you might want to bring the people with the money people up to speed.

It's an "open source service mesh." They're assuming you know what that is, and what their existing product (Linkerd) does in that space. I think that's reasonable for a blog post, and you can go look at their solid documentation if you don't.

This explanation was helpful to me: https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-d...

Had to google this myself, here's a link to their description of a service mesh.

https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-d...

Typically a mesh is a set of proxies that are colocated with each service you deploy. The proxies facilitate service discovery and secure communication with other services. They abstract this functionality such that each service doesn't have to individually (and poorly) reinvent that wheel.

Your service would connect to database:5432, but instead of going straight to the database it goes to a proxy and the proxy has all the smarts to find and connect to the database, even if it's in another data center. The proxy can also do it securely such that you can go over the public network if necessary.

There's a good chance you don't need this complexity, but if you are hosting 100's of small services across multiple cloud providers, regions, or data centers, this starts to become an attractive option.

I just force every application to use a standard reliable transport library. Didn't realize I built a service mesh startup by accident...

Great, another OSS project with a name that’s a commonly used word. Let’s hope they’ve checked there’s no existing projects with that name. (Spoilers: they haven’t.)

What I associate with the name is a malware browser ad/toolbar network a few years back.

Please don't release a toolbar ;)

Reply on news.ycombinator.com