A Kubernetes Operator in Rust | Hacker News Reader

A Kubernetes Operator in Rust

thetechtrailblazer.blog

54 pointsalykafoury21 comments

I see "operators"--as in people responsible for keeping k8s infra healthy--are still not using "operators"--as in the k8s code--as much as they should. It's a powerful abstraction to extend a control loop into worker nodes. One customer we work with standardized around "operator or nothing"--i.e. never deploy anything in a cluster that is not an operator. Their infrastructure remains minimalistic, yet super functional and debuggable.

It's interesting to see the different views around it. I tend to agree that we tend to exclusively use operators to make Kubernetes speak the same language as our business. It's sort of like object-oriented deployments...can make some pretty power abstractions.

object-oriented deployment... huh... That's an interesting mental model that I didn't have. Thanks for sharing!

For me, it's a standard which works well for short-cycle control loop. Standards are never perfect, but the best ones are minimalistic and built out of real world experience. Operators check these boxes for me.

Not sure if you can share, but if you can, would love to know the type of business logic you moved into operators.

I see issues with going full on kubernetes-as-controlplane due to how operators do reconciliation. The represented resources are more or less eventually consistent which present challenges if you have workflows that need explicit orchestration and strong requirements for certain dependencies to be met before progressing. If the logic for a particular operator (quality varies here) doesn't handle missing pre-reqs or downstream resources being in a weird state gracefully you end up having to chase down tons of events and logs that are distributed across many components with no real way to quickly and effectively pinpoint where things went sideways (this comes with experience for most human cluster operators but most application developers are not going to know you need to go check the logs of these 5 controllers and check the status of their attendant custom resources to see what needs unwedging)

I understand Flux has some capabilities to pause or perform incremental deployment until certain dependencies are met, which is a step in the right direction.

I hear you, and am not advocating a full, one-size-fits-all, controller solution for control plane and workers. My take was mainly on using operators for short-cycle control loops inside workers which are inherently simpler in their business logic.

I do agree with you on deployment systems taking up more responsibility in understanding app properties and abstracting infrastructure away from app developers. However, it's notoriously hard to get deployment systems to do this correctly. At Amazon, this was a somewhat solved problem but took their best brains, many years of pain, and multiple iterations to get right. If you haven't already, I would recommend reading this thread from Joe Magerramov where he talks about the challenges of deployment systems and how many stars need to line up to get deployments done right: https://twitter.com/_joemag_/status/1587283479448150016.

Both Flux and Argo are doing splendid work to solve this problem. I just wish we can democratize the known wisdom, failures and patterns/anti-patterns from cloud operators, so others don't have to repeat the same mistakes. I'll see if I can put together some blog posts on this topic.

Nicely done! There are however, a couple issues with the code that are concerning to me if it's going to be used as a starting point by folks coming from Go — particularly around error handling and formatting.

Can you go into detail? I'd really like to know what you mean by formatting, but more important is the error handling part of your criticism.

There is also https://metalbear.co/blog/writing-a-kubernetes-operator/

I’d love to play with and explore this code. Please put a license on it.

Code without a license should be assumed toxic and if you’re paranoid (like me) shouldn’t be read.

This sort of thing feels like a good Apache or MIT fit.

Opened an issue. https://github.com/AlyHKafoury/Operator-rs/issues/1 hopefully the author adds one.

Author has already committed the change and now the code is properly stated as MIT-licensed. Thanks everyone!

Finally! A non-Golang operator!

Well done.

What's wrong with the Golang-based operators? Why do you prefer Rust to be used in the operator?

Diversity is good.

I think it's a breath of opportunity to hear that there is tooling for Rust to write Kubernetes operators, previously my impression is that there is no match for Golang when it comes to writing operators, and that has invited me to try and learn Golang and the codegen that comes with writing operators, but I never enjoyed trying to learn things about Golang, the language feels like an exercise in ambiguous minimalism to me personally.

> previously my impression is that there is no match for Golang when it comes to writing operators

Strimzi exists for some time and they use Java.

Have you tried using kube-rs? It removes many of the sharp edges of client-go, and is a pleasure to with with. It also has much tighter dependency discipline than the Go library.

There are other reasons to prefer Rust (sum types just to name one) but the quality of the Kubernetes client alone is enough to prefer it over Go in this case.

So what’s the usefulness of the above mentioned when library quality is like heaven and earth?

Nothing! It's just good to see some language diversity here.

kopf exists for python: https://github.com/nolar/kopf

Nice one

Reply on news.ycombinator.com