Hacker News Reader

Comment by water554 | Hacker News Reader

My conclusion is that using Ansible with Kubernetes is the WRONG way to go about it. So now instead of a directory of yaml or helm charts we have a bunch of ansible uselessly wrapping the yaml for application.

View on HN

The problem is when you're in a situation where "over here we need YAML document A, and over there we need YAML document B." You either shove a templating system in front of your YAML and fight with that or throw in the towel just use a programming language where constructs in your YAML map to objects.

Helm3 made it a solved problem years ago.

I think we need to burn this whole thing down and rethink it at this point. A huge amount of the total operational cost of our business is working around weird YAML problems and helm charts.

Had one a while back where quoted strings containing padded numbers in were used for something. "01" fine ... "05" fine, "06" fine, "07" fine, "08" kaboom. 01-07 were treated as strings and 08 was suddenly being treated as octal, which it's not and exploded.

Helm is a pile of garbage. k8s speaks JSON just fine as expected.

Try Tanka + Jsonnet to restore some of your sanity, also allows you to keep using Helm during your transition to a better place.

This is just the same problem in JSON, another loose schema’ed mess. Got my fair share of problems with that at the moment as well.

If you feel really strongly about getting type safety then there is Pulumi + Typescript though I really hate how it tries to be a "better" kubectl apply and fails miserably.

Definitely room for someone to make something better than Helm but I think you should give Jsonnet a shot if you haven't already.

> tries to be a "better" kubectl apply and fails

How does it try? How does it fail?

I don't see any interactive CLI commands that could be seen as a replacement for kubectl.

It's not about the commands. `pulumi up` is what you are looking for and the k8s resources that accompany it.

kubectl apply, especially with server side apply is effective just a mechanism to push the declarative definition of your k8s objects to the server.

Pulumi instead tries to manage the lifecycle of each of those objects itself when it really has no business doing so. This results in a lot of roundtrips to the Pulumi state store (which is implemented horrendously inefficiently btw) and generally takes forever and a day compared to kubectl apply.

Most of this is down to an impedance mismatch between the models themselves. Pulumi + k8s works but it's just not a great experience. I say this with an absolute -ton- of experience with both options.

This can be turned off by setting an annotation on the resources.

I'm aware but it's not really ergonomic.

For instance Pulumi also generates random names for resources, again you can override these but again, not very ergonomic to work around etc.

IMO tools like Tanka + Jsonnet, or Cue are just better fit for k8s because you can stick with server side apply which is a much more native experience.

Pulumi also supports server side apply

https://www.pulumi.com/registry/packages/kubernetes/how-to-g...

In what way is the kubernetes API loose schema'd?

It’s not loose schema at all. All the tools that wrap it are.

You don’t know something is going to work unless you actually do it.

Ah gotcha.

I wonder if anyone's converted the API spec into json-schema, might be easier to plug into tools then.

edit: yes, and the tooling in front of it. https://github.com/yannh/kubeconform

following on original work from https://github.com/instrumenta/kubernetes-json-schema

Reply on news.ycombinator.com