Hacker News Reader

Comment by dpkirchner | Hacker News Reader

Maybe this will finally break me of my habit of using helm charts, period.

I’ve never used Helm charts. I learned K8S in a shop in which kustomize is the standard and helm is a permitted exception to the standard, but I just never felt any reason to learn helm. Am I missing out?

Sometimes the limitations of kustomize annoy me, but we find ways to live with them

I wouldn't say you're missing out. If kustomize works for you, keep using it. I personally use helm because I cannot for the life of me wrap my head around kustomize. I've looked at tutorials, read the docs, and it just doesn't make sense to me. Helm, on the other hand, immediately clicked and I was able to pretty effortlessly write charts for our use. It's just a case of different preference in tools, imo.

Kustomize feels like less of a hack to me, without the gotpl madness, but it’s way more painful to get something done in my experience. I’ve landed on just writing real code to craft the objects I want (using the actual types, not text), if I absolutely can’t get by with static manifests.

Every time I try to get into Kustomize, I feel it just has a documentation problem. It's written in Ph.D.ese without suitable examples

But, if it helps, my mental model of Kustomize is "it's a sequence of JSON Patch [1] operations"[2] which makes it absolutely stellar for mutating someone else's helm chart[3] (or yaml) since you can target anything for updating to your needs

My secondary complaint is that even JetBrains don't have good support for it, which makes working with them extra hard mode

1: https://www.rfc-editor.org/rfc/rfc6902.html

2: it's a very "functional programming language" mental model https://kubectl.docs.kubernetes.io/references/kustomize/kust...

3: https://kubectl.docs.kubernetes.io/references/kustomize/buil...

Helm gives you more than enough rope to hang yourself with. At $dayjob we barely use 3rd party helm charts, and when we do we eventually run into problems with clever code.

We do package our own helm charts, not in the least because we sign contracts with our customers that we will help them run the software we're selling them. So we use package docker and helm artifacts that we sell in addition to running locally.

So we write some charts that don't use most helm features. The one useful thing about Helm that I don't want to live without is the packaging story. We seem to be the only people in the ecosystem that "burn in" the Docker image sha into the Helm chart we package, and set our v1.2.3 version only on the chart. This means we don't have to consider a version matrix between our config and application. Instead we just change the code and config in the same git sha and it just works.

We just set chart version equal to image version, they live in the same repo and are released and built together (and the chart is only published after successfully publishing the image, so it's always valid). The chart allows to override the image version but we almost never do that, it's for emergencies.

Replacing with hash is a neat idea, might start doing that too.

Would you like to count the number of spaces that various items in your manifests are indented and then pass that as an argument to a structure-unaware text file templating engine? Would you like to discover your inevitable yaml file templating errors after submitting those manifests to the cluster? Then yes, you are really missing out!

I really, really, wish we could move away from yaml for tools like this.

If YAML is a foot-gun, templated YAML is a face-cannon.

I never understood why people didn’t serialize from JSON if they run a transformation step on the YAML anyway. Read JSON, alter in your favorite language, dump YAML as the last step, deploy. Instant prevention of a lot of sadness.

Yet somehow, all we have is YAML templating?

Kubectl will handle JSON without converting it to YAML. So if you are doing this, then just do kubectl apply -f deployment.json

So why doesn't helm output to json instead of yaml, and avoid all the problems with significant whitespace?

crickets

CDK8s works really well. I'm sad it hasn't really taken off.

https://cdk8s.io/

A couple of years ago I was using https://kapitan.dev/ which I found really good at the time when using jsonnet files to generate the configs. I haven't used it for some time though.

The main advantage of helm in my experience is:

1. having the ability to create a release artefact helm chart for a version, and store that artefact easily in OCI repositories. 2. being able to uninstall and install a chart and not have to worry about extra state. Generally in Kustomize people just keep applying the yaml and you end up in a state where there’s more deployed than there is in the kustomize config

Some people like that Helm:

- Makes it possible to go from zero to fully running k8s integrated components in 5 seconds by just running 'helm install --repo https://example.com/charts/ mynginx nginx' (very useful: https://artifacthub.io/)

- Gives the ability to transactionally apply k8s configs, and un-apply them if there is a failure along the way (atomic rollbacks)

- Stores copies/versions/etc of each installation in the server so you have metadata for troubleshooting/operations/etc without having to keep it in some external system in a custom way.

- Allows a user who doesn't know anything about K8s to provide some simple variables to customize the installation of a bunch of K8s resources.

- Is composeable, has templates, etc.

So basically Helm has a lot of features, while Kustomize has... one. Very different purposes I think. You can also use both at the same time.

Personally I think Helm's atomic deployment feature is well worth it. I also love how easy it is to install charts. It feels a bit like magic.

> zero to fully running (...) in 5 seconds by just running helm install

Realistically, a plain helm install without any values rarely if ever gives you the deployment you need, so you have to study the chart anyways.

> rollback on failure

This is hardly unique to helm.

> history metadata without (...) some external system

In 2025 you should probably be using gitops anyways, in which case the git repo is your history.

> a plain helm install without any values rarely if ever gives you the deployment you need

works for me most of the time

> This is hardly unique to helm.

So what? The guy was asking what is nice about Helm vs Kustomize. Does Kustomize have rollbacks?

> In 2025 you should probably be using gitops

Gitops is literally just "hey I have some configs in Git and I run some command based on a checkout", i.e. infrastructure as code in a git repo. Gitops does not track live server metadata and deployment history. I don't get why people over-inflate this idea.

> Gitops is literally just...

Please check https://opengitops.dev/

What do you mean by atomic deployment? There are no transactions in the Kubernetes API. Helm has to make one request for each object it creates or modifies, like any other client.

It's a misnomer, but I don't think OP invented that language, it's the word Helm uses for that flag: https://helm.sh/docs/helm/helm_install/#:~:text=the%20instal...

I believe(!) that the "rollback" that helm attempts to put back all the mutated objects, which it can - in theory - do because it maintains the previous state in the Secret objects that contain the rendered(?) and the values for the prior revision

  try:
    for obj in manifest_objects:
      kubectl_apply(obj)
    revisions.push(manifest_objects)
  except:
    old_revision = revisions.pop()
    for obj in old_revision:
      kubectl_apply(obj)

type deal

One thing I haven't seen mentioned in comment. Dunno if Kustomize has something here. But: Helm is a shit but at least some kind of composition tool. Some way to have resource of various types associated to some top level idea.

Very very little else seems to bring this basic sense to Kubernetes. Metacontroller kind of could do that. Crossplane's whole business is this, but it's been infra-specialized: but the Crossplane v2.0 release is trying to be much more generally useful. https://docs.crossplane.io/v2.0-preview/whats-new/ . Would love other examples of what does composition in Kube.

Kustomize is nice but you’re missing out on objects lifecycle management.

Kustomize had the issue that it would leave objects dangling in the cluster and you had to manually clean them up of you removed them from your kustomization file.

If you use the kustomize controller from the FluxCD project, you can set it to prune unmanaged objects.

In true GitOps, I think it's should be default on.

This doesn't happen for me, due to ArgoCD–the kustomize YAML is in a PR, and when it merges to the main branch, ArgoCD automatically deploys it. And yes I've found ArgoCD removes most objects when they are deleted from Git.

Grafana's Tanka is a very underappreciated tool if you have to do something similar to Helm.

I work at Grafana, and Jsonnet powers our whole k8s infrastructure. It can get a little baroque sometimes but overall it’s tremendously powerful, and it’s fun to work with.

Most on this thread are viewing helm from a user perspective: "I want to install X and I can use somebody's chart for it or I can use another tool."

There is another category of users who want a way to mange multiple vendor offerings in a consistent manner into their clusters. If they're all packaged with Helm, the user can have standard process and tooling to do that. It's done for K8s apps what containers did for executables.

Is it great? No, see the grief and pain in sibling threads. Are there alternatives? Sure. But Helm is sort of a standard at this point, warts and all.

I work for a vendor that sells to the second category usually, my chart has some 45 images with some intricate hooks for install and upgrade, subcharts, multiple namespaces, etc. You'd be hard pressed to repackage our stuff for every release we give you.

Why do you want to stop using helm charts? Genuine question, as I'm new to Kubernetes and helm.

Write a few Helm charts and you'll understand why people want to stop using it. `nindent` will become a curse word in your vocabulary. It's a fine tool at the user level, but the DX is an atrocity.

I mean, I have written a few (like 5-10?) and I don't understand either. I find that Helm is quite a nice tool which does its job very well.

What are you planning on moving to ?

Probably kustomize, as my needs are simple. If I care to get fancy, I’m pondering giving Yoke a try.

What's wrong with `kubectl apply -f xxx.yaml`?

We use kustomize because we have four environments that run basically the same stuff (dev with k3s, test, and two cloud regions). If we didn’t use kustomize, we’d be forced to reinvent it to avoid duplicating so much yaml.

I used kustomize to build an ArgoCD install at a previous company, and I was impressed at how powerful it was. Our setup was pretty involved, and kustomize was able to handle all the per-environment differences easily, and the code was easy to work with.

I'm using either opentofu(terraform) or plain yaml. I'm not a huge fan of HCL but at least it is structured and easily manipulated without worrying about whitespace.

easily manipulated? How?

Consuming one that is well written isn't too much pain, IME. But writing or modifying one can be really annoying. Aiui the values.yaml has no type schema, just vibes. The whole thing is powered off using text templating with yaml (a whitespace sensitive language), which is error prone and often hard to read. That's basically the main issues in a nutshell, it may not sound like much, but helm doesn't exactly do a whole lot and it does that limited set of stuff poorly.

I share your dislike of Helm, but FYI there are schemas for values, see for example https://github.com/bitnami/charts/blob/main/bitnami/postgres... and docs https://helm.sh/docs/topics/charts/#schema-files

Ah, I must have met some lazy charts then. Thanks for the correction. Still, it seems like that schema would end up a little inconvenient to integrate into your editor for writing the templates...

Oh, yeah, but who can blame them because writing JSON Schema files by hand is some onoz

Ironically (in the context of this submission) Bitnami has a "--schema" option to their README generator, which is why so many of their charts ship with .schema.json files: https://github.com/bitnami/readme-generator-for-helm/tree/2....

There are likely other "give me a schema for this example JSON/YAML file" but almost certainly wouldn't come with the nice {"description":""} blocks, nor the {"required":false} that an annotated .yaml can offer

While digging up that link, I also spotted the tag line in their GitHub org which is hilarious https://github.com/bitnami#:~:text=trusted%20by%20ops

> a little inconvenient to integrate into your editor for writing the templates...

Another way that JetBrains tooling shines, because it automatically turns on JSON Schema support for the chart when it has one, no extra # schema=file://... dumbness required

That is when you are writing the values when using the chart though, right? Yeah that's not too hard.

I was talking about the go templates that make up the actual chart. Ensuring they only use string values for annotations for example seems like a hard problem to solve. Even just showing the type and description from the schema seems nontrivial.

I don't exactly follow which part you're talking about, but yes JetBrains propagates the type and description from the JSON Schema into the golang templates inside the manifest files, same as it does with any type completion. They are reflected as their JSON Schema types, not golang's types, which I could imagine is potentially confusing to someone used to golang's view of the world but is for sure considerably better than "hurr, the type is whatever you say it is" which will be the case for values.yaml files without an accompanying .schema.json file

---

As for the "only string values for annotations," I think you mean patternProperties:

  {
    "$schema": "http://json-schema.org/draft-07/schema",
    "type": "object",
    "properties": {
      "annotations": {
        "type": "object",
        "description": "this is the description of the annotations key itself",
        "additionalProperties": false,
        "patternProperties": {
          "^[a-z][a-z0-9./]+$": {
            "type": "string",
            "description": "holds the value of the annotation"
          }
        }
      }
    }
  }

and if I put the following, then the 123 gets flagged as a schema violation

  annotations:
    alpha: 123

as does

  annotations:
    _: xxx
    0xcafebabe: denied

---

If you mean schema checking on the output, that is checked by the OpenAPI built into Kubernetes itself, so yes, doing something silly in golang templates will not, by itself, check the result - that's one of the major limitations of Helm's moronic choice of using a text templating language for a structured file format

Oh, OK. I didn't realize it made them visible in the golang templates.

I will leave it to others: https://noyaml.com

Golang string templating in a whitespace sensitive config language suuuuucks.

I might use Helm charts for initial deploys of operators, but that's about it.

Kustomize is, IMO, a better approach if you need to dynamically modify the YAML of your resources and tools like ArgoCD support it.

I suggest checking out Anemos (https://github.com/ohayocorp/anemos), the new boy in the town. It is an open source single-binary tool written in Go and allows you to use JavaScript/TypeScript to define your manifests using templates, object oriented approach, and YAML node manipulation.

You can read a comparison with Helm here: https://www.ohayocorp.com/anemos/docs/comparison/helm

P.S. I am the author of the tool.

Reply on news.ycombinator.com