Comment by sofixa | Hacker News Reader

Comment by sofixa | Hacker News Reader

Parent Full thread

> You can easily write tools that generate json/yaml for you based on your needs. For most applications you're deploying you'll need:

Can you with yaml? Helm tries, but isn't that great, jinja ( depending on how its done ( jinja in yaml as in ansible or jinja to yaml as in SaltStack)) does an ok job, but fundamentally managing a language which uses spaces for logic with code is hard, and certainly YAML wasn't created to be managed that way.

JSON i agree, you can easily convert your language of choice's data structures to it. But do you have to? It adds another layer of complexity, and you can have unexpected bugs ( since you have a logic layer that creates JSON, which could be wrong due to a logical error).

HCL is a DSL that needs learning and few use outside of Hashicorp products, but IMHO it's not that hard ( docs are great, tutorials are plenty) and it's worthwhile to have data+logic in the same human and machine readable layer.

In any case, Nomad accepts JSON for everything ( mostly for when non-humans interact with it).

Helm, in my personal opinion, is one of the biggest mistakes of the kubernetes ecosystem. Not embracing Jsonnet, Cue, Dhall, or even an in-language DSL like Pulumi has set the usability of kube back quite far.

$DAY_JOB replaced helm with custom tool that does equivalent of kubectl apply by uses Jsonnet as templating engine.

I intend to write an OSS version of it because Jsonnet really is probably the best fit here (though Cue/Dhall are interesting I think Jsonnet is easiest to get others to adopt).

I've done the same in the past. The important thing is making it so it's easy to build up abstractions and to work in a sensible deployment flow.

kubectl is close but doesn't understand secrets/variables, the concept of clusters, or changing image tags programmatically. It's difficult to get right.

Yeah this is all stuff this tool understood, multiple clusters, managing kube contexts, external secret storage or k8s secrets, native git integration, etc.

Definitely agree that it's difficult to get right but the authors of the tool definitely struck a really good balance.

Hopefully if I implement it myself I can do their original design and implementation justice.

> kubectl is close but doesn't understand secrets/variables, the concept of clusters, or changing image tags programmatically. It's difficult to get right.

Isn't that the problem kustomize (which kubectl now has native support for applying) is trying to solve? I ask that honestly, because I haven't used kustomize in anger in order to compare its workflow to that of helm

My mistake. I meant kubecfg which is built off of jsonnet.

I did the full gambit of helm 1/2 tiller/tiller-less and kustomize. Personally kustomize is only marginally better than helm.

I need to check out kubecfg, I haven't used it yet.

I also did something like this a few years ago (with Go templating instead of Jsonnet), but we recently replaced it with Kustomize (https://kustomize.io/) as it worked quite well for the basic use case of getting shared Kubernetes configs to deploy code to different environments at different times.

The fact that trivial things like ingresses don’t have transformers is a PITA.

No need to build it, there's already kubecfg: https://github.com/bitnami/kubecfg

We use this heavily at InfluxData to manage multiple clusters in multiple regions of multiple clouds. One of the maintainers of kubecfg recently joined InfluxData as well.

Helm isn't that connected to the broader Kubernetes system. It was popular in the early days when Kubernetes knowledge was sparse and things were still being figured out. None of the official Kubernetes docs really mention or describe using helm, and I see more getting started tutorials are starting to avoid it too. The official kubectl tool supports a -k mode which applies Kustomize logic and I'd argue Kustomize is probably the official way forward with app deployments by the Kubernetes project.

Yet, the embedded kustomize version is years behind standalone.

https://github.com/kubernetes-sigs/kustomize/issues/1500

A team at my workplace uses Jsonnet for this problem and it seems to work very well. It's a nicely human-readable DSL for generating JSON in a way that respects JSON structures - you can construct lists of objects (deployments, services, etc.) by looping over templates and composing modifications.

That seems a lot better than either SaltStack's use of Jinja (using text-based templating on a structured format) or Ansible's (using YAML as syntax for an imperative programming language that has Jinja-evaluation functionality).

Reply on news.ycombinator.com