Comment by cwiggs | Hacker News Reader

Comment by cwiggs | Hacker News Reader

Parent Full thread

Helm Charts and Terraform are different things IMO. Terraform is better used to deploying cloud resources (s3 bucket, EKS cluster, EKS workers, RDS, etc). Sure you can manage your k8s workloads with Terraform, but I wouldn't recommend it. Terraform having state when you already have your start in k8s makes working with Terraform + k8s a pain. Helm is purpose built for k8s, Terraform is not.

I'm not a fan of Helm either though, templat-ed yaml sucks, you still have the "indent 4" insanity too. Kustomize is nice when things are simple, but once your app is complex Kustomize is worse than Helm IMO. Try to deploy an app that has a ING, with a TLS cert and external-DNS with Kustomize for multiple environments; you have to patch the resources 3 times instead of just have 1 variable you and use in 3 places.

Helm is popular, Terraform is popular so they both are talked a lot, but IMO there is a tool that is yet to become popular that will replace both of these tools.

I agree, I wouldn’t generate k8s from Terraform either, that’s just the alternative I thought the OP was presenting. But I’d still rather convert charts from Helm to pretty much anything else than maintain them.

Lack of Variable substitution in Kustomize is downright frustrating. We use Flux so we have the feature anyways, but I wish it was built into Kustomize.

I don't miss variable substitution at all.

For my setup anything that needs to be variable or secret gets specified in a custom json/yaml file which is read by a plugin which in turn outputs the rendered manifest if I can't write it as a "patch". That way the CI/CD runner can access things like the resolved secrets for production without being accessible by developers without elevated access. It requires some digging but there are even annotations that can be used to control things like if Kustomize should add a hash suffix or not to ConfigMap or Secret manifests you generate with plugins.

Trivial to work around using _envsubst_ utility (part of _gettext_). If you write something like: name: app-$KUSTOMIZE_NAMESPACE and run `kubectl kustomize | envsubst "$(printf '${%s} ' ${!KUSTOMIZE_} ${!CI_} ${!GITLAB_*})" | kubectl apply -f -` it will substitute the env vars that match the wildcard.

You can deploy your Helm charts through Terraform, even. It's been several years sinc so the situation might have improved but last I worked this way it was OK except of state drift due togaps in the Helm TF provider. Still found it better then either by itself.

Re your kustomizen complaint, just create a complete env-specific ingress for each env instead of patching.

- it is not really any more lines - doesn’t break if dev upgrades to a different version of the resource (has happened before) - allows you to experiment with dev with other setups (eg additional ingresses, different paths etc) instead of changing a base config which will impact other envs

TLDR patch things that are more or less the same in each env; create complete resources for things that change more.

There is a bit of duplication but it is a lot more simple (see ‘simple made easy’ - rich hockey) than tracing through patches/templates.

Reply on news.ycombinator.com