Comment by stackskipton | Hacker News Reader

Comment by stackskipton | Hacker News Reader

Parent Full thread

We piloted two jobs ago and aborted. Wouldn't recommend at your size because it seemed to do effectively, you need a "Backstage Team" which is not Ops Team who occasionally does Backstage work. Probably why Spotify is considering it because sweet consulting money.

While I don't know your exact situation, I'm currently at company of your size engineers and stop torturing your Developers with Helm is off the shelf advice I would give. See if Kustomize would get the job done and be honest. I've found Devs can handle Kustomize a ton easier since kubectl kustomize > testoutput.yaml is alot easier for them to parse.

Helm is great if you have a bunch of different things to launch with a ton of different options but most companies don't do that. They need HPA/Deployment/Service/Ingress which is easily handled by Kustomize.

I'm somewhat new to helm, after a year I feel like I'm pretty good at it, but struggle to teach other ML folks what I've learned the hard way over the last year. Some of that is helm, plenty of it is Kubernetes. Most of our charts are ingress, service, deployment, service account, scaled object (keda), but there's plenty of small variation even across ML applications, plus the different needs of other teams.

Any good resources you can point me to on how Kustomize might be easier for us?

Documentation is best: https://kubectl.docs.kubernetes.io/references/kustomize/

However, biggest is it's just YAML patching. So you don't have this {{if such}} littered throughout your project that people can't easily pick up. Values is biggest reason I see people want Helm but if it's all env variables, kustomize lets you put in .env file it turns into config map and injects into deployments.

It's possible Kustomize doesn't fix your problem esp if Developers have decided it's not their problem. I just find Devs find Kustomize much easier.

Helm is what you should use as your cluster "package manager" to install all of your favorite off the shelf tools. Kustomize is probably a better choice for shipping your internal projects due to the variations you mentioned, but also because the learning curve isn't much steeper than learning about how to work with plain Kubernetes manifests.

Reply on news.ycombinator.com