Hacker News Reader

Comment by wronglebowski | Hacker News Reader

As someone new to k8s and EKS it blows my mind things are only supported for such a short period of time. I find it very difficult to build a product on a platform in such constant flux. I'm struggling to see the benefits of k8s when things are rapidly being replaced and EOLd.

View on HN

Just stick to stable Kubernetes API resources[1] and they are guaranteed to continue to work through all 1.x versions.

[1] https://kubernetes.io/docs/concepts/overview/kubernetes-api/...

If it helps, it may not be k8s, but more EKS. We have had terrible luck at my company with EKS, being more painful than just an on-prem k8s cluster in my experience, getting the networking working, load balancers, etc.

Could you indulge me with any details? Not AWS or someone who works on k8s, I'm just always interested in war stories, and cloud providers "not working" is an interest too.

We got an on-prem cluster working with Rancher and MetalLB with L2 configuration (MetalLB spoke ARP). It was great. We tried to do the same on EKS. we got it iniitally working, but then we needed to get the nodes on certain subnets so that HQ would be happy or something, that was hard. It felt like any time we tried to make any changes, such as adding worker nodes or deleting old ones, the ALB configuration got all messed up and we couldn't get it fixed. Super brittle. We tried with NLB too, but eventually just got tired of fighting with a tool that was supposed to make this stuff simple. I'll tell you what was simple: MetalLB on our own on-prem network.

One plus side of such a rapid release cycle is the changes tend to not be earth shattering. Big changes are deployed slowly and methodically, and the release notes are pretty easy reading. Compared to something with a yearly cadence, you could find yourself drowning under massive changes.

It's not quite as bad as it seems. We have had a vanishingly small number of breakages as we upgraded from early verions of Kubernetes. Mostly just tweaking a few resources where the API got promoted from beta to stable.

k8s is for teams that already manage something in flux. it's for teams that would be writing their own scheduler/orchestrator/manager/control-plane/platform.

sure, it's useful at much smaller scales too, but then you are not likely to feel the upgrade crunch, because you'll likely only use (and will be exposed to) a fraction of all the moving parts of it.

Reply on news.ycombinator.com