Hacker News Reader

Comment by ForHackernews | Hacker News Reader

Sad to see this, but I will again shill for HashiCorp's Nomad as a better alternative to Kubernetes.

Nomad is way easier to self-manage than K8s, but GCP does that for me, with all the compliance boxes checked, for extremely cheap. Every cloud provider is in that boat. Nomad will be more work and more money, be it compute or enterprise fees. I'm sticking with k8s.

The convenience and ease of use isn't worth the increased costs?

Who's to say it doesn't cost more in time and effort?

I agree up to a certain scale. I've managed a large Nomad/Consul setup (multiple clusters, geographically separated), and it was nothing but a nightmare. I believe fly.io had a similar experience.

Having worked with a very large Nomad cluster, I cannot disagree more.

For simple use cases, sure, but you could also just use AWS ECS or a similar cloud tool for an even easier experience.

Can you quantify “very large”?

20k+ nodes and 200k+ allocs. To be fair, Kubernetes cannot support this large of a cluster.

Most of my issues with it aren't related to the scale though. I wasn't involved in the operations of the cluster (though I did hear many "fun" stories from that team), I was just a user of Nomad trying to run a few thousand stateful allocs. Without custom resources and custom controllers, managing stateful services was a pain in the ass. Critical bugs would also often take years to get fixed. I had lots of fun getting paged in the middle of the night because 2 allocs would suddenly decide they now have the same index (https://github.com/hashicorp/nomad/issues/10727)

Definitely not better than Kubernetes, but I don't regret working on it and I like it as a simpler alternative to Kubernetes. I remember trying to hire people for it and not a single person ever even heard of it.

> I remember trying to hire people for it and not a single person ever even heard of it.

I know, it's really sad. Kubernetes won because of mindshare and hype and 500,000 CNCF consulting firms selling their own rubbish to "finally make k8s easy to use".

It's also sad how much Hashicorp apparently ignores marketing for it. Especially when it works with Consul which is already in a lot of places. Also another funny memory is going into a decomm meeting where I was gonna ignore it like usual since 99% of it was another team decommissioning on-prem stuff. And out of nowhere I hear "we just got approval to decomm Consul. We'll be shutting down the servers in two weeks and waiting to see if anything breaks.." followed by my very stern "NO YOU WILL NOT" ahaha. What a mess that company was

I vastly prefer a hole in the head, personally.

Reply on news.ycombinator.com