Hacker News Reader

Comment by rglover | Hacker News Reader

Kubernetes is hard because it's over-complicated and poorly designed. A lot of people don't want to hear that because it was created by The Almighty Google and people have made oodles of money being k8s gurus.

After wasting two years chasing config files, constant deprecations, and a swamp of third-party dependencies that were supposedly "blessed" (all of which led to unnecessary downtime and stress), I swapped it all out with a HAProxy load balancer server in front of some vanilla instances and a few scripts to handle auto-scaling. Since then: I've had zero downtime and scaling is region-specific and chill (and could work up to an infinite number of instances). It just works.

The punchline: just because it's popular, doesn't mean it's the best way to do it.

View on HN

It’s not overly complicated, it’s just trying to serve everyone’s use cases. I’ve tried deploying to 10k servers with custom scripts in Jenkins, bamboo and AWS auto scaling groups but I’ve found kubernetes is the only tool that will elegantly handle a problem. You can probably write a script for the happy path but for a production service I’d bet my money on something that can handle all of the problems that come along with the statistics blow ups at scale. That said, I can be complete overkill for most systems.

right

the thing is that a tool that's good for 10k servers isn't really good for 10 servers

For a happy medium, check out Nomad. I've been managing our infrastructure on Nomad for years by myself, with upwards of 40 nodes (auto-scaled) and the number of problems we've had can be counted on one hand (and was almost always a simple user error or fixed by upgrading). I spend most of the time I would otherwise spend doing tedious ops shit actually building things.

That said, Nomad and stateful services don't mix. Don't try. I think the same goes for k8s though.

What do you do with stateful services?

Run them on EC2 or whatever managed service whatever cloud host provides (RDS, S3, etc). It's possible to run stateful services on Nomad (and I assume k8s) but from my understanding the cost is extremely high, usually much higher than the benefit.

You can setup a solid k3s cluster in 30 minutes. I'm sorry you had a hard time but just because you didn't succeed at your attempt doesn't mean it actually is super hard.

Setting it up is relatively easy, keeping it running consistently while navigating the schizophrenic levels of change and indecision on how things are done is another bag of chips. No need for passive aggression.

Yeah we use k3s for local and testing environments, where it shines. I wouldn’t want to use it to run stuff in prod and figure out how to do zero-downtime upgrades with multiple nodes.

Reply on news.ycombinator.com