Hacker News Reader

Comment by websap | Hacker News Reader

I couldn't disagree with this more. I manage and run Kubernetes clusters as part of my job, and I can tell you that configuring, running and installing these clusters is no small feat. I don't know much about Nomad, but I would highly encourage most users to not think K8S is simple by any standard. Multiple cloud providers now provide ways to run your code directly, or your containers directly. Unless you have a reason to manage that complexity, save yourself some heartburn and make your developers happy that they just need to worry about running their code and not what happens when a seemingly innocuous change might cause a cascading failure.

This has some great account of war stories - https://k8s.af/

View on HN

Honestly if you use a managed kubernetes provider it's pretty simple once you nail the concepts. Though you'll get hit with cloud provider issues every now and then but it's really not terrible.

I'd manage services in Kubernetes before I start deploying VMs again, that's for sure.

Sure, you pay someone else to keep k8s alive, it's not so bad, but it's expensive to do that. You generally need a full-time team of people to keep a k8s deployment alive if you are running it yourself.

I keep Nomad alive part-time. It's operationally simple and easy to wrap ones head around it and understand how it works.

A lot of the cloud providers don't bill directly for kubernetes management, instead it's just the node resources.

Either way, as another comment points out, Rancher and many other solutions make the orchestration of creating your own Kubernetes cluster really boring.

We run a few kubernetes clusters on premise, and for the longest time it was just 1 person running some Kubernetes clusters.. we even have other teams in QA/Engineering running their own with it.

Which cloud providers don't? I'm only familiar with AWS and GCP, and they both have a base hourly charge per-cluster.

Azure as far as I know doesn't have base charges unless you want a paid SLA, It's Azure though.

DigitalOcean and Linode that I'm aware of, may be others

DigitalOcean charges extra for k8s nodes(compared to VPS), So does Linode last I bothered to check.

Scaleway

You can also use something like Rancher or k3s to keep it alive part-time.

RKE2 is the closest thing we've found to a turnkey, on prem, production ready Kubernetes distribution. Though we were last looking about a year ago so I concede there might be better or comparable options now.

nope. have a GKE cluster running "unattended" for months now. looks fine. ;)

GKE is the industry's first fully managed Kubernetes service...

I.e. you don't run k8s, Google does it for you. I'm talking about where YOU run k8s, not run ON k8s. I agree running ON k8s is pretty easy.

Almost everything is pretty simple "once you nail the concepts". It's getting to the point where you have the concepts nailed that is the measuring stick for complexity.

> Despite its reputation, Kubernetes is actually quite easy to master for simple use cases.

This has so many assumptions rolled up in it. When we move from niche assumptions to the masses of app operators and cluster operators (two different roles) it tends to fall apart.

Which ultimately leads to things like...

> This has some great account of war stories - https://k8s.af/

We shouldn't assume other people are like us. It's worth understanding who folks are. Who is that ops person who wants to run Kubernetes at that insurance company or bank? Who is the person setting up apps to run for that auto company? What do they know, need, and expect?

When I answer these questions I find someone far different from myself and often someone who finds Kubernetes to be complicated.

I also maintained several kubernetes clusters and never found it very difficult. We never had any major outages, and the only downtime was trying new features.

What's the scale of your clusters? If you're running anywhere between 5 - 20 nodes with a 10 pods on each node. I think that's a very small deployment and you'll be able to breeze by with most of the defaults. You still need to configure the following though - logs, certificates, authentication, authorization, os upgrades, kubernetes upgrades, etc.

I'm not sure if all of this is worth it, if you're running a small footprint. You're better of using more managed solutions available today.

5 clusters of around 100 nodes each on average

I dunno man, k8s is pretty simple, but only if you make it so, I have built (early on) complicated setups with all kinds of functionality, but the complexity was rarely used. I now manage clusters and deployments on those clusters with code, terraform to be exact (tcl expect for instances when terraform can't hang yet) I built a gui interface for terraform to manage deployments of kubernetes on various clouds and bare metal and to manage the deployments To kubernetes. \n Maybe one day this will get too complex and we'll do something else but so far so good.

Reply on news.ycombinator.com