Hacker News Reader

Comment by vp8989 | Hacker News Reader

"Kubernetes is the biggest quality-of-life improvement I've experienced in my career."

Can you elaborate? What exactly about Kubernetes improved the developers life?

1. Reliable rolling deployments.

We didn't have these before. Yes, you can implement them without K8s (I have, at other companies) but to get the full set of features K8s provides, such as deploying N services in parallel, taking no more than X% of your capacity offline at a time, short-circuiting in the event the app is dead on arrival, connection draining with timeouts, you end up with a VERY complicated multi-threaded codebase.

2. Seamless horizontal scale-out.

Want to scale up your app in a test environment from 3 replicas to 6 to do some performance testing? This used to be a DevOps ticket that would take a few days -- DevOps engineer tries running Terraform, but oh no, a CentOS package update seems to have broken our Puppet manifests and we have to fix that. Now the developer makes a PR to the GitOps repo where they adjust a single YAML setting.

3. GitOps/ArgoCD.

ArgoCD is possibly my favorite piece of software of all time. It provides incredible visualizations for what's happening with your Kubernetes infrastructure. It really increases your confidence and trust in the system to be able watch a deployment rollout or scaling operation happen in real time. ArgoCD makes it spectacularly obvious when something has gone wrong -- you still sometimes need to go spelunking through the GCP console or use kubectl to inspect resources, but to a much lesser degree. I cannot emphasize enough how magical it is.

These are all things that our initial implementation has delivered, we are also planning to try to leverage K8s for things like on-demand pull request preview environments, hosted developer environments, and canary deploys that are MUCH harder to implement in a world without K8s (trust me, I've done it).

You’re using hosted k8s! That explains everything! I was so confused how a devops person wasn’t cursing the name. Implementing and managing k8s is where all the complexity and headache lives.

Updated original comment to clarify :)

Although, to be fair, the article is about GKE.

If your company is implementing and managing their own k8s clusters they are doing it wrong. Use a cloud hosted solution.

Have you seen kops? rancher? loft? I don't think its that terrible to manage k8s at all using the tools available to you. k8s definitely used to be be difficult to manage but that isn't the case anymore.

But literally every cloud provider out there has a managed solution so at this point you really only need to do it for DC work or if you like to do it.

Amazon - EKS Google - GKE Azure - AKS

anything beyond those 3 is a rounding error but...

Linode - LKE DigitalOcean - DigitalOcean Kubernetes IBM Cloud Kubernetes Service Rackspace - KAAS

I've used Rancher and while it is very nice it still punts a lot of the hard bits like interacting with the outside network, vm creation, and storage (especially shared persistent storage) to the ops team to figure out.

Creating a big-ole cluster of app servers was never really the hard ops problem which is what k8s does really well.

Creating the cluster of servers isn't the value of kubernetes. We all had clusters of servers well before k8s, openstack, and things like AWS.

The benefit of k8s is the orchestration of those clusters. Spinning up 6 new http servers and getting them added to the load balancer automatically. Generating 4 new memcached nodes and getting them registered in DNS so clients pick them up and add them to the hashring.

The benefit of k8s is the scaling and elastic capabilities. It can trigger vertical scaling by spinning up larger pods or horizontal scaling by adding/removing pods.

Anyone thinking that people are using k8s because it can create app servers doesn't understand why anyone is using k8s. If all we needed to do is create a cluster of app servers, we wouldn't be using k8s.

That being said, a cluster of app servers still needs orchestration and config management and we had a ton of crazy solutions for that prior to k8s.

I previously managed my companies Kubernetes clusters using Kops on AWS. My company switched to Google, so I had to move everything to GKE. I miss Kops, it gave me more control and made life easier. I don't want to use the old crappy deprecated DNS server, but with GKE that's what you get. I don't get to control what goes on the masters on my clusters because they're not mine anymore. Having GKE do the masters means more headaches and complexity because I no longer have control. I love K8s, but GKE does a pretty poor job implementing it. It boggles my mind that the main company behind K8s is so incompetent in implementing it.

Its a bit of a pain, but you can definitely switch to CoreDNS on GKE if you need to. Also as someone who has used GKE, EKS, MicroK8s, and Minikube, I would say that the easiest to use implementation of Kubenetes as a Service is GKE. Not saying its easy or the worlds most thought out product, but I gotta give google just a tiny bit of credit for at least being easier to use then the competition.

Yeh this list is not surprising and none of these things you mentioned exclusively require Kubernetes or running containers in Prod. Like I mentioned in another comment, it feels like the purpose of Kubernetes is that it's "brand" provides political cover to introduce these practices to the engineering org.

I don't know what to tell you except that with years of experience implementing this infrastructure outside Kubernetes, letting Kubernetes handle it is cheaper. That doesn't mean Kubernetes is a good fit for every organization or workload.

You can roll your own anything with enough time and manpower. Whether it makes sense to do so depends on your circumstances.

Well for example, I'm a lead devops on a team of 2-3. We have 6 core services.

For the last several years, due to static load we've been fine with 2 instances per service. Now that we want autoscale (even though all the real load is really just the DB), it seems we could get ourselves onto AWS autoscale in ~1 month, though it would require some coding.

Spending 3 devops on 1 year doing something is a red-flag to me. If your changes save devs 1 hour per deploy, it'll pay off 6,000 deploys from now.

It's not the only thing we've been doing, just the most important.

I think it's great you guys have 2.5 dedicated people operating 6 services across 12 prod instances, with that kind of ratio our team would be 15 people!

I agree with your point overall, I just want to add that some DevOps efforts aren't to save DevOps time, but to prevent errors.

When you make a change manually, there's a chance that you forget something, have a typo, etc. Those problems disappear when those same things are automated.

It is not a brand, it's when all the capabilities come together in a single package with one general way to do things that you get the benefits. You could do all this stuff with IaaS but there were too many ways and not enough lines in the sand between dev and ops in the IaaS world. IaaS should be considered a legacy approach.

None of it requires kubernetes, kubernetes just does it out of the box better than any hand-rolled custom stack that tried to do the same. Its a true joy to work with compared to the cobbled together stuff I've built and/or used in the past.

I agree that ArgoCD is amazing. What's so beautiful is how well it integrates into K8s. Every element of ArgoCD is a CRD (custom resource definition). It's less of a piece of software (other than the interface) and more of an extension to K8s to provide Continuous Delivery. Saying ArgoCD does not require K8s is like saying you don't need Photoshop to run a Photoshop plugin.

I've used Puppet, Chef, and Ansible in production over the last decade. For me, Ansible replaced Puppet and Chef five or six years ago. For the last two years the only thing I have used Ansible for is to manage desktops and Android boxes AT HOME. Configuration Management systems like Ansible are fun and cool, but have been mostly obsoleted by the combo of Terraform and K8s. If you're on AWS, Kops can replace a lot of what Terraform does.

Kubernetes makes running services easier and more reliable. I've spent more time learning Istio that K8s itself. The learning curve of K8s is minimal compared to Drupal.

Can only speak for myself personally, but having gone through a similar moment, it was 50% docker QoL improvements and 50% k8s QoL improvements -- most significantly around prod/dev parity. Every engineer in my org really did gain a significant amount of confidence around being able to spin up and deploy new services (when necessary) without being nervous about something going wrong in prod that wasn't configured properly in a lower environment.

I'm really happy with the local development story too. We use skaffold to pick and choose which resources we run locally, and then we use the same definition to generate our artifacts which keeps things in line.

It reminds me of docker-compose file, except I can publish it too.

Reply on news.ycombinator.com