Hacker News Reader

Comment by DevKoala | Hacker News Reader

I work for a mid size company with 30-40 engineers managing 20-30 very diverse apps in terms of scale requirements and architectural complexity. It took our devops team (4-5 people) probably 18 months to learn and fully migrate all our apps to Kubernetes. The upfront cost was massive, but nowadays the app teams own their deployments, configurations, SLA's, monitors, and cost metrics.

Introducing Kubernetes into our org allowed us to do this; we would have never gotten here with our legacy deployment and orchestration Frankenstein. The change has been so positive, that I adopted Kubernetes for my solo projects and I am having a blast.

I understand Coinbase's position, and they need to stick to what works for them. I just wanted to bring up a positive POV for a technology I am becoming a fan of.

View on HN

Do you have any intuition as to what percentage of the benefits your company has seen come from kubernetes specifically, and how much just from the exercise of spending 18 months working on a modern, coherent, efficient infrastructure?

Kubernetes is almost a standard, all our engineers are eager to learn it. It is extensible, some app teams have started building operators to tackle problems in a more efficient way. It allowed us to standardize metrics and monitoring, we didn’t have a clear story for this before. It is cloud agnostic and with great compatibility; I was part of a migration from GKE to EKS, and it was painless.

One other advantage of Kubernetes that is overlooked in the article is the benefit with a heterogeneous cluster of instant auto scaling. For example you have 10 apps on a k8s cluster that each use the same resources, you can give 20% buffer for the cluster, which would let any single app use 300% of their allocation instantly. With VMs, you’ll be stuck waiting for VMs to spin up or have to give each app their own large buffer to handle bursts of traffic.

It amazing to me how many people associate this with kube. You’ve had this for years in numerous ways, the only difference is some yaml and marketing.

What other solution is cloud agnostic as well as has an easy local dev story?

_any_ non-proprietary tool is "cloud agnostic". Kubernetes is bundled software, and achieves the things those pieces achieve. There is nothing holy about k8s specifically, the tools you train with are easy, and it's very easy to get skewed opinions on that.

For example a lot of people would find writing scripts cumbersome, but not a person who's written a lot of them. They're not any more fragile than other logic error capable software is

You didnt avtually answer the question: What free, cloud agnostic tools let me specify "keep 60% cpu load average" and work the rest out?

With k8s, a horizontal autoscaler is a few lines in a yaml file and the result works in any cluster run by any vendor.

Any load balancer will let you do that. Scaling is a few lines of scripts to do on any platform, and well worth the couple minutes, you don't even need a tool for just that.

No tool "works the rest out". It's _always_ a compromise, because inherent complexity can never be removed, only moved. What you gain in one area, encumbers another.

You may freely use k8s, but it's not magical nor easier to use than any existing systems. In fact adopting it often takes non-trivial time and the web is full of failure stories with very benign warnings and catastropic results.

Isn't that the benefit of cloud?

Maybe I'm spoiled by GCP and it's not the same in other providers; but I can have a brand new debian VM configured and operational in less than 10s (less the time that I run my own startup script).

Debian machines spawn with incredible speed; not much slower (if at all slower) than a new container.

Spawning a bare bones VM is easy and fast, the problem is getting the applications there, which is where k8s shines.

Maybe I'm spoiled then, it takes less than 2s to spin up my application on a new instance. :\

There are also many options for creating Debian based image files with all you need baked in.

I guess it depends very much on the application you have.

A lot depends on whether "bake application into AMI, use ASG to scale and zero-downtime deploy" is applicable or cost effective.

I think I've been in total of one company where it was somewhat true, and even there we didn't do it.

I'm a fan of cloud native kubernetes.

But: I've also been put in the terrible position of supporting a platform sold to run on kubernetes in an air gapped, on prem, bare metal environment.

But: I've also been put into the terrible position of fighting vendors armed with agile, k8s and microservices selling that combined mayhem as a replacement for an openmp,openacc based massively parallel, on prem, bare metal HPC system with a strongly conservative development and operational tradition.

I understand loving efficiency and time savings but not everything simply works better with k8s.

> I've also been put in the terrible position of supporting a platform sold to run on kubernetes in an air gapped, on prem, bare metal environment.

Would you share your experience?

Sorry, can't share details as I enjoy being employed.

Exactly. A lot of the comments here are along the lines of "we setup a k8s cluster and now managing it is a huge burden" which is not surprising. The power of k8s is to allow separation of concerns in your technology organization. You can have a dedicated team to build and maintain the underlying cluster and then app development teams are consumers who deploy their application on the common infrastructure. Kubernetes provides a nice abstraction layer so those two teams/orgs can interact through a well-defined API. As a dev team, we can manage our own infrastructure and pipeline through declarative configurations and let someone else manage the underlying compute and network infrastructure. As long as you don't fall into the anti-pattern of "every team builds it's own k8s cluster" then you should be able to derive some nice economies of scale.

You're comparing 18 months of migrating to K8s to doing nothing, rather than 18 months of other possible solutions.

Reply on news.ycombinator.com