It makes a lot of sense to build portable infrastructure, but you can scale a long ways with much simpler technologies.
- App Engine has a bunch of weird limitations and slow deploy times. Qualitatively, it feels like the spotlight has moved on.
- Running my own compute instances felt like reinventing Kubernetes, especially once you roll your own deploy mechanism and throw load balancing in the mix. I also don't buy that it's easier.
- Cloud run is promising but for database heavy apps it's a non-starter.
- GKE was pretty smooth. It feels like it gets a lot more love than App Engine. The UI was functional with lots of depth. Once I push a docker image, GKE updates the nodes to serve the latest version. Load balancing was a matter of ~3 yaml files at 10 lines a pop.
There are a lot of businesses that will never have to deal with dynamic scaling, engineering autoscaling into those solutions is pointless. If you need that kind of scaling then K8r is fantastic. My point is a lot of people turn to K8s well before they need to or without understanding why they might need it.
It's an insanely complex solution to a very niche problem - scaling stateless web app backend nodes written in scripting languages.
Stray even a little bit off the garden path and you start feeling pain.
K8s sounds like a good idea on paper - you get reproducibility and resilence "for free" - but then I found out I have to effectively roll my own everything with k8s anyways and went with Jenkins instead.
K8s solves a far wider problem space. Need to run a data store? Use a StatefulSet and PersistentVolumes. Need an occasional task? Jobs and CronJobs. Need to know what's happening? Metrics and logs have APIs. Load balancing? Ingress? Firewalls? Security?
If someone knew nothing of operating systems other than a class on MINIX, I suspect running a massive datacenter would be easier with k8s than running a medium system of debian boxes.
Way too complex for my tastes, but if you're convinced you need a highly automated control plane for those things then pushing .deb or .rpm packages doesn't even come close to solving that problem.
As usual it comes down to how you define the problem. You can't dissuade people from using k8s by comparing and contrasting k8s to alternatives; you've already ceded the debate over how to define the problem at that point. W'ever its relative merits, k8s is a reasonable approach to the problem of automating the control plane for "scaleable" services.
Not saying it is the answer to everything, but it can do a whole lot more than software. Most of our problems are self made we can unmake them by changing the rules we operate under.
> As usual it comes down to how you define the problem.
Totally.
Redefine the problem until the solution is tractable and simple. K8s is the problem to a solution.