With all that said, while I have no "hate" for the stack, I still have no plans to migrate our container infrastructure to it now or in the foreseeable future. I say that precisely because I've seen the source, not in spite of it. The net ROI on subsuming that level of complexity for most application ecosystems just doesn't strike me as obvious.
* Its secrets management was terrible, and for awhile it stored them in plaintext in etcd. * The learning curve was real and that's dangerous as there were no "best practice" guides or lessons learned. There are lots of horror stories of upgrades gone wrong, bugs, etc. Complexity leaves a greater chance of misconfiguration, which can cause security or stability problems. * It was often redundant. If you're in the cloud, you already had load balancers, service discovery, etc. * Upgrades were dangerous and painful in its early days. * It initially had glaring third party tooling integration issues, which made monitoring or package management harder (and led to third party apps like Helm, etc).
A lot of these have been rectified, but a lot of us have been burned by the promise of a tool that google said was used internally, which was a bit of a lie as kubernetes was a rewrite of Borg.
Kubernetes is powerful, but you can do powerful in simple(r) ways, too. If it was truly "the most amazing" it would have been designed to be simple by default with as much complexity needed as everybody's deployments. It wasn't.
It's only useful for the degenerate "run lots of instances of webapp servers running slow interpreted languages" use case.
Trying to do anything else in it is madness.
And for the "webapp servers" use case they could have built something a thousand times simpler and more robust. Serving templated html ain't rocket science. (At least compared to e.g. running an OLAP database cluster.)
Oh, and just in case your first rebuttal is "having thousands of containers means you've already failed" - not everyone works in a mom n pop shop
The majority of folks, whether or not they admit it, probably do...
Just because k8s is the only game in town doesn't mean it is technically any good.
As a technology it is a total shitshow.
Luckily, the problem it solves ("orchestrating" slow webapp containers) is not a problem most professionals care about.
Feature creep of k8s into domains it is utterly unsuitable for because devops wants a pay raise is a different issue.
I truly wish you were right, but maybe it's good job security for us professionals!
What aspects are you referring to?
>> is not a problem most professionals care about
professional as in True Scotsman?
No, I mean that Kubernetes solves a super narrow and specific problem that most developers do not need to solve.
Must be some artifact of hosting on Azure, because I can't imagine any other reason to do something this contorted.