So instead of learning how to deploy on GCP, AWS, and Azure, which is only 3x more complicated than deploying to a single cloud, you should learn K8s, which is 10-15x more complicated, in addition to still having to learn about all the various ingress controllers and weird quirks that are completely different on each cloud provider. Doesn't really track for me.
Probably the biggest one is understanding you don’t ever do anything directly with Kubernetes.
I use gpt-4 through the API where you can set your own system prompt. I developed one that basically instructed it to give me kubectl commands to solve my problems and then wait for me to give it the result before continuing. Through this I learned the practical techniques and which kubectl commands you use on a daily basis, which is so much more helpful than reading the documentation which just gives all commands equal weight.
EDIT: Oh, and definitely watch a few "TechWorld with Nana" videos on YouTube. She does a great job of explaining the architecture, terminology, and philosophy of k8s which I think it very helpful to know.
That mindset of not doing anything directly comes from understanding that you are not setting up pods up, but rather, you are setting automated processes up. These processes knows what the desired state is, and if there is something that changes things, it takes actions to get back to that desired state.
So you don’t create pods. You create deployments that maintain a set of pods. You don’t assign pods to nodes. You define pod affinities, and use node selectors on nodegroups. You use pod priority. You make graceful startups and shutdowns work correctly. And so forth.
If you understand that, you then know you can add processes (such as operators or the cluster autoscaler).
It’s a mindset shift. Thinking of it as “using” kubernetes, as if it is a monolithic thing that you directly control, will greatly increase the difficulty in understanding and reasoning through what’s going on within a Kubernetes cluster.
The complaints are real, because in practice a company needs both aspects and when a small company struggles to setup and manage the kubernetes infrastructure correctly, the application operators are suffering the consequences (e.g. log collection infrastructure doesn't work, it's hard to provision nodes with the right capacity, things like that). They see the infrastructure operation team struggle and they partake in that struggle because what is advertised to be easy, it's not.
That said, K8s is a very good way to build an "API" between infrastructure and application "teams". It can work very well if the people involved set things and processes up correctly. It can be a nightmare if botched up
Agreed.
Someone has to actually build it with a product mindset — including product-market fit. I’m one of those oddballs that have set up and scaled up infra, and have put together and delivered applications before.
In one gig, I had people joke about naming what I had setup after Heroku. I didn’t realize its significance until I came to a place where it was not done this way. Many on the application team have expressed dissatisfaction… but it’s like the infra team is oblivious to that.
You can learn k8s in a day. It's really simple.
> various ingress controllers and weird quirks that are completely different on each cloud provider
Which are thoroughly documented and not that hard to implement or understand. You'd be reading about each cloud's nonstandard ingress even without k8s.
The beauty of k8s is you can run your software locally and have a much easier time lifting and shifting to another cloud.
Fitting to the shape of a cloud provider is a great way to never leave.
Another benefit of k8s is that you treat your services as cattle you can easily spawn and kill. Adoption of k8s naturally leads to anti-fragility, anti-brittle best practices.
But I think most developers don’t care, and instead, should interact with a platform built with Kubernetes as a foundation.