Can someone explain how logging is supposed to work after this change? I’m complete bloody lost.
Actually I’m nearly always lost with kubernetes. It’s either broken or changing.
Actually I’m nearly always lost with kubernetes. It’s either broken or changing.
You have a Pod definition, which is basically a gang of containers. In that Pod definition you have included one or more container image references.
You send the Pod definition to the API Server.
The API Server informs listeners for Pod updates that there is a new Pod definition. One of these listeners is the scheduler, which decides which Node should get the Pod. It creates an update for the Pod's "status", essentially annotating it with the name of the Node that should run the Pod.
Each Node has a "kubelet". These too subscribe to Pod definitions. When a change shows up saying "Pod 'foo' should run on Node 27", the kubelet in Node 27 perks up its ears.
The kubelet converts the Pod definition into descriptions of containers -- image reference, RAM limits, which disks to attach etc. It then turns to its container runtime through the "Container Runtime Interface" (CRI). In the early days this was a Docker daemon.
The container runtime now acts on the descriptions it got. Most notably, it will check to see if it has an image in its local cache; if it doesn't then it will try to pull that image from a registry.
Now: The CRI is distinct from the Docker daemon API. The CRI is abstracted because since the Docker daemon days, other alternatives have emerged (and some have withered), such as rkt, podman and containerd.
This update says "we are not going to maintain the Docker daemon option for CRI". You can use containerd. From a Kubernetes end-user perspective, nothing should change. From an operator perspective, all that happens is that you have a smaller footprint with less attack surface.
The fact that so many other orgs, many of which are startups or just small to medium sized tech companies, use a system this complex is ludicrous to me.
[0] most folks emphasise the control loop aspect, I think it's more helpful to point to blackboard / tuple-space systems as prior art.
Ah, nevermind then, that gives me rest concerning the viability and transparency of it all.
It's kind of like if you had a shell script to launch programs, and it used to move the mouse to press icons, but now you've deprecated that and will only run programs directly.
I've seen way too many Ansible nightmares grown out of deceptively "simple" mutable VM deployments.
k8s makes our life so much easier because it eliminates a whole bunch of other complexity. Easily reproducible development environments, workload scheduling, sane config management...
It's not more complex. If you're small, the overhead is probably not worth it. If you're big enough, you manage the k8s control plane, and you don't have to manage your tenants infra.
Is a programming language too complex for hello world? Perhaps, but that's not all it does.
As I understand, dockershim makes docker daemon cri compliant. But dockerd already uses containerd which is cri compliant. So, why can't kubelet directly interact with containerd APIs without dockershim?
Each kubelet does its thing through the Container Runtime Interface (CRI), so in a sense it doesn't know what it's running on. If it used containerd's interfaces directly, it wouldn't be possible to substitute in a different option.
For example, there are emerging VM-based approaches like Firecracker and VMware "Project Pacific" (disclosure: I work at VMware).
If someone wants to use kubelet + docker so that they can, for example, ssh into a node and type 'docker ps' to see containers, or have something else using the docker api see the containers the kubelet started, that won't work after re-pointing the kubelet from docker to containerd.
The difference here is namespacing[0], but not the linux-kernel-container-namespace, rather the containerd concept by the same name to allow "multi-tenancy" of a single containerd daemon.
In addition, I don't think you could have docker + cri run in the same containerd namespace since they end up using different networking and storage containerd plugins. I think that terminology is right.
So yeah, repointing the kubelet to containerd directly works fine, but it won't be the same thing as running docker containers.
[0]: https://github.com/containerd/containerd/blob/9561d938/docs/...
Glad I'm not the only one. I'm sure I'm not the smartest engineer/sysadmin in the world, but I'm also not the dumbest and I have never gotten an on-premises Kubernetes installation to work.
The way I manage containers is lxc and shell scripts. I understand it, and it works.
Don't tell the boss or the customers, but most of the time when release notes for a new version come out, I look at them and go "WTF, why do we need that, I better do some research." It's fast changing, complex as hell, and absolutely brutal. That said most things are there for a reason and once I dig in I usually see the need.
That said I do love it despite its warts. There's no doubt some Stockholm Syndrome at play here, but I love the API (which is pretty curlable btw, a mark of a great API IMHO) and the principles (declarative, everything's a YAML/JSON object in etcd, etc). I see it the same way I did C++ (which I also loved). It gives you great power which you can use to build an elegant, robust system, or you can create an unmaintainable, complex, monster of a nightmare. It's up to you.
[0] https://goteleport.com/blog/kubernetes-release-cycle/ , HN discussion: https://news.ycombinator.com/item?id=16285192
It's not Kubernetes but it doesn't try to be at all.
What is deadly difficult is getting networking to work. Even a comparably "easy" thing with a couple of two-NIC machines (one external, internet-routable, one DMZ) cost me a fucking week.
What's even worse is when one has to obey corporate restrictions - for example, only having external interfaces on "loadbalancer" nodes:
- First of all, MetalLB only has one active Speaker node which means your bandwidth is limited to that node's uplink and you're wasting the resources of the other loadbalancers.
- Second, you can taint your nodes to only schedule the MetalLB speaker on your "loadbalancer" nodes via tolerations... but how the f..k do you convince MetalLB to change the speaker node once you apply that change?!
- Third, what do you do when you want to expose N services but only have one or two external IPs? DC/OS was way more flexible, you had one set of loadbalancers (haproxy) that did all the routing, and could run an entire cluster on four machines - two LBs, one master, one worker. There is no way to replicate this with Kubernetes. None.
That said, depending on where you were working, things could also change fast. You could find yourself finding that a kernel system call had changed because someone patched it the evening before.