Hacker News Reader

Comment by dilyevsky | Hacker News Reader

they do if cluster admin didn't setup proper system-reserved and kube-reserved (both are kubelet flags) and configured enforcement.

View on HN

In this case, the cluster admin would be whoever is provisioning the cluster nodes. In Google Kubernetes Engine, the "Capacity" and "Allocatable" info shown on the nodes are different (I see some mem/cpu reserved for probably system stuff). This makes me think GKE probably subtracts node capacity from what's allocated for the system automatically.

P.S. I work at Google.

correct, it should be provisioned by k8s provider (AKS in this case) and is what GKE is doing https://cloud.google.com/kubernetes-engine/docs/concepts/clu...

note, it also needs to match node configuration (how cgroups are setup specifically) so I doubt this works well on EKS which is BYO node. Maybe it's the issue with AKS too, I don't know enough about how it works...

AKS now reserves 20% of memory from each agent node and a very small amount of CPU to protect docker daemon and kubelet to function with misbehaving customer pods. However, that just means customer's pods will be evicted or no place to schedule when all resource is used up. This is something we see now in customer support cases.

That seems crazy high. If I have a node with 512G of RAM the kubelet/sys will take 100G? why would kubelet ever need this much?

AKS caps at 4G.

4GB of RAM per machine, or 4GB reservation of a 20GB machine?

Reply on news.ycombinator.com