Hacker News Reader

Comment by aksels | Hacker News Reader

From my experience it's quite expensive, at least in GKE. As Kubernetes takes a lot of cpu & memory by itself. I run a pool of one node n1-highcpu-2 (2 vCPUs, 1.8 GB memory) and it costs me about 72 euros per month.

kube-system is taking about 730 mCPU of the node's 2000 mCPU.

The issue is that the kube-system is deployed on each node of the pool so (tell me if I misunderstood/misconfigured something). If you have a pool of 3 nodes which have 1 vCPU each, with kube-system taking approximately 700 mCPU on each you have only 300 * 3 = 900 mCPU allocatable.

If you have any tips on how I could reduce the costs of my personal projects I'm listening !

View on HN

I just saw a change the other day which makes system pods fit better on nodes with one vCPU. It reduced their numbers by the same factor (i.e. keeping the same proportions). It was meant for GCE and I am not sure if/when it will make it to GKE, but it looks like Google is looking at the tiny node scenario. Also, Tim Hockin led a discussion about something similar at last year's Kubecon.

Here it is. It's in 1.12, actually, not on HEAD as I recalled:

https://github.com/kubernetes/kubernetes/pull/67504

Still not sure if these are the configuration files that GKE uses behind the scenes, but the author is a Google employee.

When creating to disable the cluster, have you tried to disable Kubernetes dashboard and StackDriver monitoring and logging services?

For logging, you could use FluentBit which is more lightweight: https://docs.fluentbit.io/manual/output/stackdriver

As a data point, on a busy GKE production cluster with 12 highcpu-16 nodes we have kube-system pods taking a total of ~700mCPU on all servers combined. I'm not sure how the CPU usage scales, but probably not linearly.

Most of the CPU usage is kube-dns, which I think requires much less CPU if you make few internal network connections.

I've seen a few comments here about k8s using a lot of CPU by itself. Do you known exactly what it does that uses a lot of CPU?

Reply on news.ycombinator.com