Hacker News Reader

Comment by flyt | Hacker News Reader

Has Kubernetes grown past its limit of ~100 underlying servers? What's the largest production Kubernetes cluster running today?

View on HN

Yes, the community is working hard to make k8s scale to 1000s of machines and beyond. There is a great blog post about the engineering effort from the various communities (k8s, etcd) to make this happen: http://blog.kubernetes.io/2015/09/kubernetes-performance-mea...

In fact someone recently made a video showing a 1000 machine cluster; let me find that.

UPDATE: here is the video https://www.youtube.com/watch?v=sgWv9sVTIYQ

why don't they add something like master slaves to make that happen? i mean 1000s machines for a single master is way more than most software will handle anyway, still kubernetes makes it easy to manage it, however even 50 machines are working really fine on kubernetes and that even with a slower master server. currently we run with 20s at production with a master at 4 gigs which works really really well.

There is an open PR to this project to document an HA control plane. Basically, you add the kube-podmaster onto each master machine, which does a leader election for the API, controller manager, etc so that only one instance is mutating the cluster. If that does down, another leader election happens.

The "master" in kubernetes is a collection of components that can be distributed. The kubernetes component that is the most loaded in a 1000 node system is the api server. The 1000 node cluster config in that video is running 10 api servers.

I'm the author of that video, BTW, so I know that for sure. :-)

The last time that I saw somebody from Google give a presentation about k8 it didn't seem like "larger cluster sizes" was a real priority at this point, and they were more focused on features. Good to hear this is getting some attention, since it was the major issue that made our team dismiss it as useful in production.

May I ask at which scale do you operate? Keep in mind that is not necessary (and I believe not advised[1]) to keep all your infrastructure in one kubernetes cluster.

[1] https://youtu.be/-8aUxpVrD40?t=3105

I've heard googlers mention that same thing, but now that I think about it I believe they were meaning to say that "for 1.0" they were targeting ~100 nodes, and once they got out there door they would start tackling the large cluster case, which appears to be progressing nicely.

Here's an updated video with a voiceover...

https://www.youtube.com/watch?v=1bHUsBhPL20

Support for 250 nodes is scheduled for October.

https://github.com/kubernetes/kubernetes/blob/master/docs/ro...

Reply on news.ycombinator.com