What yields the lowest risk - spending a ton of time hardening one cluster, or building multiple clusters to reduce the blast radius of bugs and misconfigurations?
Not sure this is a valid dichotomy.
If you are spinning up multiple clusters, you are presumably doing so in an automated fashion. If so, then the effort of hardening is very similar. It doesn't really matter where you do it.
Multiple clusters may have a smaller blast radius, but will have a larger attack surface. Things may be shared between them (accounts? network tunnels? credentials to a shared service?) in which case an intrusion in one puts everyone else at risk.
Nope. If the clusters are separate it limits how damaging a compromise of the cluster is. This is why cloud providers don’t stick you on the same k8s cluster as another tenant.
> Multiple clusters may have a smaller blast radius, but will have a larger attack surface. Things may be shared between them (accounts? network tunnels? credentials to a shared service?) in which case an intrusion in one puts everyone else at risk.
It’s not really clear what you’re trying to say here. If someone compromises credentials shared between all clusters that’s the same as compromising credentials used by one mega cluster.
But if the clusters are configured similarly, a flaw in one is likely present in the others. GPs point is that if you invest in hardening, you can easily apply it to multiple clusters.
> It’s not really clear what you’re trying to say here.
I assume they mean having more clusters present means there are more opportunities to be compromised (e.g. more credentials to leak, more API servers to target, possible version skew, etc.).
That doesn’t matter. The point is that you isolate applications/tenants into different clusters. So if someone exploits their own, they haven’t gained access to some other application.
> assume they mean having more clusters present means there are more opportunities to be compromised (e.g. more credentials to leak, more API servers to target, possible version skew, etc.).
That doesn’t even make sense though. In our strawman scenario these are cookie cutter things. Many is not more vulnerable than one in this case.
Having multiple clusters may help reduce the blast radius of _certain_ attacks, to some degree. However, managing multiple clusters is a lot more difficult than managing one, and you will potentially replicate bad practices, vulnerabilities to multiple places and increase maintenance burden.
Can you elaborate?
One exception, as other have mentioned, is blast radius - with a single cluster, a problem with Kubernetes itself could take down everything.
Multiple clusters protects you from these types of configuration mistakes by reducing the blast radius and providing an additional landing zone to roll out changes over time.
1. 1 cluster was awesome
2. Many clusters means I rebuild the world
3. I wish there was a way to get the benefits of one cluster across multiples.
Which I believe is a solvable problem and partially what we've been poking at at https://github.com/kcp-dev/kcp (although it's still so early that I don't want to get hopes up).
Of course that reduces the benefits of Kubernetes from a cost perspective, and increases administrative overhead, so it's a trade-off.
If "kubernetes" is Greek for the ship's pilot, what's the name for the captain or maybe the admiral of the armada?
Hive for OpenShift [0] - Provisioning of the K8s Fleet
Advanced Cluster Management [1] - Management of the K8s Fleet
[0] https://cloud.redhat.com/blog/openshift-hive-cluster-as-a-se...
[1] https://cloud.redhat.com/products/advanced-cluster-managemen...
[Disclosure: I'm an IBMer]
https://caylent.com/kubernetes-cluster-federation-with-admir...
Google says the Greek for that is Ναυαρχείο. Now if you see a product named Navarxeio or Navarcheio, you'll know what it is.