Whatever you use to manage the nodes directly is basically what K8S provides (and more) already, so you're really just replicating effort and complexity instead.
Whatever you use to manage the nodes directly is basically what K8S provides (and more) already, so you're really just replicating effort and complexity instead.
k8s orchestration is also useful for running multiple replicas of a database and using leader election to promote a secondary to a primary if the primary goes down. Doing this without k8s would require some kind of clustering anyway, to coordinate the leader election. So if you're already using k8s for everything else, you might as well also use it for this instead of introducing something else like pacemaker.
The one disadvantage is that leader election under k8s does need to be implemented from scratch, because it isn't a first-party concept in k8s. All it gives you is the first-party implementation of compare-and-swap for resources. So you have to make a StatefulSet for your replicas, make a ConfigMap to store the leader election data (which is where the CAS ensures atomicity), and write a sidecar container to handle the healthchecks and leader election and dynamic scale-up/down of replicas. You'll probably end up making an operator to do all that for you so that you can deploy a custom resource for your replicas and they're converted to StatefulSets with Pods containing the sidecars, etc automatically.
Of course, for the big popular DBs, someone has probably already written such operators for you.
I haven't yet seen good documentation on how these K8s databases handle upgrades. Additionally, they all seem very quick to stream from master to create a new node. That is great, for very, very small db's. But a 1TB db would be royal pain to do that with.