People tend to say Kubernetes is complicated because they don't understand the problems it's solving in the first place. If they did, it would be rather straightforward or they would recognize they don't need it in the first place.
That may be deliberate resume-padding. Same as most Hadoop implementations.
unless you have a single server with no availability your system is probably not "simple".
* what do you use for master election
* what do you use to deploy your stuff
* what do you use on the tcp level for HA (keepalived?)
* what do you use to keep your system up to date
these would be my first questions about your system. and if you would use k8s all of them could be answered with "kubernetes"
Not trying to be snide, but so far I haven't found k8s to be much more than primitives with which to build those things.
I've also only begun working with it recently, so I do not have much experience, and any guidance would be much appreciated.
what do you use to scale your databases:
databases can be scaled with statefulsets there is a good library that helps with that: github.com/zalando/spilo it is actually way easier than scaling over vms (especially on bare metal) since 1.10 you can even use block storage with local storage, so no need for a storage system. so basically k8s is just a good layer to scale such things, before we used patroni without k8s and it is way harder to actually do service discovery (more on that later)
what do you use for master election:
master election can be done with configmap or endpoints (actually github.com/zalando/spilo uses github.com/zalando/patroni for running postgresql and does master election over endpoints or configmap annotations) actually etcd also exists when you use k8s (actually it's the hardest thing to install when you deploy k8s)
what do you use to deploy your stuff:
well deployment/apps are extremly easy to use and do zero downtime deployments when you just change your image version
what do you use on the tcp level for HA (keepalived?): some folks at google created metallb: https://metallb.universe.tf/ it's actually a load balancer on top of k8s that is so simple to use (just run https://metallb.universe.tf/tutorial/layer2/ and it will work)
what do you use to keep your system up to date: well that's more tricky however ContainerLinux (formerly CoreOS) (not related to k8s) works very well for keeping the os up to date. you can even use https://github.com/coreos/container-linux-update-operator to only update one node at the time in a k8s env. however it also works without k8s, but you still need to run mostly container software on ContainerLinux.
If you are new to k8s and want to know more about "easy" to use on bare-metal stuff you should look into ContainerLinux ignition and kubeadm, it's really simple to get up and running from scratch, it's mostly just creating a nice cloud-config and run kubeadm once on the nodes you wanna install.
And it's good for deploying big systems with lots of moving parts (such a OpenStack, or any proprietary system, let's say a big e-commerce site).
And even if the problem is straightforward (just run these docker containers, just do this rolling upgrade, just do setup these nginx load balancers, just add these to monitoring), all of these are a chore, especially if done right. And config managers always fail at the premise of apply changes to a "known state" (or they spend an eternity checking the state), so why not fuse actual monitoring with config management, and a lot of nice concepts fall out automatically, like Deployments. No longer do you have to disable the alerting part of monitoring, or mark certain parts of the telemetry time series as irrelevant for SLA accounting because we were deploying (or cobble together scripts to handle this for you), and then evolve the whole system as the underlying components (the things that are being deployed) change.
Sure, k8s is not there yet, as the problem is big, complex, and hard to get right even in parts, but a lot better already than the simple one-man-army homegrown solutions. (IMHO)