Either you use it a lot (likely at work) and are therefore comfortable using it outside of work as your one-size-fits-all solution to everything, or else you don't use it a lot and see it as a complexity behemoth that hides all of your system behind esoteric CLIs and config files that just make everything a nightmare to manage.
Opinions are fine and you may as well hate K8s or any tech being discussed. I just hope to see less of "people who like this just because they don't know any better vs the smart ones like me who know The Truth™" comments.
I'm trying to grapple with the fact that k8s is apparently ultra popular technology, yet every time I try to read up on it or start using it, it quickly turns into a suck fest in terms of learning curve and complexity. So my conclusion is that the people using it "for fun" must have acquired a taste for it first by using it all day at work and I wanted to see if anyone who acquired the taste might chime in as to what made them see the light.
I've very recently moved over to a home k3s cluster--a couple old desktops and some relatively new ARM SBCs with NVMe slots. I don't use it at work, at least not directly, but it's been the least-painful solution I've found to run stuff; once you understand the model (not trivial but not crazy) and find the headspace to explain why things in k8s are the way they are, reasoning through it is pretty easy (which is really just the story of any moderately complex system, exacerbated in a few ways because of some Google-attitude carryover), and I can also yank half the machines out of the cluster before an application fails.
(Both Swarm and Nomad are fine tools, but the lack of easy resources to get up to speed led to me failing them out of consideration.)
In the context of local or home servers: my raspberry pi currently has like a 3 year uptime now, but if it randomly croaked I'd just buy a new raspberry pi, docker pull the image, and get it up and running again.
So you are saying the main benefit of k8s is that if some of your hardware in the cluster dies then your services are automatically still up and running? Isn't a home k8s setup vulnerable to the same failure as well if the master node gets smoked?
Seems like a lot of complexity cost for something that happens extremely rarely in a home setup at least (I could see how it would be useful for high reliability cloud systems).
The benefit to me is if one of my nodes physically dies i have enough room on my cluster to still run all my pods, they just automagically reschedule to another node.
If my master dies then I have to do a little extra work, but not much.
- I have three control planes, not one, for about 20 x86-64 and 24 arm64 cores and ~160GB of RAM across 7 nodes. For much the same reason as my NAS uses raidz2, I don't have to drop everything and fix the universe if something fails because Home Assistant just keeps going.
- With Longhorn, I have three instances of every data volume in the cluster. Longhorn also provides incremental data backups to an S3 server--Minio on the NAS, in my case, which replicates to an offsite.
- I can run things like CrunchyData's Postgres operator, which provides a solid baseline setup and additional features like point-in-time restore--and as I run personal but public-facing apps on the cluster this is beneficial.
- Logging and monitoring are centralized and were easy to do. I don't have to go attach to a container, I just go look at Grafana.
- I have a consistent interface for working with all my applications, and I like it more than Swarm's by a lot.
By having this cluster, made out of spare computers, some cheap NVMe drives, and a couple SBCs I wanted to experiment with anyway,I've removed everything I care personally about from cloud providers except for offsite backup and my email sending. I know where it is, I can hack on projects trivially, and I know it'll be there tomorrow so long as the house doesn't burn down.
And I didn't learn any of k8s at work.
-can run any workload
-on any hardware
-at any scale
-with load balancers
-and persistent disks
-and auto scaling
-and resource limits
-and certificate management
-and no single point of failure
-and etc.