Kubernetes is a cloud operating system
home.robusta.dev
home.robusta.dev
But while true & a reasonable framing, this view doesnt cover the most important thing about the kubernetes "operating system". Yes unlike many other cloud systems Kubernetes has pretty wide coverage of concepts it has under it's roof. Versus serving a purpose or two well, it, like an OS is a fairly general & comprehensive platform, with the capabilities you'd need for most tasks, & built in OS extensibility via CRDs. This is all trye enough but still shy of how we ought think about Kubernetes versus other things, what came before, cloud or no cloud, os or other.
Kubernetes is autonomic. Kubernetes, through a variety of controllers/operators, enacts all changes to the system. Rather than being, as most automation or other aoftware, a tool wielded by the user to make ends happen Kubernetes wields itself, & the user is simply stating what they want, makes their intent known, & Kubernetes makes sure that happens & stays that way. Taking people out of the loop of having to learn how to do computers, how to build a house themselves- Kubernetes knows how to build what you tell it. It's something so unlike software of any sort which came before, this ongoing (as opposed to single shot), autonomic (self maintaining), self-operating system, and I cannot contrast strongly enough how liberating it is to have the machines working for you, adjusting themselves, based off the manifest of inventory you've told it you want.
It's a totally different operational paradigm, beyond just a system & having developed a weak internal agency (a control loop & it's suite of tools to stay on happy paths through the loop), and it means way way less figuring out how to run the dozens of various whirligigs and gizmos that comprise the computing environment. Kubernetes is more than an operating system; it is a self-governing control loop.
I like the core idea of Kubernetes (eventually consistent declarative infrastructure) though I'm not fully sold on the implementation and the tooling around it.
Regarding the implementation and tooling, what's lacking in your opinion?
0. YAML everywhere. Augh.
1. Vanilla kubernetes by itself doesn't really do much; everyone basically assumes you'll use a cluster in a cloud somewhere that comes with all kinds of add-ons and functionality on top of the default. If you want to run an on-premises cluster, once you start adding external components to a Kubernetes cluster to make it actually useful, it starts getting complicated.
2. I don't think configuration management of the resources you do put in a cluster is a fully solved problem, especially if you depend on CRDs.
3. kubectl being context-sensitive is rather annoying. You basically can't share a kubeconfig between two terminal sessions without severe danger of footgunning.
4. If you just use Helm packages with defaults, you'll end up running software with very collision-prone UIDs and security contexts, so two separate components that you deploy can end up running as the same user on a host. I don't think that's good security design, but the system doesn't encourage you to do the correct thing.
5. All the "easy" tooling is basically just "run this magic incantation and it will do things for you". The culture around how things are done seem to encourage ignorance about what you're actually deploying. Many operations also modify cluster state and thus should be managed and not done willy-nilly, but that's also ignored by default. See also #2
Kubernetes has its upsides (something like ArgoCD is very nice to use when set up properly) but I guess my main beef with it is that it can make things look easy that actually aren't, and mislead its users.
Regarding (3) there are tools for that like kubie. I've still shot myself in the foot before.
I don't know how to write a file or install a program, I just tell MacOS to do this and it happens magically.
Is it the level of abstraction that is different?
Or are you saying that kubernetes lets you do this at a different scale or with different kinds of applications than you could before?
Kubernetes is level-triggered. If you could say that you want a certain file in a certain spot, the computer might be able to have choices for how to make it so. Maybe it looks in other directories. Maybe it downloads it via http or bittorrent. Maybe it scans your archives. If the hard drive dies, it can reget the file. If your sibling deletes the file, the computer can rebuild it. There's snapshotting policies it can setup on your behalf to help. There's records about how & where & when it pulled the file in from.
This is all an elaboration on autonomic processes, on Kubernetes maintaining state for you, letting it do the operations to give you what you asked for. It detects when the level is not right/monitors itself, & detects when it needs to trigger (level triggered) control. it has it's own toolkits to do the work for you, and you are merely registering your ask.
yes you might not fully understand the tool you are using & the full depth of a file system, but even as an end user you are doing the work in tbe conventional macOS model. abstracting the work away from the user, & using a general repeatable practice for declaring your intent (manifests) frees you from having to learn a panolopy of tools & systems, each unique an different (confinung ourselves just to this one example, a file manager, a web browser, a bittorrent client, a usenet client, a snapshot tool, file search system).
and it maintains itself, keeps working, consistently & with explanation, where-as most computers defy explanation: what scripts have been run, what the history is is unknown. by making a compact - letting the machine do the the work instead of us doing it- we have central bookkeeping of what our system really is, radically reshaping the clarity of what we have & what's going on.
these skills being portable across problem domains is a third ultra-strong benefit. almost all aspects of the world get maintained via manifests. learning something new looks similar to & used many of the same tools as what you did to setup the last tbing. this is not really an autonomic benefit, byt is a huge part of the win.
generalized autonomic system is a huge improvement versus shepparding our individual systems carefully forward.
This is what Kelsey Hightower is hinting at when he says Kubernetes is a platform for building distributed systems.
What enables that is the declarative/autonomic nature.
Declarative/autonomic systems aren't a fit for everything (e.g. for observability and monitoring the edge changes absolutely matter) but they're certainly a good fit for many things.
IMO there are a lot of parallels with functional programming.
In any event, strongly agree with everything you wrote. It's a good topic for another blog post!
You don't declaratively specify your intent that it should have certain contents.