Hacker News Reader

You don't need 3 control planes. If your control plane will be down for some time, your cluster will be OK. Even if your control plane will be down for prolonged time, your small cluster likely will survive. 3 control planes is good, but if you can't afford it, it's not necessary. Just make sure to backup etcd and keys for your control plane and learn how to recover it in case of hardware failure.

Kubernetes does not require storage cluster. If you need it, you can run it. But it's not required.

Please think about your arguments when comparing kubernetes with any other solution. If you don't need storage cluster with any other solution, then you don't need it with kubernetes.

Kubernetes at its minimum is 3GB RAM and 0.5 CPU for one control plane. That's all you need. Everything extra is not necessary.

People often try to build some kind of super-resilient cluster with kubernetes, which is suitable for huge company with hundreds of servers, while comparing it with one host docker compose. If your task is simple, don't make it more complex than necessary. Kubernetes can be simple or it can be complex, it depends on you.

What you do need for kubernetes is knowledge and experience. I spent several months tinkering with cluster before I dared to deploy something valuable there. Although I used kubeadm, with cloud offering few things are simpler.

> If you don't need storage cluster with any other solution, then you don't need it with kubernetes.

That is only half of the truth. If your application needs a database, and which application doesn't, you have to store its data somewhere. So you either need a storage cluster, or you open the can of warms called local volumes and (anti-)affinity rules.

For smaller application deployments, hosting a separate kubernetes cluster is extremely expensive in terms of time spend to debug infrastructure issues. k8s introduces orders of magnitudes of additional complexity to networking, dns, storage, routing, logging, etc. All in the name of making these exact same things easier for the developer. This can be a good separation of work, if you are large enough to afford a dedicated k8s maintenance team. But often it is not.

Keep your db outside the cluster. Use a managed db or just host it on its own server. You should be doing that even if not using k8s anyway. Don’t share memory and cpu with your db.

Regarding storage, try to build stateless apps that only need temporary storage that don’t need to outlive a request (like uploads). For long lived unchanging assets, you can try including in the container (ie. defaults.json, countries.json, samples.json, etc). For assets that need to be fresh, try to have the app generate at startup (eg. Email vírus definitions) or via init container. I think this will cover most use cases.

This won’t work with apps that depend on shared storage like Wordpress, but Wordpress is horrible and I can’t support you running it :-)

I'm not sure I follow you about local volumes and affinity rules. I followed instructions on kubernetes website to create local volume and pod which uses this local volume was scheduled on the proper node every time. I think it works with volumeBindingMode: WaitForFirstConsumer option.

So if you think that you need storage cluster either way, go ahead and deploy it. If you think that you're OK with single unreplicated database, just use local volume, it works. At least I've yet encounter any worm with this thing. For me it just worked.

Another option is to use local volumes and replicated database setup. So you can run, say, two database instances in a replicated mode on two servers, but storage will be simple local volume. It provides some reliability and availability benefits without complex storage cluster solutions. I used cloudnative pg operator for this and it worked for me. I guess one can configure ordinary postgres replication without any operator, I just don't have experience with that.

My opinion is that even single-node kubernetes cluster is worth it. That's because I have some experience and won't need time to learn basics. If someone does not have that experience, then I agree. Deploying kubernetes requires learning curve and that time might be spent for other things.

At least it's hard for me to imagine any kubernetes complication that won't arise with, say, docker. At the same time, kubernetes provides lots of polished software line ingress-nginx, cert-manager, cloudnative-pg which could save some time. And, most importantly, using kubernetes from the start, provides very easy option for moving into cloud, for scaling, for zero-downtime deployments. Those things are valuable by almost everyone.

> All-together I would've required 9 servers, 3 control pane, 3 app or pods, 3 storage. … Yes, k8s is a nice comfortable way, if you can afford it. But it’s expensive and wasteful of resources. And we do have a climate crisis.

You can run each of those roles on the same machine, achieving high availability from 3 total. For learning purposes you can run this all in VMs on a single machine.

There are plenty of good reasons not to try kubernetes when it doesn’t fit your use case, but the overhead it imposes is mostly in time and expertise. Claiming it’s more costly in compute resources, and especially contributing climate crisis, is reaching.

Someone is building a systemd based kublet replacement.

Why? Because Kubernetes has some great ideas on self-healing and resiliency.

But if you are small, and can work with a single instance for availability, then probably should stick with systemd. You could also use podman to get a small bit of the kubernetes ideas for working with containers.

The next level of scale wouldn’t be the full k8s but more like k0s, or k3s.

> Someone is building a systemd based kublet replacement.

I'm working on a product that uses something like this internally, basically, we provide multiple preview-environments in the same big server, these are managed by systemd.

This is the first time I get to think that this may be a project by itself, ping me if you have any ideas or want to get into a discussion on this topic.

I forgot who built it, but it isn’t a strict replacement. One of the rationale for doing so was identifying daemonsets as an anti-pattern.

K3s/k9s can be ran on a single server and gives you the declarative benefits and the entire helm ecosystem.

Everyone assumes you have to run multi node.

We went through these loops where I work due to the "you have to use k8s now" thing.

Your control plane and worker node can be the same machine.

Rancher + Rancher Kubernetes Engine made it a lot easier for us to manage as maintaining that is simply "run this docker container" and it deploys all the k8s gubbins for you. And k3s is a lot more lightweight on resources than k8s. Stuff like certificate rotation and kubernetes version upgrades are handled for us, it's reduced to clicking buttons on UI.

Storage can be local, it's just not / wasn't well explained in the documentation. PersistentVolume[Claim]s can use directories on the host, those PVs/PVCs can be shared between many pods and you can mount subdirectories into each pod using `subPath`.

You can get it all running on a laptop using a couple hundred MB of memory, all in all. So, not going to be ideal for running on a Raspberry Pi, but on par with a lightweight linux desktop environment in terms of resource usage.

I hate the complexity sometimes, but then I still end up having the same learning curves with things like SystemD when I use them anyway - every "run a bunch of stuff on my machine and restart it if it breaks" system seems to be rife with edge cases and behaviours which you have to dig deep into the documentation to see - and we don't _want_ to RTFM if we don't expressly _have_ to :-)

^ Although, okay, SystemD doesn't require you to have random HTTPS authenticated endpoints to talk to it and TLS certificates which expire every year and consists of _one_ init process rather than like 7 interdependent docker containers which can restart at any time and rely on container infrastructure to be running and argh. Okay yeah the flashbacks are happening and I agree with everything you've said again.

It's better to pay the cost of abstraction when you've good evidence of its necessity

With "kind" (kubernetes in docker) setting up a one node cluster is as easy as typing "kind create cluster"

Setting up a one node cluster is as easy as typing kubeadm init. You don't need anything fancy for that.

there are a few k8s distributions works on single server like microk8s, kind, and they are not difficult to setup on a single server.