How would you configure a cluster? I’m trying to explore lightweight alternatives to kubernetes, such as docker swarm, but I think that the options are limited if you must support clusters with equivalent of pods and services at least.
Right now I'm running,
- podman, with quadlet to orchestrate both single containers and `pods` using their k8s-compatible yaml definition
- systemd for other services - you can control and harden services via systemd pretty well (see https://news.ycombinator.com/item?id=44937550 from the other day). I prefer using systemd directly for Java services over containers, seems to work better imo
- Pyinfra (https://pyinfra.com/) to manage and provision the VMs and services
- Fedora CoreOS as an immutable base OS with regular automatic updates
All seems to be working really well.
Yes. Though unless you have a very dynamic environment maybe statically assigning containers to hosts isn't an insurmountable burden?
So, unless you have a service that requires a fixed number of running instances that is not the same count as the number of servers, I would argue that maybe you don't need Kubernetes.
For example, I built up a Django web application and a set of Celery workers, and just have the same pod running on 8 servers, and I just use an Ansible playbook that creates the podman pod and runs the containers in the pod.
My setup is a bit clunky (having a Hetzner cloud instance as controller and a local server as a node throught Tailscale), from which I get an occasional strange error that k3s pods fail to resolve another pod's domain without me having to re-create the DNS resolver system pod, and that I so far failed at getting Velero backups to work with k3s's local storage providers, but otherwise it is pretty decent.
YMMV
Contrast:
ansible -i server1,server2,server3 deploy_fake_pods.yaml
ssh server1 sudo shutdown -h now
# aww, too bad, now your pods on server1 are no longer
kubectl apply -f deployment.yaml
for i in $(kubectl get nodes -o jsonpath='{.status.hostIP}'); do
ssh $i sudo shutdown -h now
sleep 120
done
# nothing has changed except you have fresh Nodes
microk8s seems exceedingly simple to setup and use. k3s is easy as well.