Running Too Much Kubernetes | Hacker News Reader

Running Too Much Kubernetes

blog.fahhem.com

20 pointsfahhem9 comments

Same! 3x raspberry pis and 2x x86 cluster.

That's the advantage of a homelab environment - the whole monolith/microservice complexity & technical debt thing just doesn't matter in that context. Anything interesting is fair game.

One more more "Why" motivation to add: Portability. I can stick it on my desktop (minikube), my homelab cluster, or a DIY cloud cluster or CP. Feels a bit like docker in that sense - common lowest denominator to build against.

> If anyone knows of a CLI that would give me that level of information, please leave a comment.

k9s tool is much better. Takes a bit of getting used to since it has the whole ssh old school vibe but its 10x more useful that the official dash.

>prometheus/alertmanager: I don't have this setup

I've found Promtail daemonset connected to Loki/Grafana to be good. Especially if you stick ever deployment into it's own namespace, since you can then use that as label in Grafana/Loki. Had a couple issues that I could only track down once I had external logging.

https://grafana.com/docs/loki/latest/clients/promtail/instal...

Another neat trick is using Kompose to translate docker-compose to yaml. Helm works too, but I find them a little opaque.

Thanks for the comment!

> k9s tool is much better

I know about k9s, but it doesn't have a "Workloads" tab that shows both failing pods and their source (deployment, daemonset, etc).

> I've found Promtail daemonset connected to Loki/Grafana to be good.

I'll look into this, thanks!

> Another neat trick is using Kompose to translate docker-compose to yaml. Helm works too, but I find them a little opaque.

I didn't mention how I wrote/got all the services' yaml, but it was a combination of kompose (and one in instance an online tool that converted a 'docker run' command I had hand-crafted a while ago to docker-compose, to chain into Kompose), chatgpt for the basic ones, and helmCharts via kustomize.

The beauty of kube is the consistency and portability of charts and deploys between clusters (when you aren’t accounting for cloud specific stuff like IAM, LB, etc). So I don’t find this to be as crazy as it initially sounds.

A couple of years ago, I was dabbling in the cloud at AWS, and I initially decided to run a MediaWiki server with its associated stack, such as MariaDB and Apache or something, on Debian. And I found a good Bitnami AMI to do it with.

But what I really wanted to do was to spin up Kubernetes and learn about microservices in some way. The techie guys I polled said I wouldn't need k8s at all for my use case and I should scrap trying to do k8s because it wouldn't do me any good. I'm still sort of miffed at that; I would've rather learned some Kubernetes in the first place rather than just become a petty sysadmin again, so a replacement project that actually did make use of k8s would've been helpful.

Anyway I chose to double down on my chosen career in the education industry, so I'm letting my sysadmin/DevOps skills lie fallow until retirement.

Honestly, getting started with k8s is easy. Get started with minikube on local desktop. Once you get the feel for it, try configuring a multi-node cluster using k3s on DigitalOcean or Hetzner and eventually using managed AKS/EKS/GKE if you feel the need.

I was surprised when I setup k3d the first time, I had a multi-node cluster on my (4 year old) desktop and was able to deploy a multi-node service I wanted to test out without any problems. I would highly recommend k3d if you don't want to pay for cloud servers and don't actually need either high-availability or the CPU power of 3 servers. Also, your desktop is probably more powerful than any base-tier server.

What do you use for load balancing and ingress? Do you rely on service discovery in this setup? I always get stuck figuring out how to get traffic into the cluster properly while allowing for graceful fault tolerance.

I’ve had success with MetalLB to dynamically announce a single IP for the ingress controller but that feels a bit overkill for a simple lab setup.

Are you asking about my "clusters" or k3d?

For my clusters, they're all single-node and therefore not fault-tolerant clusters, so I don't worry about graceful fault tolerance. I just use nginx-ingress which opens a port on the node (via iptables, so don't expect `lsof` to help!). Then my router port-forwards so my home IP:80 eventually hits my NUC

I'm interested in Kubernetes the same way I'm interested in Erlang. I find distributed systems so much fun to reason about, and trying to understand certain implementation details and design choices shed a light on the reasoning of very smart people on very hard problems.

Now, would I want to deal with those technologies all day? Not really. I also like diesel electric locomotives, but I don't wanna run a fleet of them or run through their maintenance sheets.

Reply on news.ycombinator.com