Anatomy of My Kubernetes Cluster | Hacker News Reader

Anatomy of My Kubernetes Cluster

ttt.io

138 pointsastefanutti36 comments

Open articleView on HN

More software engineers should build Kubernetes clusters like this at home...

So they don't have to do it at their work.

(Unless they work at a large enough company to justify one.)

A $500 per person "get it out of your system at home" fund, I love it

$500 might be on the low side, but it would cover a good percentage of the cost. :)

This is neat. I've been doing the same with Pi 1s and 2s for a few years (http://github.com/rcarmo/raspi-cluster) but stopped short of upgrading to 3s and 4s because of heat/power/cost.

I've found that it is possible to run k3s on SD cards quite well if you're patient, but that the Pi 2 master node spent a _lot_ of time just doing internal stuff even on an otherwise idle cluster, so I went back to Swarm.

(I also had quite a bit of trouble with setting up a container registry, which was hard and undocumented in the first few releases of k3s. Ingress was also a bit of a pain, it's much easier to get traefik working in Swarm...)

This seems to be a Kubernetes thing. Why the hell does it idle using 25% of my 4 CPUs?

> This seems to be a Kubernetes thing. Why the hell does it idle using 25% of my 4 CPUs?

Remind me why we need this technology again?

That makes no sense. Try looking at logs and such at least.

The logs don't say much. It's mostly the API server getting hammered by nodes and various components.

Ok so it's just the API servers. Not every node which is what it sounded like. Typically you wouldn't run normal workloads on the same host as the API. Although these quick "how I did it" articles have a tendency to skip best practice. Still sounds like something isn't right in your environment. They shouldn't be hammered that hard unless you have a very large cluster.

As a general thought, this kind of power supply seems like what the author was first looking for but couldn't source at the time:

https://www.ebay.com.au/itm/Switching-Power-Supply-DC-5V-12V...

It's a 5V, 20Amp one (thus 100W). There seem to be several of them on Ebay from various vendors.

Using that would mean the 12V laptop power supply, and buck converter to change the 12V to 5V wouldn't be needed. Would probably bring down the cost a little bit too.

Wow it's really beautiful, looks like a real rack.

I wonder, the wires seems to be very hard to bend. When I built custom PC, there's custom build soft power supply / SATA wires for sale. Maybe there should be similar things for USB or other wires that could be easier to tuck in the case.

Wonder what a setup using PiBlades might look like

http://www.bitscope.com/product/blade/?p=about

"59 Watts, That’s well above the power supplied by any AC mains to 5V DC external power supply I could find"

MeanWell has their RS-75-5, which is 60 Watts. Usually around $20.

If you’d like to do something similar, check out the Turing Pi, which uses RPI compute modules and runs off a ATX power supply.

https://turingpi.com/

Network throughput should be tested with iperf ( https://iperf.fr/ ), certainly not scp

Real question coming from a non-k8-user. What would you use something like this for? Server hosting or?

Ultimately the question is, "how much of the total cpu/memory is used just running kubernetes?" I know prior to rpi4, clusters like this were basically demo/education/bragging rights projects.

It's great for edge computing. Chick-fil-a is using something like this at all their restaurants.

https://medium.com/@cfatechblog/bare-metal-k8s-clustering-at...

https://medium.com/@cfatechblog/edge-computing-at-chick-fil-...

It looks like this is being used more as an educational tool for the authors daughter than it is being used for a production workload.

Kube is basically fancy systemd built to run on multiple machines. If you wanted to you could use this for anything you could use one bigger machine and systemd to do.

I started playing with Elixir on mine, but installing Elixir on ARM is enough of a pain that I opted for a Docker image instead so I only had to build it once.

If you're serious about messing around with your own k8s cluster, you're way better off just using virtualization. If you really want to run on your own hardware, grab an old laptop or desktop computer and KVM and have at it.

The Pis are just fun toys for people who enjoy tinkering with hardware. This kind of setup isn't really useful (or even educational in any kind of general way outside of the Pi specific details), but some people enjoy it.

Nicely done. I've been meaning to dive into k8s clusters (with either k3s or kind) -- what is the advantage of building a Pi cluster over something like multiple vagrant nodes? Not criticizing at all, just trying to understand.

Not the OP (see my other comment). For me it was the cheap hardware and the fun aspects of the project. It's a tangible thing.

But I also like ARM chips and do occasional embedded development, so I had other motives :)

Power supply:

I took my desire to build a Pi cluster as an excuse to upgrade to a bigger, newer Anker charging hub.

Beautiful. Like a work of art.

Why tho? Why not just launch N KVM virtual machines on your linux box? This literally takes minutes, unlike this project which probably took days. And it's much faster once you get it up and running.

Because tinkering is fun and fun is good.

It just seems like there are much more interesting projects available for "tinkering" than building a slow, barely usable Kubernetes cluster.

Like what? Interested to hear what you deem interesting enough to tinker on.

Just about anything not Kubernetes related. I'm currently building a cat toy which combines object detection and reverse kinematics to play with my cat using a laser pointer when she steps into view of the camera.

Why tho? Why not just use a laser pointer yourself? This literally takes minutes, unlike this project which probably took days. And it's much faster once you get it up and running.

AI and robotics tho. Way cooler than some stupid Kubernetes that doesn't even work well on this hardware.

I know it's a novel concept that not many know about, but: Everyone has their own interests and curiosities.

It's just a purely personal preference thing. :)

Because this is literally what "Hacker News" should be about.

Nice write-up.

Reply on news.ycombinator.com