Hacker News Reader

Comment by tail_exchange | Hacker News Reader

Kubernetes is incredibly daunting for someone who only used VMs, but after spending some time with it, it's not really that hard. I also unironically run my personal stuff on K8s just because it's familiar to me, and it is also a good exercise for a very useful skill. People complain about the price, but you can easily get a 3-node cluster on GCP for less than $15/month.

View on HN

Yeah, I've been learning Kubernetes recently and a simple setup isn't too difficult. However, there's a lot of concepts involved with Kubernetes to grasp and I've definitely not learned everything there is. For example, Jobs still baffle me in regards to what is the purpose outside of CronJobs.

I use Jobs for helm hooks like integration testing and DB migrations associated with deployments. They work great for one off workloads like that.

You just beat me to it!

Jobs can get useful in more complex workflows. The most common use I’ve had for them us running a smoke test suite or a database migration as part of the larger workflow for a big Rails app.

But why wouldn't I just run a database migration as part of my CI flow? I don't understand why I would add a persistent element to my Kubernetes cluster for just a one-off task, since Jobs do not actually automatically remove themselves from the cluster. And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch.

Well, CI doesn't necessarily mean CD. In more than a few cases, I've found it advantageous to use CI to do automatic container builds, but manually attend to the deployment upgrades, particularly when things like db migrations are happening.

> Jobs do not actually automatically remove themselves from the cluster.

set `ttlSecondsAfterFinished` and they will

> And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch.

If you need to run a job on demand, you can always 'kubectl apply -f' the metadata, or you can create a cronjob and `kubectl create job --from` your cronjob. Or, if it needs to run at a particular part of your installation/upgrade workflow, use helm hooks. Jobs are intended to be a one-time thing, so the lack of functionality to repeatedly run them is intentional.

There's a TTLAfterFinished feature you can enable since 1.12. It's been in alpha forever. :/ Lets you add a ttl to your job spec to automatically clean up completed jobs.

I developed a simple wrapper around vegeta, called vajra, that can be given the URL to a file containing test URLs (in vegeta format), a target QPS/duration, and then easily run a load test.

Gets deployed as a k8s `Job` via GitLab (can be scheduled/on-demand), with a simple script echoing back the vegeta status every 1 minute. Jobs don't yet support automatic cleanup, so another simple GitLab job deletes the job from k8s upon completion. The execution log is anyway available in GitLab/Kibana.

All in all, an extremely simple way to run load tests against a service deployed in k8s.

What grew as a personal utility is now being used by many teams for quick load tests.

The term "batch job" (as opposed to "service") appears frequently in the Google K8s-like scheduler papers:

https://research.google/pubs/pub41684/ https://research.google/pubs/pub43438/

In other words, jobs seem like the go-to way to take advantage of unused CPU/memory.

One more unironic personal user here - I use k3s for cheap-as-possible.

I actually compared K3s with Docker Swarm in my master's thesis where i also wrote a simply Python tool to automate setting up servers for deploying containerized workloads on then.

While i can't really share the text (because it's all in Latvian), i did find that K3s is a pretty good option for when you want to use Kubernetes, but have pretty limited hardware resources. So much so, actually, that it used only slightly more resources than Docker Swarm (and, as a consequence, the deployments that were running on the nodes could serve almost equal amounts of requests under load).

Personally, i do believe that Kubernetes is sometimes overrated for simpler deployments, but if people don't mind the manifest syntax and are comfortable with the tool and its ecosystem, i think that K3s is probably the way to go in those situations (maybe with something like Portainer/dashboard running for graphical administration, if necessary). Such a good distribution, could probably even run it in prod with a HA setup.

> but you can easily get a 3-node cluster on GCP for less than $15/month.

Just wondering, how are you exposing your services from the cluster? (I assume you aren't using a load-balancer ingress)

Reply on news.ycombinator.com