I would argue that k3s on a vm is not too much harder to setup than a normal reverse proxy setup. And the configs are a bit easier to find on a whim when coming back after months.
Obviously a full self hosted setup of k8s is pretty large task but in other forms its a competitive option in the "simple" area.
My use case was pretty straightforward, a series of processing steps with the caveat that storing the actual results in a DB would be bad due to the blobs size. So, instead, the DB is used to signal downstream consumers files are ready and instead write the files in the ReadWriteMany volumes so that downstream files can simply read them.
I tried Longhorn, but even if I manage to get a volume showing up in the UI, it was never detected as healthy and after a while I refactored the workflow to use Apache Beam so that I could drop databases and volumes and run everything from within a single beefy VM.
Is it still an issue (it was a while ago TBH)?
If you are already familiar with k8s volumes and csi's its not a huge problem but if you arent its not worth learning if your goal is simple. At least in my opinion.
Yeah, that is also my opinion after that experience.
Also, it was really frustrating as the number of moving parts that have to play nice to each other is high.
Many hobby projects are just that, simple apis and can benefit from the simple declarative yamls that exist in say a sqlite db on the vm vs random config in /etc, /var and ip table scripts etc.
But trying to do anything out of that simple mold? just do it the simpler and easier traditional way.
I'd describe a really simple setup as this:
Pod: you put 1 container inside 1 pod - you can basically replace the word "container" with "pod". Let's say you have 1 backend in python and 1 frontend in React: you deploy 1 pod for your backend, and 1 pod for your frontend. The simplest way to deploy pods is not with "Pod" but with "Deployment" (1).
Service: Kubernetes really doesn't want you to connect directly to pods, it wants you to have a service in between. This is sensible because pods are ephemeral - they can be killed due to crashes, updates, etc. You define a service, point towards the pod, and all internal requests should be sent to the service, not to the pod. (2)
(1) https://kubernetes.io/docs/concepts/workloads/controllers/de...
(2) https://kubernetes.io/docs/concepts/services-networking/serv...
Note: technically, a pod is a group of containers. It can get more complex than this, but it's not unusual to have a group of 1 pod, especially in simpler setups.
In the deployments example, what gets me is they called 4 different things "nginx." It's hard to tell what does what. Like I've read 6 times what matchLabels does, and it's still not clear if that's supposed to mirror template.metadata.labels.app, or what happens if it doesn't. Sure I'd figure it out if I set up the real thing and messed with the config to see what it does, but it shouldn't take that.
Of course it's easy to follow tutorials without fully understanding how stuff works, and I did. Slightly more advanced things were a lot of trial and error. At some point I lost interest because I didn't really need this.
Keep in mind, for K8s to work you also need to containerize your application, even if it's just static files (or I guess you can break containment and do volume mounts, or some sort of static file resource if you're serving static content). For non-static applications, you still need to configure and run it, which is just harder with k8s/docker in the mix.
On the other hand if you are running thousands of containers it's hard to beat k8s, simpler solutions like ECS fall short at some point.