Hacker News Reader

Comment by worldsayshi | Hacker News Reader

I'm not sure how podman fits into this thought but I feel what's needed to make kubernetes as easy to use as docker-compose is a bunch of sane defaults for things that you mostly don't want to care about for smaller projects. And then maybe an option to easily "eject" the configuration when the project grows.

There has to be some attempt at this right?

View on HN

I would recommend https://www.devspace.sh I use for just about everything. It also works well as a good gateway into the kubernetes world when coming from docker compose.

We're trying to do this at Kurtosis! (https://github.com/kurtosis-tech/kurtosis)

The idea is to write your environment in Starlark (a minimal Python subset) and we'll handle running it on Docker or Kubernetes for you.

We started with "reduce Kubernetes complexity" so we don't yet have the full Kubernetes ejection seat, but we're slowly marching towards it - feedback welcome!

Can you clarify what's hard about using kubernetes for smaller projects and what do you want to omit?

helm + go-templating in YAML is pure madness.

I've found k8s more manageable with straight manifests, or tooling to generate the manifests from some other source.

I do need to give jsonnet a hard look, as I hear it's less crazy than yaml+templates

You definitely don't have to use helm and go-templating for small projects. I never used it (other than from the user side to install 3-rd party software). Just ordinary yamls is absolutely fine. Kustomize might be useful for configmaps and secrets, but that's a small layer of complexity.

What I want to try to generate yamls is to write typescript code. Emitting JSON is natural in typescript. Adding types for schema compliance is doable. And then you've got full typescript power on top of it to remove any kind of duplication.

I agree with that approach, and it's one I want to try as well. I'm an old Perl guy, but the transition to being a JS/TS native to use modern interfaces + toolings has proven a pretty straightforward path, especially with nodejs systems side for scripts and what not.

The 'declarative' nature of YAML + configuration for defining infrastructure is definitely a good thing, but the tooling for producing those declarations are in a very rough state. I think a "TS => YAML => git => deploy" workflow for deployments could make using K8S a breeze, especially if there is a sensible library in TS that could provide the base classes for manifests, that could be subclassed or wrapped with defaults + definitions for the local environment

Sounds like cdk8s+

https://cdk8s.io/docs/latest/plus/

Thanks for that pointer, that does look excellent

There's a number of solutions. One is to use a maintained Helm Library Chart that exposes all the values most could ever want like The Helmet [1]. The other is to move over to something like Timoni that is analogous to Helm but with better templating [2].

[1] https://github.com/companyinfo/helm-charts/tree/main/charts/...

[2] https://timoni.sh/comparison/

Perhaps I'm biased by working on mostly big projects. I haven't tried a very small project recently so I might be on thin ice.

The thing that I would expect would be a bit complicated in a small project is networking rules, load balancing, mounting directory from host. That kind of thing.

Kubernetes networking requires installing CNI plugin. Something simple like flannel will work fine and does not require any configuration. That's the only required addon for kubernetes, I think.

With docker you'll probably bind container port to the host. With kubernetes you can create Service of type NodePort and it's exactly the same.

Load balancing: I'm not really sure what do you mean, what do you use for docker load balancing? Just expose your kubernetes host to the Internet and that's about it. Or hide it behind separate reverse proxy, if you want.

Mounting directory from the host is supported with kubernetes, no problems here. Use hostPath volume in the pod spec.

Besides CNI networking potentially encompasses Services, Endpoints/Slices, Load Balancers, Ingresses and most recently the Gateway API. I'd say that's quite a bit complex.

Kubernetes allows you to make complex networking, which is especially useful for cluster setups involving multiple nodes. But if you're comparing kubernetes with docker, docker does not have load balancers, ingresses, so you can just skip it, if you feel that it's too much of complexity. They're not mandatory.

k8s is a kit of (very useful) parts imho. The abstractions it presents are not human-usable. It needs some layer that adapts concepts that make intuitive sense to a human to its internal model. e.g. Vercel. You can make a tool for example that will deploy compose files onto a k8s cluster, and that thing will be totally useable by someone who already understands k8s. But give them raw k8s and they will spend a month scratching their head.

Are you talking about pods/deployments/etc or lower level abstractions like informers and things like that?

Abstractions are absolutely human-usable. My only issue is overly verbose yamls, but some people might even find it a good thing.

It's been a while but iirc my primary issues weren't about configuring a small project, it was the effort of setting up and administering a k8s cluster that was the problem.

With docker-compose you realize you need a persistent volume and go "volumes: ..." and you're good.

With your own k8s cluster you go, "okay I need a persistentVolumeClaim to a PersistentVolume with a particular StorageClass that my cluster needs to offer". Then you go learn the "Right Way" to set up a StorageClass and 2 hours later you finally have a persistent volume. Repeat every time you try to do something that would be one line in a docker-compose file.

If you have easy access to an administered k8s cluster then kubernetes becomes way more approachable. What would be ideal for me is an offering that presents itself as a batteries included k8s cluster running on your local machine. That way I could make pod definitions and run them without having to understand k8s internals while gaining the option of deploying on a real cluster in the future.

Setting up kubernetes is easy with kubeadm. There are other distros like k3s, but I don't have personal experience with them.

For storage, just use hostpath volume and that's about it. You don't need to deal with PersistentVolumes at all. I never used named docker volumes, honestly, and I don't understand why they're useful, I always prefer to explicitly specify disk location, so I can back it up, etc. PersistentVolumes are useful, when you have separate storage system and full-fledged multi-node cluster, but for simple setup, there's nothing wrong with hostpath volume, IMO. Just don't forget about backups.

Reply on news.ycombinator.com