Show HN: kenv – an environment file injector for Kubernetes resources | Hacker News Reader

Show HN: kenv – an environment file injector for Kubernetes resources

github.com

49 pointsnextrevision14 comments

Helm is another great option here.

Helm uses Go templates. You define your variables in a top level values.yaml, and then install the chart to your cluster. The variables get expanded on chart install.

Here is a snippet from a deployment:

    env:
        {{- range $key, $value := .Values.my_env_vars }}
          - name: {{ $key }}
            value: {{ $value  | quote }}
        {{ end }}

Man, this is definitely useful, but I hope you'd agree with me when I say that I hope you can safely get rid of this tool in the near future. I really want to see a better way of reusing resources in k8s (for example, a first-class Environment object that mimics /etc/environment).

I'd really like to get rid of my custom hacky kubectl wrapper scripts and either just use a cluster add-in or have a kubectl build-in processor that helps reuse.

If you know Ruby and how class inheritance work and don't mind reading source code, check out my https://github.com/matsuri-rb/matsuri

I wrote that one after putting into production: bash scripts (for Docker), Docker Compose, AWS ECS, and finally K8S. It was designed to reuse Kubernetes resources, but doing it in a way where it takes into account differences across environments.

Some of the code is old though. There is a whole part that brings up K8S in a container, which with microkube, nanokube, and kubeadm should just be taken out completely of Matsuri. It would also be great if I get an example repo written as a starter kit for people.

That said: have you checked out Helm? Helm overlaps Matsuri in functionality, is better documented and doesn't require you to know Ruby.

I might as well plug in something I wrote to solve this problem in a different way. Warning: the project is undocumented, and require knowledge of Ruby. Obviously, if you don't like Ruby or like reading code, this will not work for you.

https://github.com/matsuri-rb/matsuri

Matsuri is built on the idea that K8S is a building block for higher-level tools.

The gist is that rather than trying to template and substitute values, it uses Ruby class inheritance and mixins to construct the manifest being sent up to Kubernetes. Each key-value in the manifest schema is broken down into smaller, composable chunks. A single method `spec` composes it all together at the top.

To customize, you use Ruby method overrides to override part or all of the methods used to generate specifications. This also allows you to mixin modules that partially override methods describing the manifest. This allows the YAML or JSON manifest to be programmatically constructed with a Turing-complete language rather than templated.

The result is a single tool that allows you to describe pods (and services, etc.) that may require different things yet share some common functionality. For example, dev pods need to mount the source tree from the host and doesn't need secrets, doesn't need replication; staging and production might require volume mounts elsewhere and require secrets; staging might use a replication of n=1 while prod might be using replication n > 1; prod might have affinity, anti-affinity, and make use of node labelling while dev and staging do not.

Matsuri also has an abstraction that groups collections of pods, services, and other K8S resources together into an "app". It also has standard hooks for common operations, such as attaching to containers, getting logs, etc.

I have not updated Matsuri since around Feb of 2016, when I left the project this was extracted from. Sometime in the next six months or so, I'll be updating it again when I get my team's current project onto GCP.

An alternative is to use Jsonnet [0] (a JSON/YAML templating language).

You can define an inheritance relationship, where your template inherits pretty much everything EXCEPT the environment variables. Or the jsonnet cli lets you specify variables to be replaced in the template.

There's some kubernetes flavored examples in [1].

[0] http://jsonnet.org/

[1] https://github.com/google/jsonnet/tree/master/case_studies/k...

We've been using Jsonnet. It's saved much boilerplate and makes the syntax more concise. In our case we are taking advantage of the +: operator to provide a compositional DSL.

Container(foo, image) + HealthCheck(port) + EnvVar(VAR,value) + UseVolume(foo, /var/lib)

Sounds neat! Have you open-sourced this?

The Stackanetes project[0] has been using KPM and Jsonette in order to template out a fully configurable OpenStack cluster running on Kubernetes. It's a pretty awesome achievement. The project is pretty far along and I've even seen them demo live migration of VMs across Kubernetes workers.

[0]: https://github.com/stackanetes/stackanetes

What is the use case for this when Kubernetes already has things like ConfigMaps and Secrets?

It looks like it builds on those resources, but lets you deal with them via a simple dotenv-style file, rather than full YAML. Could be helpful for integration with those kinds of files or if you're migrating from an older .profile-based config.

This was prompted largely by a move from Docker Compose, where we used env_file quite a bit. kenv will both create the Secret/ConfigMap resource as well as inject the env lookups into an existing Kubernetes resource (deployment, replicaset, daemonset, or replication controller). This allows us to keep our environment specific configuration alongside the app and just inject at deploy time.

Very cool! I created a similar project, ktmpl[1], which does require a template (as the name implies). It's a client side implementation of the proposal for templates[2] that has been accepted for implementation in the Kubernetes apiserver directly. The benefit is that ktmpl will eventually be obsoleted by Kubernetes itself, but you can use the same templates.

[1] https://github.com/InQuicker/ktmpl [2] https://github.com/kubernetes/kubernetes/blob/master/docs/pr...

I've just gone through the process of creating kubernetes configs for our entire infrastructure. I essentially created an identical set of scripts myself to deal with all our existing configs. Would have been nice to have this 4 weeks ago.

I just do it with python, you can dynamically modify anything in a resource pretty easily with the pyyaml module.

Reply on news.ycombinator.com