Hacker News Reader

Comment by caw | Hacker News Reader

My coworker and I have been fighting with Docker for the last few weeks like the OP, so maybe HN can help us here. Right now we use Chef to provision the hosts and we run services normally. Now we're trying to introduce Docker into our test environment, so we don't need to replicate a multi-node database cluster out of machines. In the process, it seems like we're having to repeat most of our Chef recipes to create the config files, and put them in the appropriate locations to let Docker mount them as a shared volume.

Are we doing it wrong? Should everything be Docker and all of our Chef recipes should be written for Docker? Or is this right and just a natural pain of having both bare metal processes and Dockerized processes? Unlike the article, so far everything is 1 process or application per container, not a full OS.

View on HN

The thing about Chef is that it tries to simplify the process of configuring and installing many applications that might run on a server.

Docker pretty much removes the the need for such complexity. Don't think of it as a provisioning tool, like chef. Don't think of it as a VM. Think of it as an isolated filesystem and process environment that eliminates the complexity of supporting multiple, disparate application stacks.

The storage for persistent storage with Docker is less compelling, because now you have shared resources between containers that should be isolated. That said, we run Cassandra clusters in Docker and have no trouble.

Also, If you want to re-use the work you've done with Chef, there are chef plugins for Docker which make it easy to provision Docker images with chef (solo, I think)

I follow that I'm not provisioning a full node, and parts of Chef installations are definitely unneeded, but where I'm stuck is I still have a config file that I need to control.

In the case of Cassandra I'd have certain cassandra.yml tweaks I need to make, possibly different for beta/staging/prod, and docker-cassandra needs that config file. I also need to spawn the containers with the right setup, such that the right ports get exposed and shared volumes get mounted so the configs get read in. The spawning is a slightly different ops problem (do I want a homogenous cluster where I just spawn containers somewhere as needed, or do I want certain containers on certain machine types), but it seems like something that would fall in the domain of a provisioning tool.

Should I just be building the configs into the container and rebuild the container if they need to change? The isolation of the process seems at odds with my desire to have a semi-dynamic, centrally managed configuration.

I'm also considering that Chef itself is overkill to manage Docker containers and there are other tools more well suited to managing Docker-based infrastructure, but I'm not sure what they are.

I would go with building the config into the image and tagging the resulting image with the environment name, if that's possible.

We've been using chef in the past to manage docker, which never felt very good because the main cookbook was somewhat buggy (but it has gotten better recently).

We now use mesos+marathon(+chronos for deployment) for running our docker container, which sounds much more overkill than chef, but turned out to work much better and cleaner.

Your best options are: a. environment varaibles (if that's an option) b. mount the file when you launch the container

I'm using (b) for our nginx configs, you mount only the single config file as read-only, then let chef copy up the config before mounting/launching.

Take a look at confd, a single statically-linked executable that can expand config file templates using environment variables:

https://github.com/kelseyhightower/confd

Basically, you add confd into your Docker image and execute it at runtime to do just-in-time config file generation. Here's an example for nginx:

https://github.com/grosskur/nginx-confd-dockerfile

You can create separate environment variable files for beta/staging/prod and pass --env-file to "docker run".

This lets you use the same Docker image across all your environments and avoid the operational complexity of mounting config files in Docker volumes.

You may want to look at how the docker-registry image is configured with environment variables. They have one image that can be used with files/S3/ceph/more configurations that are selected with variables. It's a pretty elegant way of having many configuration options in my opinion, except you still need to figure out how to 'docker run' the image.

https://github.com/docker/docker-registry

Reply on news.ycombinator.com