Hacker News Reader

Comment by stavros | Hacker News Reader

Hey everyone! I have a home server that runs some apps, and I've been installing them directly, but they kept breaking on every update. I wanted to Dockerize them, but I needed something that would manage all the containers without me having to ever log into the machine.

This also worked very well for work, where we have some simple services and scripts that run constantly on a micro AWS server. It's made deployments completely automated and works really well, and now people can deploy their own services just by adding a line to a config instead of having to learn a whole complicated system or SSH in and make changes manually.

I thought I'd share this with you, in case it was useful to you too.

View on HN

> I needed something that would manage all the containers without me having to ever log into the machine.

Not saying this would at all replace Harbormaster, but with DOCKER_HOST or `docker context` one can easily run docker and docker-compose commands without "ever logging in to the machine". Well, it does use SSH under the hood but this here seems more of a UX issue so there you go.

Discovering the DOCKER_HOST env var (changes the daemon socket) has made my usage of docker stuff much more powerful. Think "spawn a container on the machine with bad data" à la Bryan Cantrill at Joyent.

Hmm, doesn't that connect your local Docker client to the remote Docker daemon? My goal isn't "don't SSH to the machine" specifically, but "don't have state on the machine that isn't tracked in a repo somewhere", and this seems like it would fail that requirement.

Have you tried NixOS?

I have, and it's really good, but it needs some investment in creating packages (if they don't exist) and has some annoyances (eg you can't talk to the network to preserve determinism). It felt a bit too heavy-handed for a few processes. We also used to use it at work extensively for all our production but migrated off it after various difficulties (not bugs, just things like having its own language).

You can talk to the network, either through the escape hatch or provided fetch utilities, which tend to require checksums. But you do have to keep the result deterministic.

Agreed on it being a bit too heavy-handed, and the tooling isn't very helpful for dealing with it unless you're neck-deep into the ecosystem already.

What do you think isn't getting tracked?

You could put your SSH server configuration in a repo. You could put your SSH authorization key in a repo. You could even put your private key in a repo if you really wanted.

How do you track what's supposed to run and what's not, for example? Or the environment variables, or anything else you can set through the cli.

Docker Compose is designed for this.

Yep, that's why Harbormaster uses it.

The killer feature of harbormaster is watching the remote repository. Can docker-compose do that? If it can, I should just leverage that feature instead of harbormaster!

The nicety here on harbormaster seems to be that there are some ways to use the same code as a template in which specific differences are dynamically inserted by harbormaster. I'm not aware of how you could use docker-compose (without swarm) to accomplish this, unless you start doing a lot of bash stuff.

I also appreciate that harbormaster offers opinions on secrets management.

For me, I don't define any variables via the cli, i put them all in the docker-compose.yml or accompanying .env file, that way it's a simple `docker-compose up` to deploy. Then I can track these files via git, and deploy to remote docker hosts using docker-machine, which effectively sets the DOCKER_HOST env var.

While I haven't used it personally, there is [0] Watchtower which aims to automate updating docker containers.

[0] https://github.com/containrrr/watchtower

What do you mean?

You run what's supposed to run the same way you would anything else. It's the same for the environment variables.

How would you track what's supposed to run and what's not for Docker? Using the `DOCKER_HOST` environment variable to connect over SSH is the exact same way.

I wouldn't. That's why I wrote Harbormaster, so I can track what's running and what isn't.

> "don't have state on the machine that isn't tracked in a repo somewhere"

https://docs.chef.io/chef_solo/

> chef-solo is a command that executes Chef Infra Client in a way that does not require the Chef Infra Server in order to converge cookbooks.

I have never used Chef. This is babble to me.

> Chef Infra is a powerful automation platform that transforms infrastructure into code. Whether you’re operating in the cloud, on-premises, or in a hybrid environment, Chef Infra automates how infrastructure is configured, deployed, and managed across your network, no matter its size.

In an imprecise nutshell: You specify what needs to exist on the target system using Chef's DSL and Chef client will converge the state of the target to the desired one.

chef-solo is a command that applies configuration statements on a host without using a separate metadata server.

Chef is a configuration management system. It lets you define lists of things to do called "cookbooks" (analogous to Ansible "playbooks" etc.).

To "converge" is to run something until it is stable. This terminology, I think, comes from the early configuration managemnt system CFEngine, where you write your configuration in declarative(-ish) "make it so this is true" steps, instead of imperative "perform this change" steps the way that a shell script would do. See e.g. https://www.usenix.org/legacy/publications/library/proceedin...

chef-solo is a command that executes Chef's client - the thing that actually makes configuration changes, that is to say, "converges cookbooks" - in a way that does not require a server component. The normal way of deploying Chef is that a server runs things on clients, the machines being configured, but chef-solo is appropriate for the case where there is no such distinction and there's just one machine where you wish to run Chef.

What is the "Bryan Cantrill at Joyent" you're referring to?

The technology analogy is Manta, which Bryan covers in at least one if not several popular talks on YouTube, in particular about contanerization.

He has a lot to say about zones and jails and chroot predating docker, and why docker and co. "won" so to speak.

Not (I think) the exacttalk/blog post gp was thinking of - but worth watching IMNHO:

"Debugging Under Fire: Keep your Head when Systems have Lost their Mind • Bryan Cantrill • GOTO 2017" https://youtu.be/30jNsCVLpAE

Ed: oh, here we go I think?

> Running Aground: Debugging Docker in Production Bryan Cantrill19,102 views16 Jan 2018 Talk originally given at DockerCon '15, which (despite being a popular presentation and still broadly current) Docker Inc. has elected to delist.

https://www.youtube.com/watch?v=AdMqCUhvRz8

awesome, thanks!

Interesting case. But did you look at other systems before this? I myself use caprover[1] for a small server deployment. 1: https://caprover.com/

I have used Dokku, Kubernetes, a bit of Nomad, some Dokku-alikes, etc, but none of them did things exactly like I wanted (the single configuration file per server was a big requirement, as I want to know exactly what's running on a server).

I use caprover on my DO instance and it works great. Web apps, twitter/reddit bots, even ZNC.

Reply on news.ycombinator.com