Comment by WestCoastJustin | Hacker News Reader

Comment by WestCoastJustin | Hacker News Reader

WestCoastJustin

There are a bunch of cool and exciting things about Docker and containers in general. Docker just automates many manual tasks, essentially creating an abstraction layer above many manual and tricky steps. Similar to how Vagrant abstracted away many tasks with automating virtual machine deployment.

There are a couple really cool things though, such as the Docker push/pull functionality, giving you access to Docker images, Vagrant does something similar to Vagrant boxes, but dockers goes a little further. But the killer idea, is that you do not need an OS in these Docker containers. You can basically have a statically linked binary, or a rails app, or sshd daemon, and that's it. You can then use cgroups to finely tune your resource levels (cpu, memory, bandwidth, etc) on this container.

These concepts have been around since at least 2008 (LXC [1], was released into 2.6.24 [2]), Docker just streamlines the process greatly. As a side note, it looks like Google has been heavily using Linux containers for years! There is a Wired article [3], which talks about the orchestration engine they use to deploy containers across their cells. Google's John Wilkes even does a talk, entitle "Cluster management at Google" [4], where he discusses the next generation of the orchestration engine.

p.s. It looks like there might even be "live container migration" in the pipeline [5]. This would allow you to move a container from machineA to machineB without any downtime, similar to Xen migrate or VMware live motion.

[1] http://en.wikipedia.org/wiki/LXC

[2] http://kernelnewbies.org/Linux_2_6_24

[3] http://www.wired.com/wiredenterprise/2013/03/google-borg-twi...

[4] http://www.youtube.com/watch?v=0ZFMlO98Jkc

[5] https://www.youtube.com/watch?feature=player_detailpage&v=LD...

  But the killer idea, is that you do not need an OS in these  
  Docker containers. You can basically have a statically 
  linked binary, or a rails app, or sshd daemon, and that's it.

Is there documentation of this somewhere? I thought docker relied on having an operating system installed, even if just a minimal busybox based system.

I'm not sure if there is documentation at this time, but there was a Google Hangout, entitled "Docker: the Linux container runtime" [1] (minute 24), where Solomon Hykes and Jérôme Petazzoni (from dotCloud, makers of Docker) talk about this very thing.

[1] https://www.youtube.com/watch?feature=player_detailpage&v=LD...

A docker container runs in an operating system, but doesn't contain one.

Well, that depends on your definition of "operating system".

Is an OS just the kernel? Then yes, Docker doesn't contain an OS.

Is the OS the userland? E.g. the init process + the shell + the C library + other runtimes? Then most Docker containers actually do contain an OS. Yes you can get rid of most of them by making static binaries, but why would you want to? Making static binaries with anything besides Go is a pain, and good luck trying to statically link MySQL or Redis into your binary.

I don't think you need the init process even without a statically linked binary, and I don't see why would having a bunch of libraries suddenly make it an OS. The linking happens "outside" the container - in the shared kernel.

I think docker takes the place of init in the container, so no.

You are totally correct.

Lets take sshd for example, you would need to create a skeleton directory structure, /bin, /dev, /lib, etc in your rootfs container, have /bin/bash, a whack of /libs, some /dev devices, and then your sshd application. So an extremely minimalistic rootfs with only the sshd requirements. You could find these my running 'lsof -p `pidof sshd`' , maybe 'ldd /usr/sbin/sshd' (and all their dependencies) too, which quickly snowballs.

Hopefully this explains where I was going with that. Also, this is not a simple task to try and strip off these services into self contained entities. There are lots of hidden issues, like how do we handle logging? Should the container have syslog too? So, there is work that needs to happen, I just like the idea of not running a full fledged OS in a container.

This is solved by the union-based filesystem (AUFS) that Docker uses. You start with one minimal rootfs like you describe, given in the "base" image, then when you install sshd you get copy-on-write semantics. So the sshd container gets its own syslog files separate from any other container.

Your original comment talks about how long LXC has existed but AUFS is one key component of Docker that became part of mainline Linux much more recently.

> AUFS ... became part of mainline Linux much more recently

AUFS is not part of mainline kernel. Many distros include it (Debian, Ubuntu). But some don't (Fedora).

http://en.wikipedia.org/wiki/Aufs

Taking that idea even further, how cool would it be to have one binary that ran a pre-built docker container?

FreeBSD jails already provided those containers for over 10 years. Even chroot is some kind of container.

http://en.m.wikipedia.org/wiki/Operating_system-level_virtua...

Reply on news.ycombinator.com