Comment by shykes | Hacker News Reader

Comment by shykes | Hacker News Reader

Parent Full thread

You're right, today there are many systems which cannot run docker. There are several things we are doing to remediate that. Obviously there are no miracles, if you run a kernel which doesn't support namespacing, you won't get namespacing :) However:

* We can make the "docker in a vm" option much better. In another thread I mentioned boot2docker (http://github.com/steeve/boot2docker) which is the smallest possible VM to run docker - currently 25MB. Of course there is a performance tradeoff but for, say, a developer's mac or a server which is underutilized anyway, it may be exactly what you need.

* We are adding swappable execution drivers to docker, so that you can replace lxc (the current default) with openvz, or gracefully degrade to a simple chroot. Also a tradeoff - obviously you get very little guarantees of isolation, but for production servers which only run trusted payloads and don't need to extra resource isolation within each server, it is a great option.

In both cases, you still get the benefit of a unique (as in byte-for-byte unique) payload which can be transferred across machines and executed with strong guarantees of repeatability and consistency. That is often a big improvement from shipping provisioning scripts which will dynamically pull dependencies each time, giving you "good enough" repeatability at best. Maybe you forgot to pin a dependency and the mirror was updated between 2 deploys? Maybe the python build on Fedora (staging machine) is not the same as on RHEL (production). Docker solves that. Yes, there are caveats and it's not a silver bullet. And yes, god knows it is over-hyped. But the claims are not bullshit :)

> You're right, today there are many systems which cannot run docker.

It would be better if you put a footer or a developer's view of the technology, otherwise overarching claims about its features could back-fire. MongoDB over-marketed themselves vis-a-vis the features it actually provided.

I think there is no shame in saying we use LXC container technology. We can run on these systems patched, on these with a custom kernel, on these inside a VM. "Virtually anywhere is a bit of a stretch" I think.

* We are adding swappable execution drivers to docker, so that you can replace lxc (the current default) with openvz, or gracefully degrade to a simple chroot.

That is a cool feature, I like it. Libvirt does this for regular VMs.

> In both cases, you still get the benefit of a unique (as in byte-for-byte unique) payload which can be transferred across machines and executed with strong guarantees of repeatability and consistency.

Good stuff. That sounds like a good plan.

What about a Docker cluster (or maybe you already have that and I am showing my ignorance here). Run docker on a variant of host platforms and docker containers could contain tags that identify what kernel/distro/arch is needed and it would be automatically started/migrated to that host. But it would done rather transparently.

Low resource VMs will be something hosting providers will jump on as that is direct money in the pocket (can put more lightweight VMs on a physical host). But instead of having a generic set of uniform host, they'd just provide the right mix of dist/kernel/arch flavors, underneath. "Oh look people love their Ubuntu 13.10 servers, quickly add more Ubuntu hosts" stuff like that.

> Docker solves that.

Docker is good stuff. I like. My criticism was not of Docker's product but its marketing.

This "marketing vs. engineering" debate is much wider than just Docker, I think, and it's a big concern for me. My big worry is that Docker becomes a target of unfair criticism just because it's successful and hyped beyond our control.

I didn't write this particular article, but I do write our release announcements. Here's the last I wrote: http://blog.docker.io/2013/11/docker-0-7-docker-now-runs-on-... . If you see any bullshit in there, please let me know.

My problem is this: On the one hand, Docker's success so far is engineer-driven. If you visit our IRC channel or follow our github repo, you'll see what I mean. This week we merged 71 pull requests from 44 people. Real engineering teams are using Docker and giving presentations about why it makes their jobs easier. Our marketing consists of helping them meetup in the smae place and show each other their latest hacks.

On the other hand, there is a lot of demand to democratize the story behind docker and containers in general - to explain in plain language why they are important and can change the way software is built and distributed. Typically these demands come from engineers who need to convince their management to allow for broader use of Docker. As a result we produced the now famous shipping container analogy, produced a high-level slide deck, emphasized the high-level advantages of Docker, etc. This has helped a lot of non-experts to realize the importance of what we're doing and how it will affect them.

As a result you get short, appealing sentences like "with Docker you can build an application once, and run it anywhere". Of course, there are caveats - there always are - this or that machine cannot run it, certain versions of Linux don't support it etc. etc. Likewise you could argue that shipping containers can't actually be used for all freight - you can't ship live animals in them! Oil tankers don't use shipping containers! Those arguments are correct, but don't change the fact that shipping containers have changed the World economy in a big way - and so will containers.

My dilemma is that, as an engineer myself, I sympathize with you, and frankly I want you, my peer, to like my work. But every project needs marketing - even Linux, which certainly wouldn't be what it is today if many people didn't spend time and money explaining how it can change the World.

Long story short: I think we engineers live in the perpetual hypocrisy of denouncing marketing without realizing just how much we depend on it.

> What about a Docker cluster

Working on it :)

Certain elements are already in place - for example docker already has a remote api, and we introduced a "links" feature to connect containers to each other. We are still fleshing this out, but in the coming weeks you should see a robust end-to-end solution for standing up an entire cluster of docker hosts and interact with them as a cohesive unit.

That sounds awfully like a grid scheduler (think Sun Grid Engine, and its derivatives), which I greatly welcome: the market (for grid schedulers) is a bit barren at the moment after Sun's buyout.

What's the status on accessing other system resources from containers (e.g. GPUs)? There was some work being done to enable this, but I'm not sure if the eventual solution was "run the container with less security so that it can access the cards", or if there was a more graceful approach

I think native integration with Apache Mesos (more than just the mesosphere docker executor) would solve that problem quite nicely. Docker seems like a perfect fit for Mesos. Perhaps you and the rest of the docker team could look into that?

Reply on news.ycombinator.com