Hacker News Reader

Comment by rdtsc | Hacker News Reader

> automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere.

Bullshit. How do I run it on RHEL 5, because DoD systems are chock full of those? What about Windows? I have to install Virtualbox, and Vagrant and then run Docker inside the virtualized linux environment and then run docker containers in that.

So "virtually anywhere" means on a handful of systems.

Sometimes technology's biggest "fans" are its biggest enemies without even realizing.

View on HN

That is unnecessarily harsh. Docker legitimately represents a new idea, and some claim, a better idea. Yes, it won't run on RHEL5; but Java probably did not run on DEC mainframes either (back when its selling point was Write once run anywhere).

The question seems to be this: does this container model represent a step forward? I think it does. (And is it hyped a bit too much? Probably :-)

> That is unnecessarily harsh.

So do you agree that it runs "virtually anywhere" is bullshit or not. I say it is. Is Windows antiquated too? What about Mac OS X?

It is harsh vis-a-vis the claim it makes. If it makes revolutionary claims, it better hold up to harsh criticism.

> Docker legitimately represents a new idea, and some claim,

I agree with that. So "Investigating container models for application deployment" title?

I don't know yet what will happen for Windows, but an upcoming version of Docker will run on OS X. (I say "upcoming" because I know some core maintainers have OS X support in their local branches, so it's just a matter of time before it hits master.)

It doesn't has the same isolation guarantees (yet) but it lets you use the same workflow.

Windows is different, but I can't imagine a scenario where you have to run your app on Windows. I.e. if it's your local machine, you can spin up a VM; if it's in your datacenter, you can use Linux. If your local policies are "we only run Windows on our servers, screw Linux!" chances are that you wouldn't run Docker anyway, even if it ran natively on Windows :-)

Linux/x86-64 Docker images (which is all of them) aren't going to run on OS X unless inside a VM.

I'd say RHEL 5 is fairly outdated and that Windows is irrelevant for a non negligible portion of development shops as a development platform and as a deployment platform.

I don't agree with that it would run "virtually anywhere" but it should at least run on every platform that has binary compatibility with the platform that a container was built for? Getting docker to work on Windows might take a while but Mac could work.

I actually do think Docker or at least the container model will change a lot, i'm considering staying with Linux as my development platform instead of Mac and that's a thought i'd have laughed at two years ago and the sole reason is Docker.

"new idea" is a huge stretch. More accurate: "new idea to the mass hacker nerd HN market."

Solaris has had—for years—container based virtualization with local virtual network and de-duplicating file system snapshots you can tar up and send across machines.

It's new in that they made a nice abstraction on top of the Linux clone of Solaris containers then marketed the crap out of it so everybody became aware of non-VM compartmentalization and fine grained resource allocation.

In short: still awesome, but not "they invented it" new.

That's what I thought too. It's just FreeBSD jails brought to Linux, finally.

But I was wrong. Start playing with it for a while, and you will realize that Docker is not just about jails on Linux. Docker is about making portable, self-contained apps easy. It's about making sharing those containers easy. It's about making using those containers easy.

The core technology consists of a jail, yes. But once you've worked with the Docker registry and the Dockerfile (and its wonderful caching mechanism), and experienced just how easy it is to get something up an running, you will see that Docker > jails. Docker is much more than that.

There is another sense in which the idea isn't new - the notion of an image in smalltalk environments provides a similar level of encapsulation as docker containers, except perhaps that running smalltalk images don't get assigned their own ip address. A metacello script is analogous to dockerfile.

Solaris containers and FreeBSD jails are like the LXC technology. Docker, the suite of tools and utilities built on top of LXC, is what the new idea is.

Specifically, docker lets you build, define, and script new instances, and then to install and launch those instances with a single command, snapshot them, re-publish them, and so on. That's the real secret sauce behind docker.

Even for those of us who knew all about the capabilities in Linux and who used lxc already, it still packages things up nicely and makes lxc easy to use, in particular, easy for a team to use because not everybody needs to become an expert on the system config and lxc config. The expert writes a dockerfile, and everyone else can use it and manage their servers as a collection of capabilities, each one clearly defined in a dockerfile.

You're right, today there are many systems which cannot run docker. There are several things we are doing to remediate that. Obviously there are no miracles, if you run a kernel which doesn't support namespacing, you won't get namespacing :) However:

* We can make the "docker in a vm" option much better. In another thread I mentioned boot2docker (http://github.com/steeve/boot2docker) which is the smallest possible VM to run docker - currently 25MB. Of course there is a performance tradeoff but for, say, a developer's mac or a server which is underutilized anyway, it may be exactly what you need.

* We are adding swappable execution drivers to docker, so that you can replace lxc (the current default) with openvz, or gracefully degrade to a simple chroot. Also a tradeoff - obviously you get very little guarantees of isolation, but for production servers which only run trusted payloads and don't need to extra resource isolation within each server, it is a great option.

In both cases, you still get the benefit of a unique (as in byte-for-byte unique) payload which can be transferred across machines and executed with strong guarantees of repeatability and consistency. That is often a big improvement from shipping provisioning scripts which will dynamically pull dependencies each time, giving you "good enough" repeatability at best. Maybe you forgot to pin a dependency and the mirror was updated between 2 deploys? Maybe the python build on Fedora (staging machine) is not the same as on RHEL (production). Docker solves that. Yes, there are caveats and it's not a silver bullet. And yes, god knows it is over-hyped. But the claims are not bullshit :)

> You're right, today there are many systems which cannot run docker.

It would be better if you put a footer or a developer's view of the technology, otherwise overarching claims about its features could back-fire. MongoDB over-marketed themselves vis-a-vis the features it actually provided.

I think there is no shame in saying we use LXC container technology. We can run on these systems patched, on these with a custom kernel, on these inside a VM. "Virtually anywhere is a bit of a stretch" I think.

* We are adding swappable execution drivers to docker, so that you can replace lxc (the current default) with openvz, or gracefully degrade to a simple chroot.

That is a cool feature, I like it. Libvirt does this for regular VMs.

> In both cases, you still get the benefit of a unique (as in byte-for-byte unique) payload which can be transferred across machines and executed with strong guarantees of repeatability and consistency.

Good stuff. That sounds like a good plan.

What about a Docker cluster (or maybe you already have that and I am showing my ignorance here). Run docker on a variant of host platforms and docker containers could contain tags that identify what kernel/distro/arch is needed and it would be automatically started/migrated to that host. But it would done rather transparently.

Low resource VMs will be something hosting providers will jump on as that is direct money in the pocket (can put more lightweight VMs on a physical host). But instead of having a generic set of uniform host, they'd just provide the right mix of dist/kernel/arch flavors, underneath. "Oh look people love their Ubuntu 13.10 servers, quickly add more Ubuntu hosts" stuff like that.

> Docker solves that.

Docker is good stuff. I like. My criticism was not of Docker's product but its marketing.

This "marketing vs. engineering" debate is much wider than just Docker, I think, and it's a big concern for me. My big worry is that Docker becomes a target of unfair criticism just because it's successful and hyped beyond our control.

I didn't write this particular article, but I do write our release announcements. Here's the last I wrote: http://blog.docker.io/2013/11/docker-0-7-docker-now-runs-on-... . If you see any bullshit in there, please let me know.

My problem is this: On the one hand, Docker's success so far is engineer-driven. If you visit our IRC channel or follow our github repo, you'll see what I mean. This week we merged 71 pull requests from 44 people. Real engineering teams are using Docker and giving presentations about why it makes their jobs easier. Our marketing consists of helping them meetup in the smae place and show each other their latest hacks.

On the other hand, there is a lot of demand to democratize the story behind docker and containers in general - to explain in plain language why they are important and can change the way software is built and distributed. Typically these demands come from engineers who need to convince their management to allow for broader use of Docker. As a result we produced the now famous shipping container analogy, produced a high-level slide deck, emphasized the high-level advantages of Docker, etc. This has helped a lot of non-experts to realize the importance of what we're doing and how it will affect them.

As a result you get short, appealing sentences like "with Docker you can build an application once, and run it anywhere". Of course, there are caveats - there always are - this or that machine cannot run it, certain versions of Linux don't support it etc. etc. Likewise you could argue that shipping containers can't actually be used for all freight - you can't ship live animals in them! Oil tankers don't use shipping containers! Those arguments are correct, but don't change the fact that shipping containers have changed the World economy in a big way - and so will containers.

My dilemma is that, as an engineer myself, I sympathize with you, and frankly I want you, my peer, to like my work. But every project needs marketing - even Linux, which certainly wouldn't be what it is today if many people didn't spend time and money explaining how it can change the World.

Long story short: I think we engineers live in the perpetual hypocrisy of denouncing marketing without realizing just how much we depend on it.

> What about a Docker cluster

Working on it :)

Certain elements are already in place - for example docker already has a remote api, and we introduced a "links" feature to connect containers to each other. We are still fleshing this out, but in the coming weeks you should see a robust end-to-end solution for standing up an entire cluster of docker hosts and interact with them as a cohesive unit.

That sounds awfully like a grid scheduler (think Sun Grid Engine, and its derivatives), which I greatly welcome: the market (for grid schedulers) is a bit barren at the moment after Sun's buyout.

What's the status on accessing other system resources from containers (e.g. GPUs)? There was some work being done to enable this, but I'm not sure if the eventual solution was "run the container with less security so that it can access the cards", or if there was a more graceful approach

I think native integration with Apache Mesos (more than just the mesosphere docker executor) would solve that problem quite nicely. Docker seems like a perfect fit for Mesos. Perhaps you and the rest of the docker team could look into that?

And they're not portable, either! What kernel is Ruby 2.0.0 build against on in his example? Maybe Docker itself may be everywhere, but you still have have to be aware of any OS dependencies that apps in your Docker container might have...

I think lxc+cgroups and Docker are a much less wasteful way to provision lots of similar systems with particular types of workloads, but this article did seem a little hyperbolic.

In practice though a very large majority of applications are already expected to support a wide range of kernel versions and flavors, and require zero modifications to do so because of Linux's outstanding track record of preserving reverse compatibility.

So deploying an application as a container is a big improvement over shipping it as source or package, because instead of having to support any combination of library versions, distros and kernels, it only needs to worry about the kernel which is the easiest part.

VMWare has had a similar concept called ThinApp for Windows http://www.vmware.com/products/thinapp/ -- albeit it is more geared towards desktop apps as opposed to server apps, but similar concept.

If I recall it was an acquisition for them 4-5 years ago.

Sure, but if you're looking at systems running the latest RHEL, there's a lot of stuff you don't get. RHEL5, when it shipped at least, didn't have memcached.

RHEL has always appealed to IT departments and server staff who value stability over functionality, much like corporate IT at the hospital where I worked finally approved Windows XP SP1 for test deploys, well after SP3 had been released. They don't want to risk the headache, so they stick with what works rather than providing functionality (or compatibility) that others could need.

There's definitely value in that, don't get me wrong, but my point is that the types of environments where RHEL5 are the norm wouldn't be rolling out Docker instances even if they could, and I don't feel as if they should be counted against the impact of progress.

But a lot of servers and platforms out there don't run on this years' Linux kernel. That is a far cry from "virtually anywhere" which was my original issue with this blogpost.

I am not arguing that I need Docker to run on RHEL personally or I need it to run in Windows. The point was about making unsubstantiated claims about a product and in the meantime while seemingly trying to help "market" the product now putting the creators on the defense having to backpedal. "Well no, contrary to what some of your may have read from our zealous followers, we don't don't run virtually anywhere. Here is a list of Linux distros we run out of the box on. Here is a list where you'll need to installed a custom kernel."

>> Bullshit.

It's actually promoted like that on their website:

http://www.docker.io/learn_more/

Well, so it is.

It doesn't make it less bullshitty. It changes that fact that it wasn't only a zealous fan driving up some excitement, they are doing it themselves. Well,I don't like it. I don't like when tech companies selling to developers mislead them or make over-inflated statements. Let 2am infomercials do that. MongoDB was (is?) in the same category, and I don't trust them and don't like their product.

Can you imagine Linus writing the new kernel version announcement like that -- "This kernel release will run anywhere. It will radically change how you look at the world. It might bring about peace too". What over-inflated claims do is break trust. If they are marketing for non-technical people, yes general statements work. Marketing to other technical people using claims like that is patronizing and disappointing.

I really wish Apple, for instance, would enable the same process/network namespacing that enables LXC to work. I'm not sure what the state of Windows is with respect to virtual network namespaces and process namespaces and control groups, but if the two other major OSes enable these basic features, it is easy to imagine universal containers that run everywhere.

Apple could choose to support LXC. FreeBSD systems have jails, so another way forward was if Apple support that, Docker could wrap a generic API on top of that. Much like libvirt is doing with various VM hosting solutions.

Or Docker could gracefully degrade to using chroot on OS X (providing a different type of isolation.) This is a plan.

One of the things Docker needs to get better at expressing is that A Docker container != Linux Containers. It may currently Build/Provision/Execute inside of a Linux Container (provided by LXC) but that is an (albeit important) implementation detail.

Given the rest of the article, I think anywhere means any cloud host other than joynet.

Reply on news.ycombinator.com