Zfs support for docker (beta)

Within Linux-land we already deal with a multitude of storage and sandboxing backends, networking topologies, kernel versions and builds, packaging and dylib versions across distros, underlying supervisors etc. We've made good progress in wrangling that "matrix from hell" so that the upper layers of the stack don't have to. But things are about to accelerate exponentially on this front: in just the last 6 months we've seen 1) Microsoft engineers get hugely involved in adding Windows support, 2) Joyent "betting the farm" on making Docker a first-class citizen on SmartOS, 3) A big influx of new Linux distros: CoreOS, Boot2docker, Atomic, Ubuntu Core - not to mention the whole schism of traditional distros over "the systemd wars". 4) more and more unofficial builds of Docker for Arm and Power (not to mention x86_32), being deployed in production today!

Meanwhile we are getting the first mergeable patches to move Docker towards fully content-addressable storage and distribution - which once complete will give us the building blocks for real repeatable builds, a-la Nix or Gitian. Yes, fetching external content over the network and executing arbitrary code will always introduce side effects, but with better tooling there are lots of great ways we could manage those side effects. This is one of my favorite areas of the project, and there are lots of awesome ideas and prototypes floating around. Ping me on irc if you're into that sort of thing!

Obviously there is no silver bullet to make computation more repeatable and portable across a variety of machines and operating systems.. But the community is scratching its own itch, which means it will happen no matter what. There will be plenty of trial and error, but I think we collectively have an opportunity to improve the state of the art. That's the beauty of open-source :)

And, to state the obvious: the solution is certainly not to pretend that every binary can execute anywhere, unmodified, with the same behavior. Machines and operating systems are heterogeneous, that is a fact. Trying to hide that heterogeneity will not make it go away. Rather, we should embrace it, and define a portable set of commands and properties which have a clear and predictable definition everywhere. Sure, the property "I require a Linux kernel later than 3.8 on x86_64 to start" will not be handled in the same way on every installation of Docker - some may present an error because they are running on another arch. But that property still has a precise and portable definition, and all Docker installations will understand the same thing. Now they can choose to process that property in the most appropriate way: perhaps they will present an informative error message to the user. Perhaps they will look for an "other architectures" field in the image manifest and point the user there. Perhaps they will redirect the request to another host which matches the requirement (hint: that is what Docker Swarm does when it receives a 'docker run'). With this kind of design we can greatly improve our daily flow as developers and sysops, and we can do it on top of the systems which are installed in the real world, today.