Breaking up the Container Monolith
developers.redhat.com
developers.redhat.com
>This tool also allows us to build containers without all the container development tools inside it (yum, GCC, secrets, etc). Basically, Buildah produces clean containers, which means the container can be much, much smaller.
With docker you basically have one stage where you load all the dev tools to build the application or library. Then you have a second stage where you merely copy your application and run it. I often have a Dockerfile that builds a java library which then merely copies the .jar file into an empty docker image. The end result is usually a docker image smaller than 10MB. Then I add this image to the dockerfile that actually needs the library by adding it as an empty multistage and as part of the last stage I just copy the files of the library image into the application image. This way you can "inherit" multiple docker library images into one application image.
You forgot to install the JVM to run the .jar, didn't you?
Perhaps a hack for a free weekend.
They literally say that a Dockerfile isn't necessary in the same sentence as the broken link you mention:
> This brings us to Buildah, which is an open source tool for building containers without using Dockerfile at all.
What's the significance of the number of commits?
They say "It has been four years and we still have to use Dockerfile, which is a horrible version of bash." They say "we still have to use Dockerfile". They _never_ _had_ to use Dockerfile.
* Leading to: inheritance for construction. Lots of images include a bunch of irrelevant material because it was easier to FROM an existing image than to tease apart the kitchen-sink into the brittle hierarchy.
* Meaning: using composition isn't really "a thing" in the sense that it's commonly used. I'm given to understand that OCI layers can, if you recalculate the hashes, be layered in arbitrary order. In an ideal world the build tool can check that there won't be layer conflicts, meaning you can treat layers as immutable mixins, rather than immutable parents. Instead of FROM, you could have UNION or PLUS.
Then of course the less abstract stuff:
* Mystery meat. Dockerfiles make development solely responsible for managing all dependencies from the OS up. An awesome tool and a problematic responsibility. You've graduated from worrying about XSS holes in your web framework to fretting about kernel bugs and glibc glitches. Where I work we have an entire team devoted to tracking CVEs that affect code we ship. Do you?
* Mystery meat, at scale. Now you have 5,000 containers running ... what, exactly? What versions of what OSes? Are the runtimes up to date? You're going to need to either rigidly control Dockerfiles -- in which case, what did it buy in terms of dev flexibility? -- or you're going to need to buy tools to scan all your images. Docker sells such a service, as it happens.
Dockerfiles are a simple, clean exposure of the original concept of layered FSes, processes and ports. But they are not without limitations.
Disclosure: I work for Pivotal, we compete with Red Hat and Docker in the PaaS space.
Most of the ordering dependencies don't show up until runtime. The bits at rest can generally be played in any order.
The support is decent and has a couple of interesting properties.
The first is bringing all of the stateful code `apt update` etc. out of the image. It is not necessary for it to be there in the first place. Because of the packages being updated out of the container, and the way nix caching works we don't need to worry about security fixes being missed because of the Docker caching.
Another benefit is declaratively building the image, This gives us a guarantee that we will get the same image on two machines. They are currently working towards reproducible builds meaning we will get to a point where two builds will produce _exactly_ the same docker image.
I agree with all that you're saying, Dockerfiles seem to be artificially constrained and a poor interface for building images.
One I've found frequently frustrating is the inability to do any sort of globbing on COPY commands. This can make it very difficult to do some types of layer optimizations. For example in the Java world I have multi-project Gradle builds which consist of a tree of directories each of which contain a "build.gradle" file. I would like to copy all of these files, preserving the directory structure, into the container so that I can run a preliminary "gradle assemble" command which installs third-party dependencies, then only after that copy the remaining contents of these directories. That way I only need to re-download dependencies if any of the "build.gradle" files change when building the container.
This turns out to be impossible to do in the Dockerfile language without manually listing out every "build.gradle" file in the directory tree.
As for "Buildah", what about Deis and build packs deployed to Kubernetes? RH has gotten better in recent years with Openshift syncing closer to Kubernetes releases, but are still a far cry from how CoreOS handles things with Tectonic.
Don't have a clue what you're talking about with Fedora, it does exactly what you want.
Buildpacks have the advantage that the developer doesn't need to care about anything below their own direct dependencies. The runtime and OS are kept up to date by the platform, so you're relieved of the broadest angles of attack.
It's not a panacea. Just as with Dockerfiles, you have other upstream dependencies (Ruby gems, NPM modules, JARs etc) that need to be tracked and updated. Some of that can be delegated to the buildpack (ie, setting version range specifiers like '^1.1.2' or '1.1.*'), but that introduces some uncertainty about what is actually running in production.
Deis I believe did what we originally did in Cloud Foundry, which is to use Heroku's buildpack code. We ran soft forks for several years, adding only enough to allow execution in disconnected environments.
The Buildpacks team have since adopted a hard fork policy and have been replacing Heroku code with a uniform codebase called libbuildpack[0]. The new ones pass the same test suite we run and should work for Deis.