Optimising Docker Layers for Better Caching with Nix | Hacker News Reader

Optimising Docker Layers for Better Caching with Nix

grahamc.com

137 pointsgrhmc13 comments

Tried this out on the day it was released, and it's been awesome so far. Usually structuring images would take a lot of manual tweaking to get the layers just right, but with this I just adjust the number of layers and watch magic happen :) Average push size down from over 300MB to 15MB for an image of over 1.4GB, and takes like 15 lines of code to write.

Nix and Docker integration is almost optimal now, no other tooling provides that type of functionality

[Habitat](http://habitat.sh/) builds on the concepts in Nix, and here is a relatively old experiment I found on optimizing Docker layering with it: https://github.com/chetan/habitat-dockerize-layers

What I like about Nix is that this kind of functionality can be integrated into the Nixpkgs package set, as a function over packages. It's just as easy as writing a machine image or an virtual machine network test, because Nixpkgs has functions you can use for those as well.

I really like this. I think it is a great compromise between the limitations of docker and everything packaged in one file while taking advantage of nix. I'm curious about one other optimization that could be made.

You could instead look at the dependency graph of several nix docker builds and increase the cache reuse even more from that.

This approach is already optimal for up to around 40 or 120 packages, depending on whether you want to support old Docker versions.

"Where before you wouldn’t bother trying to have your application and database images share layers, with Nix the layer sharing is completely automatic."

Great post on the issues of Docker's linear layering method (which a tweet[1] by the author of the post also explains well independently of computers) and how to improve on it with Nix.

The "old" way of building Docker images with Nix also provided some of these benefits, but for quick builds often ended up being slower due to the dependency on qemu. This is an awesome improvement on that!

I'd recommend people building & deploying a lot of Docker images (especially in languages with many dependencies on native libraries and such, which can be tricky to get right but which Nix mostly just handles for you) to see if building images with Nix could improve your workflow!

[1]: https://twitter.com/grhmc/status/1046529919897886720

Neat. It seems like Nix could become the preferred way to build Docker images, if it's made a bit more user friendly. It's hard to beat the simplicity of

    FROM ubuntu:latest
    RUN apt-get update
    RUN apt-get install -y mysql php

I'm very optimistic about Nix. If you haven't read it yet, I suggest starting with this article:

https://www.infoq.com/articles/configuration-management-with...

I also gave a talk about why I'm excited about Nix:

https://vimeo.com/223525975

"Simplest" can be subjective. There's not much that can compete with editing your nixos config file and adding

    services.mattermost.enable = true;

and then running

    # nixos-rebuild switch

to have a mattermost instance running.

I'm specifically talk about building a Docker container. The example in the article has a couple dozen lines of nix config and several shell commands.

Wouldn't a multi-stage dockerfile fix the problems of having N layers or is it N for the entire file?

Docker layers for windows bases images are of huge sizes

Reply on news.ycombinator.com