This is true, but Docker does almost nothing to support reproducibility. As soon as you do an apt-get, reproducibility goes out the window.
> they are certainly "incremental" due to caching
Caching is layer-based. Docker has no awareness of whether or not a particular dependency has changed or what is necessary to rebuild it. Docker only understands layers, which are inherently linear--you can only try in vain to force your dependency graph into that linear layer structure. An incremental build tool only rebuilds things that have changed (or whose dependencies have changed).
> Does nix power merit such a big compromise against simplicity and readability?
This is a good question. First of all, understand that Nix isn't intended to be a build tool, it's a package toolkit. It's intended to replace all of the stuff that the debian and centos people use to build and manage packages in apt and yum repos. That said, I personally think Docker's build system is sooooo bad that under many circumstances, Nix does a better job.
For example, if your whole repo is Go or Rust or some other language with a sane build and deployment story, then they already have incremental, reproducible build tools and it's fine for Docker to call into them (or to invoke them outside of Docker and just have Docker copy in the static artifact). But if you have a heterogeneous tree including C or Python or other languages which lack sane build tooling, then you need something to stitch all of that together in some (somewhat) reproducible, incremental fashion if you hope to be able to build reliably and in some timely fashion.
That said, there are a few other tools that are purpose-built for this problem; however, they all tend to be poorly designed, buggy, and hard to extend. I'm thinking of Bazel, Pants, Buck, etc--all of which are clones of Google's internal build system, Blaze. So far, Nix seems to be the best in class, even though it doesn't aspire to be in the class at all.
Basically this works out to: anything you install or configure with Nix, is immutable and reproducible. Anything you don't is at your own risk. Unfortunately there are still some "pockets of state" that haven't been solved yet, most notably applications storing state in ~.
For most every language dependency management system, there's a Nix equivalent of some sort that allows you to manage those dependencies without invoking the non-reproducible mutable tooling.
And it's pretty damn robust.
One of the more fundamental bugs is this one: https://github.com/bazelbuild/bazel/issues/4558
I've even had to clean --expunge and clear my disk cache to fix some build errors before. Bazel paints a rosy picture of the world, but the real world is messy, and Bazel builds aren't as reproducible as it makes you want to think. (Which BTW is one thing that's had me asking so many questions about Nix.) At least the fact that it does not track changes outside the workspace means you can update your system compilers (or Python, or whatever) and end up with build outputs that will not be reproduced from scratch.
It takes a lot of time to track the issues down, so I don't necessarily know what goes wrong every time; these are just two issues I remember off the top of my head. But if you haven't run into bugs with Bazel then you haven't used it seriously enough.
And then there's the Python support which is even more half-baked right now. Their own comments readily mention that the Python support is not idiomatic. (Not just in terms of syntax, but some of it goes against Bazel's underlying design.)
Of course I say this with the full realization that "fixing" these things can be a huge undertaking (up to and including hooking the compiler programs manually), but that doesn't mean they're not bugs.
Disclosure: Ex-Googler
Rebuilding the Dockerfile will give you different results if the packages in the distribution repositories change.
A Nix expression specifies the entire tree of dependencies, and can be built from scratch anywhere at any time and get the same result.
Using Docker without any package manager doing this job would be a completely different story.
And btw, you can also use Nix to build docker images. For example, here is what would be the equivalent of a Dockerfile but with Nix: https://nixos.org/nixpkgs/manual/#ex-dockerTools-buildImage
> the DockerFiles themselves can also describe a reproducible process
It's trickier. You need to pin all versions of all packages you use, even indirect dependencies.
This requires:
1) that it is on you to specify the correct order 2) you are limited to 42 layers 3) it doesn't work well even if you know what to do and care about it:
- in most setups that I've seen once you trigger build system it still starts from scratch (for example each time, docker images pull all dependencies again and again every build). nix build won't build stuff that was already done, unless you explicitly instruct it with --check option
- typically people use `COPY . .`, because it is less work, but in reality certain files need to be pulled earlier than others, but then you're more likely to run into layer limit, also it's more work, more prone to errors (like typos, forgetting about a file etc). With declarative configuration, nix will only rebuild derivations where input changed. It doesn't matter what order you specify dependencies.
> Does nix power merit such a big compromise against simplicity and readability?
The simplicity causes complexity everywhere else. Is this really that complex? Here's for example everything that's needed to build pgbouncer[1] (you can ignore the meta section, that's informational only). I'm using that since I used it recently.
Nix actually isn't a complicated language, just different. I believe the big problem people are having is just different programming paradigm, but the programming paradigm used by nix is really what is needed to solve the problem correctly (purely functional -> for the same inputs (source code, dependencies, system etc) you always get the same output; lazily evaluated -> build only things you actually depend on). The properties of the language allow to define how to build specific program in a declarative way.
[1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sq...