Hacker News Reader

Comment by exdsq | Hacker News Reader

One really cool thing about NixOS is you declare the OS just like you might a DockerFile, and you can choose which configuration file you use at booting time. This lets you install applications just during up-time (to try out a new tool), add it to the configuration file if you like it, and roll back to the version before if it breaks something. Versions are declarative too (PSql == X.X.X) and upgrades/downgrades are set there too. If I want to share an environment with someone I can just share the configuration file minus the secrets.

View on HN

> One really cool thing about NixOS is you declare the OS just like you might a DockerFile,

But DockerFiles have a really terse, self-evident, trivial to read language. It is obvious what a given DockerFile means even if it's the first one you see.

For nix; well, I have seen a few nix files and each one leaves me more confused than the last. Can anybody point me to good examples of beautiful, single-file, complete, self-contained terse nix examples that describe a few simple systems? Everything that I seem to find are small fragments of stuff.

This is because Nix needs to express a lot more so builds are reproducible and incremental. Docker doesn’t care about these things.

Even if Docker doesn't really care about this, the DockerFiles themselves can also describe a reproducible process, and they are certainly "incremental" due to caching (not sure of what you mean by "incremental"). Does nix power merit such a big compromise against simplicity and readability?

Because Dockerfiles delegate all the complexity to a package manager (apt, apk, ...), which has its own languages (eg. debian/rules for apt).

Using Docker without any package manager doing this job would be a completely different story.

And btw, you can also use Nix to build docker images. For example, here is what would be the equivalent of a Dockerfile but with Nix: https://nixos.org/nixpkgs/manual/#ex-dockerTools-buildImage

> the DockerFiles themselves can also describe a reproducible process

It's trickier. You need to pin all versions of all packages you use, even indirect dependencies.

Dockerfiles which just pull packages from distribution repositories are not reproducible in the same way that Nix expressions are.

Rebuilding the Dockerfile will give you different results if the packages in the distribution repositories change.

A Nix expression specifies the entire tree of dependencies, and can be built from scratch anywhere at any time and get the same result.

> the DockerFiles themselves can also describe a reproducible process

This is true, but Docker does almost nothing to support reproducibility. As soon as you do an apt-get, reproducibility goes out the window.

> they are certainly "incremental" due to caching

Caching is layer-based. Docker has no awareness of whether or not a particular dependency has changed or what is necessary to rebuild it. Docker only understands layers, which are inherently linear--you can only try in vain to force your dependency graph into that linear layer structure. An incremental build tool only rebuilds things that have changed (or whose dependencies have changed).

> Does nix power merit such a big compromise against simplicity and readability?

This is a good question. First of all, understand that Nix isn't intended to be a build tool, it's a package toolkit. It's intended to replace all of the stuff that the debian and centos people use to build and manage packages in apt and yum repos. That said, I personally think Docker's build system is sooooo bad that under many circumstances, Nix does a better job.

For example, if your whole repo is Go or Rust or some other language with a sane build and deployment story, then they already have incremental, reproducible build tools and it's fine for Docker to call into them (or to invoke them outside of Docker and just have Docker copy in the static artifact). But if you have a heterogeneous tree including C or Python or other languages which lack sane build tooling, then you need something to stitch all of that together in some (somewhat) reproducible, incremental fashion if you hope to be able to build reliably and in some timely fashion.

That said, there are a few other tools that are purpose-built for this problem; however, they all tend to be poorly designed, buggy, and hard to extend. I'm thinking of Bazel, Pants, Buck, etc--all of which are clones of Google's internal build system, Blaze. So far, Nix seems to be the best in class, even though it doesn't aspire to be in the class at all.

Great description, thanks. Question: do you know how Nix deals with language package managers (like pip)? If I do sudo pip install (leaving aside the usual debate as to its merits), does that then wreck my Nix install, especially if it happens to contain another version of the same package? And does it instantly make things non-reproducible again (just like the apt-get issue you mentioned)? Or does Nix somehow get around it?

Nix doesn't pay any attention to the rest of your system. It keeps everything it needs (including its own Python interpreter) in its own "nix store" directory, at /nix/store. You can crudely think of it as having its own virtualenv for every distinct Python target you use.

It's totally possible to introduce mutable, non-reproducible state on a system that uses Nix, even if that system is NixOS. However, you can't do so in your Nix store, which is where all the Nix-installed things live (at least, not without a lot of hackery - Nix tries very hard to make the entire store immutable).

Basically this works out to: anything you install or configure with Nix, is immutable and reproducible. Anything you don't is at your own risk. Unfortunately there are still some "pockets of state" that haven't been solved yet, most notably applications storing state in ~.

For most every language dependency management system, there's a Nix equivalent of some sort that allows you to manage those dependencies without invoking the non-reproducible mutable tooling.

Bazel is buggy?! Bazel is not a clone, it's the refactoring of the internal build system, basically. I am 99% certain that Blaze currently has Bazel at the core.

And it's pretty damn robust.

Bazel is definitely buggy. One of the silliest ones is this one (and as far as this one goes, I do not understand how this issue exists when Blaze has been used in Google for such a long itme): https://github.com/bazelbuild/bazel/issues/9419

One of the more fundamental bugs is this one: https://github.com/bazelbuild/bazel/issues/4558

I've even had to clean --expunge and clear my disk cache to fix some build errors before. Bazel paints a rosy picture of the world, but the real world is messy, and Bazel builds aren't as reproducible as it makes you want to think. (Which BTW is one thing that's had me asking so many questions about Nix.) At least the fact that it does not track changes outside the workspace means you can update your system compilers (or Python, or whatever) and end up with build outputs that will not be reproduced from scratch.

It takes a lot of time to track the issues down, so I don't necessarily know what goes wrong every time; these are just two issues I remember off the top of my head. But if you haven't run into bugs with Bazel then you haven't used it seriously enough.

And then there's the Python support which is even more half-baked right now. Their own comments readily mention that the Python support is not idiomatic. (Not just in terms of syntax, but some of it goes against Bazel's underlying design.)

Of course I say this with the full realization that "fixing" these things can be a huge undertaking (up to and including hooking the compiler programs manually), but that doesn't mean they're not bugs.

For a long time, Python3 support was advertised and there were lots of Python3 flags, but it was patently broken and there were many tickets that acknowledged as much. Allegedly that has changed now, but my experience was so bad I ran from it.

It's probably better than when you used it now, but it might depend on what you're trying to do (e.g. Cython has more room for improvement).

As far as bazel bugs, it's safe to say that if you stick to the ways that Google uses blaze, then you are unlikely to run into bugs. E.g. Google doesn't run blaze on windows or use system toolchains, so it makes some sense that those are where the issues are. Unfortunately, Google also uses their own rules, so the rule quality for open source rules varies somewhat since they haven't all been battle tested in the same way. blaze and the Google codebase have also co-evolved for like 15 years to work nicely together, whereas anything adopting bazel today might be an awkward 3rd wheel for a while.

Disclosure: Ex-Googler

Yeah, I've figured as much in hindsight. It'd have been nice if they were more forthcoming about this than leaving it as a surprise for everyone to figure out (including, now, for the parent to whom I replied).

It's worth noting that the bugs aren't evenly distributed across language plugins. Bazel might be solid if you're writing Java or something.

Dockerfiles are as reproducible as a bash script (and that's not a hyperbole). Yes, they are incremental but as you pointed out it is per statement.

This requires:

1) that it is on you to specify the correct order 2) you are limited to 42 layers 3) it doesn't work well even if you know what to do and care about it:

- in most setups that I've seen once you trigger build system it still starts from scratch (for example each time, docker images pull all dependencies again and again every build). nix build won't build stuff that was already done, unless you explicitly instruct it with --check option

- typically people use `COPY . .`, because it is less work, but in reality certain files need to be pulled earlier than others, but then you're more likely to run into layer limit, also it's more work, more prone to errors (like typos, forgetting about a file etc). With declarative configuration, nix will only rebuild derivations where input changed. It doesn't matter what order you specify dependencies.

> Does nix power merit such a big compromise against simplicity and readability?

The simplicity causes complexity everywhere else. Is this really that complex? Here's for example everything that's needed to build pgbouncer[1] (you can ignore the meta section, that's informational only). I'm using that since I used it recently.

Nix actually isn't a complicated language, just different. I believe the big problem people are having is just different programming paradigm, but the programming paradigm used by nix is really what is needed to solve the problem correctly (purely functional -> for the same inputs (source code, dependencies, system etc) you always get the same output; lazily evaluated -> build only things you actually depend on). The properties of the language allow to define how to build specific program in a declarative way.

[1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sq...

This file explains how to configure the system when you first set it up [0]. I've just added the one I currently have set up on my dev box to Github - it's not complex by any means but has what I need :) [1]

[0] https://nixos.org/nixos/manual/index.html#sec-configuration-...

[1] https://github.com/oneEdoubleD/nixos/blob/master/config.nix

I don't know if it's beautiful, but here is one of mine.

It sets up me as a user: https://github.com/cswank/dotfiles/blob/master/nix/configura...

Sets up a hobby app I built via systemd: https://github.com/cswank/dotfiles/blob/master/nix/configura...

Defines a cron job that backs up my music files: https://github.com/cswank/dotfiles/blob/master/nix/configura...

Opens up ports in the firewall (they are all closed by default): https://github.com/cswank/dotfiles/blob/master/nix/configura...

The rest is pretty much boilerplate from the default configuration you get when starting from scratch.

> But DockerFiles have a really terse, self-evident, trivial to read language. It is obvious what a given DockerFile means even if it's the first one you see.

Is it, though? It's obvious what commands it runs, yes, but that doesn't translate to actually understanding what the end state is, and that's precisely the bit that matters.

Yet that's something you have to infer from a pile of operations that mutate global state, which is precisely the sort of thing that's very difficult to do reliably.

Sure, the result is that a Dockerfile is less code - because you're outsourcing a lot of the "understanding what this does" work to the reader's brain rather than to the code. That's not a good thing.

> Can anybody point me to good examples of beautiful, single-file, complete, self-contained terse nix examples that describe a few simple systems?

It's not completely self-contained (eg. the hardware configuration is in separate files for infrastructure migration reasons and some custom abstractions are used), but here's an example of the actual configuration of two of my servers, warts and all: https://git.cryto.net/joepie91/morph-rc/src/master/configura...

In practice, "self-contained" is something you're not likely to see in real-world usages of Nix.

Sure, people start out with a self-contained configuration, but they tend to discover pretty quickly that configuration is code, and that means that you can abstract out the repetitive and messy bits to separate modules, and now it is no longer self-contained.

Basically, asking for a self-contained Nix configuration is going to yield similar results to asking for a self-contained source file for a piece of software. You'll either get a) non-real-world code, b) a big mess of stuff dumped into a single file, or c) multiple files.

Reply on news.ycombinator.com