Hacker News Reader

Comment by xwowsersx | Hacker News Reader

Seeing a tweet like the one from Mitchell makes me want to try Nix. Or at least I want to want to try Nix. Then I read the comments here and am reminded that no one can even succinctly explain what Nix is. I've read dozens of comments here and I still don't have a clear idea!

View on HN

I suspect you may take other technology you use as granted, and have forgotten the inherent subtlety that you've had to work through to attain the working understanding you have today. Consider the confusion you might have being an aspiring developer, on your first day cracking open a book on $INSERT_LANG_HERE, and then someone mentions Docker. Imagine your confusion:

"What is docker? It builds things? So, like, a "build system"? But it invokes something called make, or cmake, or... well, aren't those build systems? If that isn't confusing, I don't know what is. Oh, you say it's software that runs on Linux -- what's that? So docker is an OS? No? It runs on an OS? And you're telling me it doesn't just build stuff, it also can coordinate the execution of stuff? Oh, you're telling me that's docker-compose -- that isn't docker itself? I mean, it had 'docker' in the name. Oh. So separate project to 'docker the container image making thingy', but can be used with that 'docker'. Good God this is confusing."

Much like the Docker and its ecosystem as whole -- along with all the enabling/leveraged tech underlying it, like chroots, user/pid/network/etc namespaces, union filesystems, seccomp, etc -- there is some inherent complexity in Nix (and its broader ecosystem of tools).

Just like docker has docker, docker-compose, docker images, stateful docker containers (is it running? dead? what subtree of my filesystem is mounted where in the container? etc), a whole syntax for "docker files", etc, we have multiple things that come together to make "the whole of Nix" what it is.

As a whole, for Nix we have (non-exhaustive, but hopefully broad enough to help paint a picture):

- Nix the language (analogous to Dockerfile syntax), which is used by

- Nix the package builder/manager (analogous to the `docker` binary)

- Nixpkgs, which is a collection of packages (a bit of a stretch if you take it too literally, but analogous to a collection of Dockerfiles, each for a different software)

- NixOS, an operating system that leverages the packages declared in Nixpkgs.

That hopefully explains the "WHAT", but says nothing of "WHY".

So to touch on the "WHY" a bit:

- Dockerfiles are not reproducible. What builds today may fail tomorrow, or next month, or on odd numbered days, or whatever.

- Nix packages are reproducible. If it builds today, it will build on any machine, any time. Guaranteed.

- On distributions like Debian/Ubuntu/etc (essentially anything that isn't NixOS or inspired thereby, like Guix), when you install a package, and something goes wrong, your system can end up in an in-determinant state that requires human intervention to manually sort out how to unbreak things. Just google (or just recall the last time it's happened to you -- and if it hasn't happened to you, it eventually will) people desperately asking for help to unbreak their system after, say, Ubuntu's apt (or whatever) gets screwy and says it can't install/uninstall anything else because $INSERT_CRYPTIC_APT_BUZZWORDS_HERE. Oh, and add to that the compounding effects of the fact that any package can include raw bash commands as post-(un)install steps -- that's just asking for packages to fail to (un)install cleanly depending on what else the user does/doesn't have installed at that point in time.

- On NixOS (and similar) the system state is guaranteed to either update fully or not at all -- as a consequence of its design, none of the chaos mentioned can happen. It's not that it's unlikely. I literally mean it simply can't happen. System updates are guaranteed all or nothing. Also, if you find out that some configuration change isn't what you wanted, you can roll back in instant (that's not hyperbole -- rolling back entails no slow error prone file copying/writing/etc, literally just one symlink change, and "poof" you're on the previous version).

In a nutshell, from top to bottom of the stack, the selling point is: imagine a world where "oops -- that failed in production, but... well, it worked on my machine, I promise!" simply can't happen. Imagine the time savings, and the reduction in anxiety if you knew that, if you could successfully build and/or deploy a particular piece of software once locally/in-staging/etc, you have full confidence that you can repeat that on any machine at any time.

Admittedly, if you enjoy those struggles with conventional non-Nix(OS) setups, there isn't much value to Nix.

And as a pointed amendment here, here's a non-Nix experience I've had, which everyone eventually has:

My team discovers a major issue in a recent release. Conceptually, we need to rollback and we need to do so fast. However, we can't just deploy the same package/docker-image/whatever from the last release (perhaps because of a change in database schema, or whatever) -- we need the previous version plus some minor patches.

So I check out that previous version from source control.

I do `docker build`, and see a screen full of errors.

Huh. That worked when this ran in CI a week or two ago.

Oh, due to some petty political issues the devs of some third-party library (that is deep within our tree of transitive dependencies) have decided to nuke every posted version to pypi/rubygems/whatever and deleted the repo from GitHub.

Yaaaay. Okay. Fuck.

Would have been nice if we had a local cache/mirror of that, but there's nothing about docker that guarantees you'll have a copy like that (Nix does provide this).

Gotta push through the panic and weight options:

Do we try to replace every dependency that uses this lib?

Holy smokes that (transitively) touches a ton of our code base. Not impossible to do, but almost impossible to consider what the ETA will be for that, and remember that we're trying to put out an active fire here.

So... we could see if anyone online has a fork of the repo...

But then how do we know it hasn't been tampered with? It's not like pip/npm/rubygems has some content-based hash we can use to assert that this library's source tree is as expected, and it's not like Docker tracks/asserts such a content-based hash (Nix does track this).

Cue an hour or two of immense stress and customers being very unhappy. Oh, and there's nothing to say that this won't happen again -- after all, docker does nothing to help with and/or enforce determinism/reproducibility of builds. I mean, it does guarantee runtime reproducibility in that multiple machines can be running precisely the same image, but docker does not guarantee that you can reproduce the build of a given Dockerfile. These two different types of reproducibility should not be conflated.

Nix is specifically designed to prevent these types of scenarios.

Thanks, appreciate the thorough reply. This is very helpful. I need to think through this some more, but your example in the follow-up (where some dependency gets nuked) help concretize this. If I can get even greater reproducibility than Docker and it's faster, that's hugely appealing to me.

Your mind must be pure to understand Nix. Work on your purity. Just kidding. You might explain why you want to try nix? That would help customize an explanation for you, because there are levels. Umm... here is my attempt. Sorry.

Nix is a programming language plus utilities that are useful to define and work with software packages in a reproducible way (https://github.com/nixos/nix/).

Each package is called a "derivation", which is a function that takes inputs and makes output. The inputs are everything that is needed to make the output. It is "pure functional" package management - for the same input arguments, the same output will be produced. Nix is really fast because each derivation is hashed and cached and the language is lazy-evaluated.

Builds are "hermetic", meaning only the inputs specified in the derivation are available at build time. Contrast this to some packaging systems, where the build is done against some staging area where packages get installed as they are built and the output can depend on the non-deterministic order that packages are built.

Nixpkgs (https://github.com/nixos/nixpkgs/) is a large collection of recipes for existing software. It contains both rules to build software as well as "modules" to configure it or extend it. NixOS the linux distribution is also part of nixpkgs. There are lots of design patterns here and it can go pretty deep. There are also tons of hacks and patches and workarounds to make software conform to the way nix works. Nixpkgs also has a lot of useful library modules built in.

Nix is the latin word for snow. Nix "flakes" are a way to combine multiple inputs as well as pin the version of inputs. Kind of like pipenv/requirements.txt or "cargo lock" or "yarn lock" but for anything.

The output of derivations go in the "nix store" which is a path like /nix/store/<hash>/, so all sorts of software can co-exist (think multiple incompatible versions of the same library) and can be referenced in a fixed way. Usually you will end up with an output that is mostly symlinks to other /nix/store/ paths.

Nix can make practically any combination of software you can cobble together trivially rebuildable/reproducible. You can write some nix code that will produce a a VM image with test scripts as well as a script to launch the VM with a patched version of qemu and run those tests. You can have all your dotfiles/configuration in code with nix installed just for your user on top of Ubuntu. You can generate a raspberry pi sd card image from a short nix source file and a single command, and then 6 months later change a single line and regenerate it without worrying it might be broken.

You can achieve a lot of that stuff with Yocto or Ansible or a Dockerfile and scripts, but it would be slower than nix and more fragile.

I need to review this again, but really helpful. Thank you! Any resources you'd recommend for learning nix?

I've heard good things about https://zero-to-nix.com/

I don't have a good guide. I had to just start getting my hands dirty and piece it together.

My strategies:

- keep the documentation open

- install nixos in a vm

- search github for "configuration.nix" to find fully worked examples for nixos. I just cloned anything that looked useful[1] and would grep through it

- read the source code for nixpkgs

- search google and search https://discourse.nixos.org/ when you have an idea of what you want to do to find other people discussing it

It took me a couple days to have a very basic system running and a month to port most of my setup into nix (running some containers, overriding system packages, defining my own packages, running unmodified binaries using nix-ld).

I still use non-nix containers for a few services that I don't want to port over (but if I had known nix when I started I definitely would've used it).

Every step of the way was a huge struggle but the tools I have learned stay useful so I hope it is worthwhile investment. Still sometimes the extra nix friction is really frustrating but I can't help but think Nix or something like it is inevitably the future of software.

Reply on news.ycombinator.com