Hacker News Reader

Comment by earthscienceman | Hacker News Reader

A question that nags me every time Guix comes up, or nix, is about the benefits relative to a normal distro on something like ZFS. Are reproducible builds ever going to be all that important to a user? Rollbacks seem like the key feature here and that seems much better left up to the filesystem, not the package tools. This way your storage is also aware of what you're doing.

View on HN

Reproducible builds are an important part of efforts to secure the software supply chain. Ideally you want multiple independent parties vouching that a given package (whether a compiled binary, or a source tarball) corresponds to a globally immutably published revision in a source code repository.

That gives you Binary Transparency, which is already being attempted in the Arch Linux package ecosystem[0], and it protects the user from compromised build environments and software updates that are targeted at a specific user or that occur without upstream's knowledge.

Once updates can be tied securely to version control tags, it is possible to add something like Crev[1] to allow distributed auditing of source code changes. That still leaves open the questions of who to trust for audits, and how to fund that auditing work, but it greatly mitigates other classes of attack.

[0] https://github.com/kpcyrd/pacman-bintrans

[1] https://github.com/crev-dev/cargo-crev

The main selling point for me, on NixOS is the ability to switch machines easily, and know that I'm basically using an identical build. Throw your config files onto GitHub or a flash drive, then do whatever you want on your desktop. Want to switch to your laptop? Pull any changes down, do a quick rebuild, and you have an identical system. If you buy a new machine, just install nix, pull down your config files and you're good to go. It used to take me hours to set up a new device, often relying on carefully crafted bash scripts, to try and replicate builds, that needed to be constantly maintained. Not any more.

BTW, I rarely rollback, and if I do it's because I've monumentality fucked something up, which is a great feature.

Rollbacks are a minor feature, I never found any use for in either Guix or Nix.

The main advantage of both of them is that installation and 'making available' are decoupled. Meaning you can install five different versions of the same app and it'll be totally fine, as they don't sit around in '/usr/bin/foo' and clashing with each other. They sit conflict-free in '/nix/store/${HASH}/bin/'. For making them available for use you have to add them to your profile/environment (i.e. $PATH and symlinks that point '/nix/') or just run them from their '/nix' directory if you prefer. This makes development and testing new versions much easier as nothing ever really breaks to begin with. You can just spawn a new environment, fill it with whatever you need and use it. And it all happens at the packaging level, so it's pretty quick and you never end up with snapshots that capture far more of your filesystem than you planed.

rollbacks are pretty useful when your are experimenting with boot process

everything else Is unlikely to break your system enough to prevent you from switching profiles

A couple of things - if the system can handle rollbacks it will be much more reliable than using the fs, as the fs knows nothing about actual state. It knows about blocks commited to disk. Usually they look the same, but not always.

Then there’s the question of how exactly you reached this state. Having nixos generations is like having an event stream of all changes. Apply your backup to a new machine, what happens? Who knows.

In nixos/guix it’s not about only ”package tools”, it’s about treating the complete system and its state as a coherent whole.

Once I got the taste of it I see no way back to opaque packages, managed by config tools with no concept of state.

And if you’re a dev - shell.nix and declarative containers ftw.

How does Nix work with less sophisticated package managers that run on top of it, e.g. Python's "pip"?

Guix, and I think Nix also, package the things from these other package managers so you can manage them all with the same package manager. This also means features like rollbacks can apply to your emacs packages.

You can run pip on Guix System, but I don't think you'd have to, ideally. Same for rust's cargo and so on.

Ok, makes sense. Does that mean that you don't get the latest updates that are available in pip?

Also, what happens if you incidentally run pip on a Nix system? Will it mess up your installation?

I can't speak for pip, but cargo, rust's package manager, packages disappear on reboot. Doesn't matter for development since libraries are stored with your project, but any tools you install via cargo are gone

You can update packages to newer commits ahead of guix itself updating the package with a simple command[0] as long as dependencies and such haven't changed. So, if guix were behind, you should be able to easily remedy it.

[0] guix install mpv --with-commit=mpv=cc4ada655aae06218b900bb434e3521566394cde

If you just use pip & co, nix will be unaware of it and won't care one way or another. It will basically treat your pip-installed stuff like it'd treat your sourcefiles or pdfs.

Alternatively you can create derivations instead in which case the resulting artefacts will be fully understood and manageable by nix (I think there are integrations to do that e.g. tools like carnix which can automatically create derivations from existing language-specific packages, not sure if there's one for pip/pypi).

> If you just use pip & co, nix will be unaware of it and won't care one way or another. It will basically treat your pip-installed stuff like it'd treat your sourcefiles or pdfs.

Also if you run pip as root (to install for all users at once)?

It fails, because NixOS mounts /nix/store (the Nix store, where all managed packages are stored) as read-only (with some trickery used by Nix itself to bypass this for its own builds).

And if you bypass that, `nix-store --verify --check-contents` can detect the issue.

This is very reassuring, thanks.

> if the system can handle rollbacks it will be much more reliable than using the fs, as the fs knows nothing about actual state. It knows about blocks commited to disk. Usually they look the same, but not always.

Could you elaborate on this? If I have state that works on v1 and after upgrade to v2 the state is now non backward compatible to v1, how nix (or guix) can help? As far as I can tell, in such case a fs rollback is better solution.

What state do you mean? Let’s say you have a backup of your system partition but your home folder is separate. In that case a rollback with the filesystem is the same as one through nix — the latter will place symlinks to all the previous versions and the finished result is identical.

Of course if you have in the meanwhile used the new system and your home folder contains some backwards incompatible changes, both solutions will fail. Rolling back your home folder may not be a good thing as you may have backwards compatible changes you prefer.

Also, nix can also manage installed application’s configs, so those could also be rolled back, on either a per-app basis or however you prefer.

Sure you could, like Solaris did IIRC, zfs snapshot before updates was applied.

What I mean is that a fs snapshot is ”dumb” in and of it self.

If you couple zfs with the nixos rebuild command, then… sure, I guess - but the previous generations are already more or less directly available, unless GC’d.

This is what the fine manual has to say:

Since Nix is good at being Nix, most users will want their server's data backed up, and don't mind reinstalling NixOS and then restoring data.

Depends on how the state is stored. If it's in configuration, Nix generated it and it lives immutable in the Nix store, so Nix will just point out it to the old version on rollback.

If it's something like the content of a SQL database, which lives outside the Nix store and which Nix did not generate, you need some other tool (like a filesystem snapshot, maybe) to perform the rollback. I think CoW filesystems sometimes have performance issues with DBs, though, so I'm not sure that's always the approach you'd take.

The Nix ecosystem does have a fairly mature tool for managing stateful components that live outside the Nix store, though: https://github.com/svanderburg/dysnomia

It's been around for a long time. Idk who all is using it

Rollbacks are a convenient benefit that you get from using Nix.

Nix (and guix) allow for a declarative description of a package, (or, e.g. a collection of packages). In addition to rollbacks, I think some of the other benefits are neat.

e.g. Usually for running some project I see on GitHub, I have to copy-paste the "apt install <whatever>" command; or maybe even run a "curl https://example.com/install.sh | sh". Some projects provide a Docker image, allowing the program to be run without changing what's installed in the system. -- Nix allows for the advantages of each of these (e.g. being able to run the program without installing it into your system, or allowing for a simple command to install it).

e.g. VSCode has a Remote Containers plugin which allows for quickly getting started with a project by using a Docker container as an execution environment. Or things like GitHub Code Spaces or ReplIt aim to provide quick-start environments for developing code. -- I think nix-shell offers similar benefits. (Nix can even be used to describe a Docker image format, instead of using a Dockerfile).

e.g. something like "install this package, but with this different set of build flags enabled" is relatively straightforward in Nix.

> Rollbacks seem like the key feature here and that seems much better left up to the filesystem, not the package tools.

It's not, at least not necessarily.

Let's say you try to change your system configuration and you fuck it up, you revert, with zfs your attempt is gone, or you have to go and hunt it down in the snapshot if you remembered to store that.

With nix/guix it's still there to be updated.

An other component is the intentionality of decisions: in nix/guix you can reach a point where the setup of the entire system is fully described and reproducible, in fact there are people who rebuild the machine "from scratch" (really from the nix store) on every boot, to ensure the system does not accumulate transient cruft.

> Are reproducible builds ever going to be all that important to a user?

Being able to take your setup with you to a new computer is pretty cool, right?

Yes, but not sure that is enough value for non developers. For developers being able to easily use different versions of tools in different projects reliably is a game changer

Reply on news.ycombinator.com