Hacker News Reader

Comment by binarymax | Hacker News Reader

I often see initiatives and articles like this but no mention of Nix. Is it just not well known enough for comparison? Because to me that’s the standard.

View on HN

I use Nix extensively, but the Nix daemon doesn't do much of use that can't be achieved by building your code from a fixed OCI container with internet turned off. The latter is certainly more standard across the industry, and sadly a lot easier too. Nix is not a revolutionary containerisation technology, nor honestly a very good one.

The value in Nix comes from the package set, nixpkgs. What is revolutionary is how nixpgks builds a Linux distribution declaratively, and reproducibly, from source through purely functional expressions. However, nixpkgs is almost an entire universe unto itself, and it is generally incompatible with the way any other distribution would handle things, so it would be no use to Fedora, Debian, and others

At work we went back to a Docker build to make reproducible images. The primary reason is poor cross-compilation support in Nix on Arm when developers needed to compile for an amd64 service and derive image checksums that are put into tooling that are run locally for service version verification and reproducibility.

With Docker it turned out relatively straightforward. With Nix even when it runs in Linux Arm VM we tried but just gave up.

Funny, I had that experience with Docker - mostly due to c++ dependencies - that were fine in Nix

In the near term it makes more sense to position nix as a common interface between app developers and distro maintainers and not as a direct-to-user way to cut their distro maintainers out of the loop entirely (although it is quite useful for that).

Ideally, a distro maintainer would come across a project packaged with nix and think:

> Oh good, the app dev has taken extra steps to make life easy for me.

As-is, I don't think that's the case. You can add a flake output to your project which builds an .rpm or a .deb file, but it's not commonly done.

I'm guessing that most of the time, distro maintainers would instead hook directly into a language specific build-tool like cmake or cargo and ignore the nix stuff. They benefit from nix only indirectly in cases where it has prevented the app dev from doing crazy things in their build (or at least has made that crazyness explicit, versus some kind of works-on-my-machine accident or some kind of nothing-to-see here skulduggery).

If we want to nixify the world I think we should focus less on talking people out of using package managers which they like and more on making the underlying packages more uniform.

Oh, I assure you, it's hard to escape knowing about Nix if you write about this sort of thing. Someone will be along almost immediately to inform you about it.

Nix wasn't mentioned (I'm the author) because it really isn't relevant here -- the comparable distributions, when discussing what Fedora is doing, are Debian and other distributions that use similar packaging schemes and such.

I agree that NixOS/nixpkgs would not be a good a basis for comparison. Do you have an opinion about the use of nix by app devs to specify their builds, i.e. as a make alternative, not as a Fedora alternative?

Quoting the article:

> Irreproducible bits in packages are quite often "caused by an error or sloppiness in the code". For example, dependence on hardware architecture in architecture-independent (noarch) packages is "almost always unwanted and/or a bug", and reproducibility tests can uncover those bugs.

This is the sort of thing that nix is good at guarding against, and it's convenient that it doesn't require users to engage with the underlying toolchain if they're unfamiliar with it.

For instance I can use the command below to build helix at a certain commit without even knowing that it's a rust package. Although it doesn't guarantee all aspects of repeatability, it will fail if the build depends on any bits for which a hash is not known ahead of time, which gets you half way there I think.

     nix build github:helix-editor/helix/340934db92aea902a61b9f79b9e6f4bd15111044

Used in this way, can nix help Fedora's reproducibility efforts? Or does it appear to Fedora as a superfluous layer to be stripped away so that they can plug into cargo more directly?

Used in this way, can nix help Fedora's reproducibility efforts? Or does it appear to Fedora as a superfluous layer to be stripped away so that they can plug into cargo more directly?

A lot of Nix-based package builds will burn Nix store paths directly into the binary. If you are lucky it's only the rpath and you can strip it, but in some cases other Nix store paths end up in the binary. Seems pretty useless to Fedora.

Besides many of the difficult issues are not solved by Nix either. (E.g. build non-determinism by ordering differences due to the use of a hashmap somewhere in the build.)

> A lot of Nix-based package builds will burn Nix store paths directly into the binary

I didn't know that, sounds like a bug. Maybe something can be done to make it easier to know that this is the case for your build.

I'd still think that by refusing to build things with unspecified inputs, nix prunes a whole category of problems away which then don't bite the distro maintainers, but maybe that's wishful thinking.

I'll continue to use it because it's nice to come to a project I haven't worked on in a few years and not have to think about whether it's going to now work on this machine or figure out what the underlying language-specific commands are--but if there were ways to tweak things so that others have this feeling also, I'd like to know them.

I didn't know that, sounds like a bug. Maybe something can be done to make it easier to know that this is the case for your build.

It's a feature. E.g. if a binary needs to load data files, it needs to know the full path, or you are back to an FHS filesystem layout (which has a lot of issues that Nix tries to solve).

I'd still think that by refusing to build things with unspecified inputs,

I haven't followed development of traditional Linux distributions, but I am pretty sure that they also build in minimal sandboxes that only contain specified dependencies. See e.g. Mock: https://github.com/rpm-software-management/mock

Oh I'm sure they do build in sandboxes... my point is that those builds are more likely to succeed on the first try if the app dev also built in a sandbox. If you let nix scold you for tempting fate and relying on the output of an unchecked curl, that's one fewer headache for whoever later tries to sandbox your stuff.

Contrary to popular opinion, Nix builds aren't reproducible: https://luj.fr/blog/is-nixos-truly-reproducible.html

That's such a weird characterization of this article, which (in contrast to other writing on this subject) clearly concludes (a) that Nix achieves a very high degree of reproducibility and is continuously improving in this respect, and (b) Nix is moreover reproducible in a way that most other distros (even distros that do well in some measures of bitwise reproducibility) are not (namely, time traveling— being able to reproduce builds in different environments, even months or years later, because the build environment itself is more reproducible).

The article you linked is very clear that both qualitatively and quantitatively, NixOS has made achieved high degrees of reproducibility, and even explicitly rejects the possibility of assessing absolute reproducibility.

NixOS may not be the absolute leader here (that's probably stagex, or GuixSD if you limit yourself to more practical distros with large package collections), but it is indeed very good.

Did you mean to link to a different article?

> NixOS may not be the absolute leader here (that's probably stagex, or GuixSD if you limit yourself to more practical distros with large package collections), but it is indeed very good.

Could you comment on how stagex is? It looks like it might indeed be best in class, but I've hardly heard it mentioned.

The Bootstrappable Builds folks created a way to go from only an MBR of (commented) machine code (plus a ton of source) all the way up to a Linux distro. The stagex folks built on top of that towards OCI containers.

https://stagex.tools/ https://bootstrappable.org/ https://lwn.net/Articles/983340/

What does high degree mean? Either you achieve bit for bit reproducibility of your builds or not.

If 99% of the packages in your repos are bit-for-bit reproducible, you can consider your distro to have a high degree of reproducibility.

And with even a little bit of imagination, it's easy to think of other possible measures of degrees of reproducibility, e.g.:

  • % of deployed systems which consist only of reproducibly built packages
  • % of commonly downloaded disk images (install media, live media, VM images, etc.) consist only of reproducibly built packages
  • total # of reproducibly built packages available
  • comparative measures of what NixOS is doing right like: of packages that are reproducibly built in some distros but not others, how many are built reproducibly in NixOS
  • binary bootstrap size (smaller is better, obviously)

It's really not difficult to think of meaningful ways that reproducibility of different distros might be compared, even quantitatively.

Sure, but in terms of absolute number of packages that are truly reproducible, they outnumber Debian because Debian only targets reproducibility for a smaller fraction of total packages & even there they're not 100%. I haven't been able to find reliable numbers for Fedora on how many packages they have & in particular how many this 99% is targeting.

By any conceivable metric Nix really is ahead of the pack.

Disclaimer: I have no affiliation with Nix, Fedora, Debian etc. I just recognize that Nix has done a lot of hard work in this space & Fedora + Debian jumping onto this is in no small part thanks to the path shown by Nix.

They are not.

Arch hovers around 87%-90% depending on regressions. https://reproducible.archlinux.org/

Debian reproduces 91%-95% of their packages (architecture dependent) https://reproduce.debian.net/

> Disclaimer: I have no affiliation with Nix, Fedora, Debian etc. I just recognize that Nix has done a lot of hard work in this space & Fedora + Debian jumping onto this is in no small part thanks to the path shown by Nix

This is completely the wrong way around.

Debian spearheaded the Reproducible Builds efforts in 2016 with contributions from SUSE, Fedora and Arch. NixOS got onto this as well but has seen less progress until the past 4-5 years.

The NixOS efforts owes the Debian project all their thanks.

From your own link:

> Arch Linux is 87.7% reproducible with 1794 bad 0 unknown and 12762 good packages.

That's < 15k packages. Nix by comparison has ~100k total packages they are trying to make reproducible and has about 85% of them reproducible. Same goes for Debian - ~37k packages tracked for reproducible builds. One way to lie with percentages is when the absolute numbers are so disparate.

> This is completely the wrong way around. Debian spearheaded the Reproducible Builds efforts in 2016 with contributions from SUSE, Fedora and Arch. NixOS got onto this as well but has seen less progress until the past 4-5 years. The NixOS efforts owes the Debian project all their thanks.

Debian organized the broader effort across Linux distros. However the Nix project was designed from the ground up around reproducibility. It also pioneered architectural approaches that other systems have tried to emulate since. I think you're grossly misunderstanding the role Nix played in this effort.

> That's < 15k packages. Nix by comparison has ~100k total packages they are trying to make reproducible and has about 85% of them reproducible. Same goes for Debian - ~37k packages tracked for reproducible builds. One way to lie with percentages is when the absolute numbers are so disparate.

That's not a lie. That is the package target. The `nixpkgs` repository in the same vein package a huge number of source archives and repackages entire ecosystems into their own repository. This greatly inflates the number of packages. You can't look at the flat numbers.

> However the Nix project was designed from the ground up around reproducibility.

It wasn't.

> It also pioneered architectural approaches that other systems have tried to emulate since.

This has had no bearing, and you are greatly overestimating the technical details of nix here. It's fundamentally invented in 2002, and things has progressed since then. `rpath` hacking really is not magic.

> I think you're grossly misunderstanding the role Nix played in this effort.

I've been contributing to the Reproducible Builds effort since 2018.

I think people are generally confused the different meanings of reproducibility in this case. The reproducibility that Nix initially aimed at is: multiple evaluations of the same derivations will lead to the same normalized store .drv. For a long time they were not completely reproducible, because evaluation could depend on environment variables, etc. But flakes have (completely ?) closed this hole. So, the reproducibility in Nix means that evaluating the same package set will lead to the same set of build recipes (.drvs).

However, this doesn't say much about build artifact reproducibility. A package set could always evaluate to the same drvs, but if all the source packages choose what to build based on random() > 0.5, then there is no of build artifacts at all. This type of reproducibility is spearheaded by Debian and Arch more than Nix.

Different notions of reproducible. This project cares specifically about bit-for-bit identical builds (e.g. no time stamps, parallel compile artifacts etc). Nix is more about being declarative and "repeatable" or whatever a good name for that would be.

Both notions are useful for different purposes and nix is not particularly good at the first one.

https://reproducible-builds.org/citests/

It's very, very complicated. It's so far past the maximum effort line of most linux users as to be in its own class of tools. Reproducibility in the imperative package space is worth a lot. Lots of other tools are built on RPM/DEB packages that offer similar advantages of Nix -- Ansible, for one. This is more of a "rising tide raises all boats" situation.

It's an article about Fedora, specifically.

Yes, I know that. But when talking about reproducible packages, we can and should learn from existing techniques. Is Fedoras system capable of this goal? Is it built the right way? Should it adopt an alternate package manager to achieve this with less headache?

"What about Nix tho?" is the "Rewrite it in Rust" of reproducible builds.

No other currently distro whether it be debian, fedora, or arch, or opensuse seems to want to switch up their design to match nix's approach.

They have too many people familiar with the current approaches.

That's not even really true. Hacker News readers seem to fixate heavily on the most popular desktop versions of Linux distros, but that is far from all they make. SUSE with SLE Micro and Edge Image Builder is a declarative build system for creating your own installer images. If you don't want SLE Micro, they even give you the Elemental Toolkit, which can be used to build your own custom distro in a declarative manner. I don't know much about Debian and Red Hat's approaches to this, but I know they have similar tooling. The "problem," if you want to call it that, is developers, if I may be uncharitable for a brief moment here, are amazingly solipsistic. The major Linux vendors are targeting enterprise edge computing use cases, whereas developers only ever think about the endpoint PCs they themselves use for development. Plenty of distro tooling will give you declaratively built, transactionally updated, immutable server images, but if they don't give you the same for your personal laptop, those efforts are invisible to the modal Hacker News commenter.

For what it's worth, there's also Guix, which is literally a clone of Nix but part of the GNU project, so it only uses free software and opts for Guile instead of a custom DSL for configuration. It wasn't a pre-existing distro that changed, of course.

Guix counts, but what suse is doing does not. It is still based on rpm packages underneath last i checked.

Because Nix is a huge pain ramp up on and to use for anyone who is not an enthusiast about the state of their computer.

What will happen is concepts from Nix will slowly get absorbed into other, more user-friendly tooling while Nix circles the complexity drain

Nix is to Linux users what Linux is to normies.

Reply on news.ycombinator.com