Lots of progress for Debian's reproducible builds | Hacker News Reader

Lots of progress for Debian's reproducible builds

lwn.net

57 pointsedward22 comments

Open articleView on HN

Why the re-post?

https://news.ycombinator.com/item?id=8950292

Debconf15 is happening right now. Here is a video with details of the latest progress: (warning 2.1G)

http://meetings-archive.debian.net/pub/debian-meetings/2015/...

Sorry, I didn't realise it had been posted before.

Unfortunately neither the excellent LWN article nor the comment thread so far talk about the main benefit of reproducible builds, which is security, at least potentially. If you're running an irreproducible build of some piece of software in your user account, you're running on hope that the person who compiled that software didn't have malware on their machine that inserted a backdoor into the software. Reproducible building makes that kind of attack enormously more difficult: not only does the attacker need to insert the same backdoor on the machines of all the developers who are reproducing the build, but now the malware is almost certain to be detected, sooner or later. Coupled with diverse double-compiling, reproducible build processes can even detect the Thompson attack.

The big push for reproducible builds came in the wake of the Snowden evidence that the NSA was specifically and systematically attacking the machines of system administrators in order to gain access to the machines they controlled. Debian and Tor developers should logically be even bigger attack targets.

If you consider the effort required for this, I always wonder how reproducible the things in Docker or NixOS really are.

There's practically reproducible and completely reproducible.

Let's say you had a bug, and are trying to see when it was introduced.

Something like the compilation time printed in a log at startup you don't really care about, the versions of each part of your build chain and all dependencies are where your focus will be. This is easier to achieve.

Something like the compilation time printed in a log at startup you don't really care about

Unless the compilation time is the source of the bug :)

Docker does nothing to improve reproducibility in the way that the Debian (and other) folks are using the term, which means "bit-identical builds". Nix, and likewise GNU Guix, already provide a great framework for enabling reproducible builds. In the Guix manual, we say that the builds are nearly reproducible because as Debian shows, there is still plenty of work to be done, but we do eliminate a lot of sources of nondeterminism already.

Docker is the opposite of reproducible. The idea is that you (as a user) have a known tested good image you can re-use it easily without having to worry about reproducibility (which is the responsibility of the other side - whoever is producing the images).

If you could reproduce that environment easily and instantly, you wouldn't need Docker since you'd just reproduce it every time.

Debian wasn't built with reproducibility in mind, whilst NixOS was. "Backporting" features to existing systems is monotonically harder than building a new system with that feature, since we have more constraints.

Even if, hypothetically, it were trivial for NixOS to obtain perfect reproducibility, that doesn't necessarily help with making Debian reproducible in a backwards-compatible way.

Both kinds of work are useful, and cross pollination is likely possible - the issue list here is enlightening:

https://reproducible.debian.net/index_issues.html

Almost none of those things are debian specific - they're things like "the timestamp of when this was built is put in the documentation" or "the kernel version is put in the binary".

Issues of those types would affect any OS and build system, and are would impact any system, even one with source purity goals like NixOS and similar.

Honest question: is NixOS really doing reproducible builds, where you get byte-identical binaries when rebuilding from source?

I only found this issue on the Tor tracker, and apparently NixOS does not produce reproducible builds in the same way as intended by Debian: https://trac.torproject.org/projects/tor/ticket/12520

The aim is to have such reproducible builds, for example builds are always performed with the system clock set to 1 second after the Unix epoch. The reality varies from package to package; reproducibility problems are counted as bugs, but as always there's a cost/benefit decisions about fixing them.

For NixOS it looks like there was some work on reproducible builds, but it seems to have stalled: https://github.com/NixOS/nixpkgs/pull/2281

Please correct me if I'm wrong, but wouldn't reproducible builds eventually cut down on the work of package maintainers for Linux distros? Looking from my outside perspective, once you get the packages building reproducibly the work to maintain them is reduced (especially if you can pass the responsibility for packaging to the developer). Couple that with a build-time test suite and the work becomes even less of a burden.

I hope this is the case, as I believe that one thing that holds Linux/BSD back from further refinement is the workload involved to maintain packages.

I don't think so. Reproducible builds means that compiling the binary packages results in byte-by-byte identical copies, even when done on different machines at different times: the packaging sources do not get any simpler (to the contrary, they may get slightly more complex due to additional concerns).

The XDG App effort to produce standardized runtimes and isolated applications is probably what you're looking for, but it is completely orthogonal to reproducible builds as runtimes for XDG App can be built in a multitude of ways, eg. with pre-built distribution packages (reproducible or not).

So this is sort of like lightweight nix in some aspects, without giving up on dynamic linking?

Nix doesn't give up on dynamic linking.

Care to explain this? My very newbish view of it was that every package is compiled against dependencies that are cryptographically hashed and it's totally bound to them.

They even try to link mutt to a particular version of gpg, which is a bit extreme as mutt calls gpg via shell so the interface between both is really really loose.

That said I really like nix.

I'm speculating here, but it seems like a case of missing the forest for the trees.

Yes they still use dynamic linking in the technical sense, as the libs are not folded into the resulting binary.

But they miss the spirit (or raison d'etre if you will) of dynamic linking by making hashing part of the linker requirements.

Its something that seems to crop up more and more in the Linux world these days.

That's exactly what I meant. Thanks for putting it in clearer words.

Now all they have to do is figure out how to build debian without systemd in and I might consider using it again.

The debian installer has a boot prompt option to install sysvinit, or if you want to be lazy, just install the sysvinit-core package and have the package manager uninstall systemd after first boot.

Reply on news.ycombinator.com