Hacker News Reader

Comment by mainguy | Hacker News Reader

I think OP is coming from a different perspective than I (corporate bespoke solution builder) do. When I say "reproducible build" I mean a build that is the same on any machine (i.e. no special magic necessary to build a "official" version of the code). Too often in corporate environments, getting a local build or setting up a new build pipeline involves arcane black magic and/or copy/pasting weird libraries that can't be pulled from any sort of "official" repository. curl/bash libraries that "automatically change versions" based on when upstream decides to change them can wreak havok when setting up a new build environment. My $0.02, it's not (in the corporate world) so much about validating binaries, but more about "how many steps beyond check out the code" exist and how easily can I validate my binary uses the same versions of libraries/dependencies as the one that a local developer tested?

View on HN

It’s a well-defined term though

https://en.wikipedia.org/wiki/Reproducible_builds

that is a good page:

The GNU project used reproducible builds in the early 1990s. Changelogs from 1992 indicate the ongoing effort. [4]

One of the older[5] projects to promote reproducible builds is the Bitcoin project with Gitian. Later, in 2013, the Tor (anonymity network) project started using Gitian for their reproducible builds.[6]

In July 2013 on the Debian project started implementing reproducible builds across its entire package archive.[7][8]

By July 2017 more than 90% of the packages in the repository have been proven to build reproducibly.[9]

In November 2018, the Reproducible Builds project joined the Software Freedom Conservancy.[10]

F-droid uses reproducible builds to provide a guarantee that the distributed APKs use the claimed free source code.[11

Well I think that's the crux of the issue, i've been using the term reproducible build to mean something different for the last 25 years. I'm after functional equivalence not binary equivalence. Now it might be impossible to guarantee one without the other, but that's all I care about.

I get where you are coming from, but “reproducible build” is a well-define term especially in the realm of kernels / OSes

For this reason I’ve been dockerizing my builds for almost five years. I was late to the Docker party, but when I saw the benefits it brings to build pipelines, I was sold.

It's true that a dockerized build isn’t any simpler than its non-dockerized ancestor, but at least there’s a Dockerfile that lays bare all the black magic and special sauce which goes into each build. And it can be version controlled to watch for drift over time.

This stuff is useful in a corporate setting, but the other fetishization of reproducible builds is just a distraction that can stay where it belongs: open source mailing lists.

But docker builds themselves are generally not reproducible, so I don't really see the gain?

Yes with some effort, they can be made reproducible but the vast majority of the dockerfiles that I've encountered do not pin the versions of every dependency.

Some might pin a few key dependencies but nearly all do an apt/rpm/whatever update at some point followed by a bunch of install commands which don't specify versions.

While your Dockerfile helps you know how a project was built at a specific point in time, it's not going to work forever. Even if the file doesn't change over time, the build it produces will. It's mainly because of installing packages using something like "apt-get install $package". It also can change if the files you're adding with ADD or COPY change.

You don’t have to download the internet upon each build.

First, in a corporate environment it’s common to run builds backed by artifact servers that’ll cache just about anything.

Second, it’s easy to place files in a Docker build context (that’s just a $25 dollar way of saying “next to the Dockerfile”) that would have been downloaded from the internet, but are stored locally instead. This is easier said than done for some formats. Source tarballs? Easy. Anything Java or Debian that requires a pesky server which works a certain way? You’re going to have to use a caching artifact server.

While Docker can be very useful against attempts to "download the internet" (possibly simulating multiple remote servers on a fake network) and aganst accidental changes to source files, configurations and tools, there are sources of intentional nonreproducibility (e.g. embedded timestamps, common in Windows executables) that need to be addressed more directly.

Reply on news.ycombinator.com