Hacker News Reader

Comment by UniqueUsername0 | Hacker News Reader

Repeatable, not reproduicible...

Are you using bazel as your build system? If not, for almost all languages, your build system itself is probably not reproducible. Gcc uses random numbers. Lots of compilers add timestamps to builds (javac does).

For almost all non-Google use cases, repeatable is generally good enough.

It's reproducible enough for most purposes most of the time. When is the last time your Python web app serving cat pictures went down because of a subtle change in the Debian Bookworm container?

...Funny you bring that up.

I was cobbling together build scripts for a mail system for Raspi/Armbian the last couple weeks. Very similar packaging stuff, but the number of little subtle differences in install/postinstall/prerm/postrm scripts took a generous level of spackling over to get just right.

Hell, once I get everything nailed down, I'm writing test frameworks for my bloody build scripts if you can believe it.

So...uh... Last week?

Fair enough, but I don't think that's a typical experience.

> It's reproducible enough for most purposes most of the time.

The freedom given in ability to limit your thinking space by reproducibility should not be under stated.

Same with static types. Sometimes it really helps to limit yourself, other times it's setting up unnecessary obstacles with no benefit. Sometimes you want Idris 2 or Rust, and sometimes you want Ruby or Clojure.

Those sound the same to me. Are you drawing a distinction between a repeatable/reproducible process vs result? Like if you run the same command to fetch a dependency, you wind up with different results if the dependency maintainer releases a new update and you aren’t pinning the dependency?

Reproducible in that context means that repeating will produce exactly the same output every time.

To use the apt analogy further up in the thread, `sudo apt install git` is repeatable in your dockerfile, but often not reproducible. Later on you will get a different build. Across say 500 packages and 1,000,000 containers (or say 1000 container images if you are deploying images) over even a week this becomes extremely... varied...

On smaller scales, this is perfectly fine. How often does Git actually release breaking changes of features that you actually need to use inside your Dockerfile? How often does Debian pull in such a version into their stable OS? And why didn't you just version-pin Git like Hadolint told you to do?

Exact reproducibility is nice for two scenarios: 1) academic research, and 2) very large-scale applications and deployments. For regular people writing boring small web apps, choosing a stable base image and pinning dependencies is good enough.

Consider also that your preferred programming language will also very likely not provide particularly reproducible package builds.

Although you can.

This just means you don't.

Try using `sudo apt install git=1:2.39.2-1ubuntu1`

That pins it to a particular version so that it should be reproducible.

Would that work, though?

I've never looked seriously into it, but my feeling is that distros will delete old versions as newer ones are uploaded: When I run "apt-cache policy git" in my Ubuntu, I only see a couple versions available to install, often other packages show only a single one (so, the latest).

I know that Debian has Snapshot for older packages but you are still at the will of other people and people are fickle, and Nix should allow you to use specific versions to build your base images from to pin to.

However, much in the same way that if you actually take your build system seriously you'll store your application dependencies in a local proxy, you can run a mirror or proxy to hold these historical packages too.

Take a look at something like apt cacher, however it is a proxy cache so you can reproduce builds using the exact same package versions but if upstream delete old packages, and you want to roll back to one you haven't previously downloaded, then you are out of luck.

Yes. This can be a problem in scientific research involving numerical code or random number generation, where results can vary even due to small inconsequential-seeming changes, leading to your results not being reproducible scientifically because they're not reproducible computationally.

Reply on news.ycombinator.com