Hacker News Reader

Comment by time0ut | Hacker News Reader

I can’t live without it.

The power of docker compose for the development process is unparalleled. Being able to declare my entire local environment in a single file in a consistent way, create and destroy it with a simple command, and share it with my co-workers eliminates so many issues.

Then being able to package my application in way that is repeatable and predictable is great. Then that same artifact is run across multiple environments spanning data centers around the world and on other developers’ machines via their compose files if needed.

We test the crap out of all of this as well. Docker helps there to generate ephemeral environments to run the bulk of the integration tests against every time we push to git.

Its just so damn powerful compared to what we had before…

View on HN

Repeatable, not reproduicible...

Those sound the same to me. Are you drawing a distinction between a repeatable/reproducible process vs result? Like if you run the same command to fetch a dependency, you wind up with different results if the dependency maintainer releases a new update and you aren’t pinning the dependency?

Reproducible in that context means that repeating will produce exactly the same output every time.

To use the apt analogy further up in the thread, `sudo apt install git` is repeatable in your dockerfile, but often not reproducible. Later on you will get a different build. Across say 500 packages and 1,000,000 containers (or say 1000 container images if you are deploying images) over even a week this becomes extremely... varied...

On smaller scales, this is perfectly fine. How often does Git actually release breaking changes of features that you actually need to use inside your Dockerfile? How often does Debian pull in such a version into their stable OS? And why didn't you just version-pin Git like Hadolint told you to do?

Exact reproducibility is nice for two scenarios: 1) academic research, and 2) very large-scale applications and deployments. For regular people writing boring small web apps, choosing a stable base image and pinning dependencies is good enough.

Consider also that your preferred programming language will also very likely not provide particularly reproducible package builds.

Although you can.

This just means you don't.

Try using `sudo apt install git=1:2.39.2-1ubuntu1`

That pins it to a particular version so that it should be reproducible.

Would that work, though?

I've never looked seriously into it, but my feeling is that distros will delete old versions as newer ones are uploaded: When I run "apt-cache policy git" in my Ubuntu, I only see a couple versions available to install, often other packages show only a single one (so, the latest).

I know that Debian has Snapshot for older packages but you are still at the will of other people and people are fickle, and Nix should allow you to use specific versions to build your base images from to pin to.

However, much in the same way that if you actually take your build system seriously you'll store your application dependencies in a local proxy, you can run a mirror or proxy to hold these historical packages too.

Take a look at something like apt cacher, however it is a proxy cache so you can reproduce builds using the exact same package versions but if upstream delete old packages, and you want to roll back to one you haven't previously downloaded, then you are out of luck.

Yes. This can be a problem in scientific research involving numerical code or random number generation, where results can vary even due to small inconsequential-seeming changes, leading to your results not being reproducible scientifically because they're not reproducible computationally.

It's reproducible enough for most purposes most of the time. When is the last time your Python web app serving cat pictures went down because of a subtle change in the Debian Bookworm container?

...Funny you bring that up.

I was cobbling together build scripts for a mail system for Raspi/Armbian the last couple weeks. Very similar packaging stuff, but the number of little subtle differences in install/postinstall/prerm/postrm scripts took a generous level of spackling over to get just right.

Hell, once I get everything nailed down, I'm writing test frameworks for my bloody build scripts if you can believe it.

So...uh... Last week?

Fair enough, but I don't think that's a typical experience.

> It's reproducible enough for most purposes most of the time.

The freedom given in ability to limit your thinking space by reproducibility should not be under stated.

Same with static types. Sometimes it really helps to limit yourself, other times it's setting up unnecessary obstacles with no benefit. Sometimes you want Idris 2 or Rust, and sometimes you want Ruby or Clojure.

Are you using bazel as your build system? If not, for almost all languages, your build system itself is probably not reproducible. Gcc uses random numbers. Lots of compilers add timestamps to builds (javac does).

For almost all non-Google use cases, repeatable is generally good enough.

Agreed, and the fact that I can launch and develop my web app from seven years ago, no matter how much my environment has changed, is a lifesaver.

There's a lot of code I've written that's not deployable any more because the versions of the dependencies it was using don't exist on my OS any more, and are too old to install.

As long as you have saved the container image and contents of all mapped volumes somewhere. Which is not much different from snapshotting a VM.

Snapshotting a VM is a pain in the ass. It takes forever and the tooling sucks. The concept of snapshotting is also completely wrong. Snapshots are tied to a VM with the specific VM being front and center, you know, the thing that is disposable with containers.

Also, from my misadventures with Ubuntu VMs I always ran into the problem of them breaking over time.

Reply on news.ycombinator.com