Hacker News Reader

Comment by dev_l1x_be | Hacker News Reader

All docker containers should have been like that. apt-get update in a docker build step is an anti pattern.

You are screwed either way. If you don't update your container has a ton of known security issues, if you do the container is not reproducable. reproducable is neat with some useful security benefits, but it is something a non goal if the container is more than a month old - day might even be a better max age.

Why is there a need for a package manager inside a container at all? Aren't they supposed to be minimal?

Build your container/vm image elsewhere and deploy updates as entirely new images or snapshots or whatever you want.

Personally I prefer buildroot and consider VM as another target for embedded o/s images.

So if i have a docker container which needs a handful of packages, you would handle it how?

I'm handling it by using a slim debian or ubuntu, then using apt to install these packages with necessary dependencies.

For everything easy, like one basic binary, I use the most minimal image but as soon as it gets just a little bit annoying to set it up and keep it maintained, i start using apt and a nightly build of the image.

IMO—package manager outside the container. You just want the packages inside the container; the manager can sit outside and install packages into the container.

Yes, how?

With Red Hat's UBI Micro:

  microcontainer=$(buildah from registry.access.redhat.com/ubi8/ubi-micro)
  micromount=$(buildah mount $microcontainer)
  yum install \
      --installroot $micromount \
      --releasever 8 \
      --setopt install_weak_deps=false \
      --nodocs -y \
      httpd

(from https://www.redhat.com/en/blog/introduction-ubi-micro published in 2021)

great. Now I have to install and learn another tool when having yum inside the container will just work?

Not just that but it will probably break later and ruin everything

For the package management, it depends on the package manager, but most have some mechanism for installing into a root other than the currently running system.

Even without explicit support in the pacakage manager, you could also roll your own solution by running the package manager in a chroot environment, which would then need to be seeded with the package manager's own dependencies, of course (and use user-mode qemu to run pre- and post-installation scripts within the chroot in the case of cross-architecture builds).

Whether this yields a minimal container when pointed at a repository intended to be used to deploy a full OS is another question, but using a package manager to build a root filesystem offline isn't hard to pull off.

As for how to do this in the context of building an OCI container, tools like Buildah[1] exist to support container workflows beyond the conventional Dockerfile approach, providing straightforward command line tools to create containers, work with layers, mount and unmount container filesystems, etc.

[1] https://github.com/containers/buildah/blob/main/README.md

There have got to be a million ways to do this by now. Some of the more principled approaches are tools like Nix (https://xeiaso.net/talks/2024/nix-docker-build/) and Bazel (https://github.com/bazel-contrib/rules_oci). But if you want to use an existing package manager like apt, you can pick it apart. Apt calls dpkg, and dpkg extracts files and runs post-install scripts. Only the post-install script needs to run inside the container.

I may be a little out of touch here, because the last time I did this, we used a wholly custom package manager.

Docker recommends using multi-stage builds e.g. Stage one image has the package manager, stage two image omits it completely, leaving only the installed software.

apk and xbps can do this. You specify a different root to work in.

Most Makefiles allow you to specify an alternate DESTDIR on install.

You don’t even need most of the files in the packages. Just pull out the files you need.

The same way you may require something like cmake as a build dependency but not have it be part of the resulting binary - separate build time and run time dependencies so you only distribute the relevant ones.

I do not talk about multi-step container images.

For example i run a gcs fuse driver, it has other dependencies apt 'just' resolves.

Your question feels insane to me for production environments. Why aren't you doing a version cutoff of your packages and either pulling them from some network/local cache or baking them into your images?

I don't just run a java spring boot application. I run other things on my production system.

It doesn't matter much were i pull them from though, i only do this with packages which have plenty of dependencies and i don't want to assemble my own minimal image.

Aforementioned security vulnerabilities don’t strike as a potential reason to you?

Friend, considering the supply chain attacks going on these days, automatically updating everything, immediately, probably isn't the perfect move either.

You need to automatically update from a trusted source. That source better audit and update constantly. Which is hard.

Stable distributions have security teams.

Ignoring the real benefits of security updates to prevent the unlikely event of supply chain attacks sounds like a weird tradeoff.

A weird tradeoff but an increasingly important tradeoff to keep in mind nonetheless. Like I said, updating immediately isn't a perfect answer. But neither is waiting. I hope you're having this discussion, at least.

That local cache is often implemented as a drop-in replacement for the upstream package repository, and packages are still installed with the same package manager (yum,apt,pip,npm).

No, this is not always the case. Regulated industries pin their package versions and store those versions for pulling.

Minimal might or might not pe your goal. A large container sometimes is correct - at that point you have to ask if maybe using a container twice so you only need to download it once and then installing the one missing part makes more sense.

> minimal

I run systemd, sshd and xpra (remote X11) inside my arch container.

For some scenarios simply building a new image every day is reasonable.

I update my docker containers regularly but doing it in a reproducible, auditable, predictable way

Could you explain how you achieve this?

If you are on github/gitlab, renovate bot is a good option for automating dependency updates via PRs while still maintaining pinned versions in your source.

have a multi layer build system where there is version of each layer. Example

- linux base: kernel + libc + some low tooling

- jvm base: adding the JRE/JDK you need

- your application version

Chainguard, Docker Inc’s DHI etc. There’s a whole industry for this.

I know it's an anti-pattern, but what is the alternative if you need to install some software? Pulling its tagged source code, gcc and compile everything?

Copying from another image is an under appreciated feature

FROM ubuntu:24.04

COPY --from=ghcr.io/owner/image:latest /usr/local/bin/somebinary /usr/local/bin/somebinary

CMD ["somebinary"]

Not as simple when you need shared dependencies

Both Debian and Ubuntu provide snapshot mirrors where you can specify a date to get the package lists as they looked at that time.

Which is only useful for historical invesigation - the old snapshot has security holes attackers know how to exploit.

> the old snapshot has security holes attackers know how to exploit.

So is running `docker build` and the `RUN apt update` line doing a cache hit, except the latter is silent.

The problem solved by pinning to the snapshot is not to magically be secure, it's knowing what a given image is made of so you can trivially assert which ones are safe and which ones aren't.

In both cases you have to rebuild an image anyway so updating the snapshot is just a step that makes it explicit in code instead of implicit.

where does the apt update connect to? If it is an up to date package repo you get fixes. Howerer there are lots of reasons it would not. You better know if this is your plan.

You get fixes that were current at docker build time, but I think GP is referring to fixes that appear in the apt repo after your docker container is deployed.

If you've pulled in a dependency from outside the base image, there will be no new base image version to alert you to an update of that external dependency. Unless your container regularly runs something like apt update && apt list --upgradable, you will be unaware of security fixes newly available from apt.

Yeah that's yet another annoying thing to consider

Also I'm tired of doing these hacks:

    # increase to bust cache entry
    RUN true 42 && apt update

Pinning to a snapshot just makes so many things easier.

if anyone wants to know how, i wrote about it here https://tuananh.net/2024/01/21/reproducibility/

Run “nix flake update”. Commit the lockfile. Build a docker image from that; the software you need is almost certainly there, and there’s a handy docker helper.

Recently I’ve been noticing that Nix software has been falling behind. So “the software you need is almost certainly there” is less true these days. Recently = April 2026.

That's been an issue for years from my impression of the state of NixOS. There are other problems too, like a lot of open source packages doing straight binary downloads instead of actually building the software.

Are you referring to how the nixpkgs-unstable branch hasn't been updated in the past five days? Or do you have some specific software in mind? (not arguing, just curious)

It’s a variety of different software that just isn’t updated very often.

I don’t mind being somewhat behind, but it seems like there are a lot of packages that don’t get regular updates. It’s okay to have packages that aren’t updated, but those packages should be clearly distinguishable.

oh, great, adding more dependency, and one that just had serious security problem

as if other sandboxing software is perfect

Nothing is perfect. (FreeBSD jails come close but still no.)

With a binary cache that is not so bad, see for example what nix does.

I don't really see how that's different from a normal binary install of a reproducible package. Especially with the lacking quality of a lot of Nix packages.

If you're in a situation where you want reproducibility you're using nix to build your own packages anyways, not relying on their packages

It's not if you can pin the package. It gives you reproducable docker containers without having to rebuild the world. Wasn't that the entire question?

Wasn't the question about modifying an existing image?

No, it was about putting apt update in docker file and it not being reproducible.

Use https://github.com/reproducible-containers/repro-sources-lis...

pretend you don't do it and add your extra software to the layer above

base image

software component image

both should be version pinned for auditing

…unless you use https://github.com/reproducible-containers/repro-sources-lis... . I've been using it for a while and it's worked really well for me!

I disagree with that as a hard rule and with the opinion that it's an anti-pattern. Reproducible containers are fine, but not always necessary. There's enough times when I do want to run apt-get in a container and don't care about reproducibility.

I agree with your opinion.

Reproducible can sometimes be a goal, but repeatable is always important.

I do think for this case specifically (base images for a specific distro), they should be reproducible.

The problem is distros often remove older versions from the repo as soon as the new version is available. Granted there is an archive that you can pull from.

This is to solve such issues that I am using and running StableBuild.

It is a managed service that keeps a cached copy of your dependencies at a specific time. You can pin your dependencies within a Dockerfile and have reproducible docker images.

I don't wanna be that guy but...

NIX FIXES THIS.

So does Bazel. :p

adding to the list, one exotic approach to this problem is stagex https://codeberg.org/stagex/stagex

This has been a solved problem for over two decades now with Nix but people can't be asked

It has been solved even without Nix for a long time, just laziness is probably why we are not doing it

Reply on news.ycombinator.com