Intro Guide to Dockerfile Best Practices

I honestly don't understand this sentiment (other than shamelessly plugging your posts, which is ok).

The tradeoff in any packaging mechanism is always "pin everything to the byte" for maximum security and minimum updateability vs. "blindly update to latest without thinking". We normally develop with the second and deploy using the first and docker is no different.

> Solution: rebuild once a week from scratch.

I’ve noticed the official docker images don’t seem to do this. E.g. the official “java” images seem to be uploaded and then are never changed, the only way to get a newer underlying base system is to upgrade to a newer version tag release. Is this true of all the official images, I wonder?

3. Don't run processes are root, most of the times you don't have to, it's an easy easy win and protects against many docker security CVEs.

I find it odd that docker pipelines don't cache all the layers by default. That is to say, that they don't push all cached layers to a central repository. I'm not even sure if it's possible.

Is there a reason why this would be bad? Clearly you'd have to clean the old cache layers regularly, but I'm more concerned with some layers taking very long times - the caching seems required.

Though I suppose if you set up your CI system to store the caches locally then you get caching, and it's more efficient as you're not downloading layers. So maybe that's just the "right" way to do it, regardless. /shrug

The flipside of 1. though is that cached images can't be hijacked and have vulnerabilities injected, or from a non security standpoint, have breaking changes introduced which mean the image no longer builds and/or runs how you're expecting.

I ended up expanding this into a blog post covering not the linked article (though it shares some of the same flaws), but the official docs: https://pythonspeed.com/articles/official-docker-best-practi...

> Solution: you need to tag and push the build-stage images too

With BuildKit cache for all the intermediate stages is tracked. You can push the whole cache with buildx or inline the intermediate cache metadata in buildx or v19.03. `--cache-from` will pull in matched layers automatically on build. Can also export the cache to a shared volume if that suites you better than a registry.

But in order to build with BuildKit you MUST BE connected to the internet during the build... So not ready to go yet...

COPY --chown

You can use the official images and tag them with the SHA image ID - that should give cryptographically enforced security and reproducibility.

As I read it, the tip is to always build in a consistent environment. I think a CI pipeline counts in that regard.

The way I read it, they're more talking about when you're developing locally everyone should be building the application inside of a container rather than on their personal machines with differing set ups.

The same problems that apply to production environments also apply to CI systems - you need to make sure those build agents are project-aware, up to date, and if you decide to move to a new JDK on one project you'll need to update your build servers, and good luck to you if you want to update only some of your projects.

The appeal of docker is completely & reproducibly owning production (what runs on your laptop runs on prod), and that also applies to the build (what builds on your laptop builds on prod). Not to mention the add on benefits that you can now use standard build agents across every tech stack and project, no need to customize them or keep them up to date, etc.

For me the big advantage of doing more in docker and lees in the CIT environment is that I have less lock-in/dependency to whatever my CI provider does. I try to reduce my CI scripts to something like

    docker build
    docker run image test 

All complexities of building and collecting dependencies go in dockerfiles, so I can reproduce it locally, or anywhere else. And importantly, without messing with any system settings/packages. No more makefiles or shell scripts that make a ton of assumptions about your laptop that need to be set just right to build something from source; just docker build and off you go. Such a hassle when you need to follow pages of readme just to build something from source (plus a lot of installed dependencies that you have to clean up afterwards)

With multi-stage builds you get a bunch of benefits. You can pull the heavy SDK when you start building the app, and that gets cached. Then when you package the image, you copy the jar that was built, but not the heavy SDK. When you run this again, the heavy/expensive steps are skipped because they're cached. Now you have a single set of operations to build your app and its production image, so there are no chances for inconsistencies.

In addition, you can build a separate container from a specific part of your multi-stage build (for example, if you want to build more apps based on the SDK step, or run tests which require debugging). So from one Dockerfile you can have multiple images or tags to use for different parts of your pipeline. The resulting production image is still based on the same origin code, so you have more confidence that what's going to production was what was tested in the pipeline.

Furthermore, devs can iterate on this Dockerfile locally, rather than trying to replicate the CI pipeline in an ad-hoc way. The more of your pipeline you stuff into a Dockerfile, the less you have to focus on "building your pipeline".

If you use the latest tag of your base image, this is definitely true. It can vary by official image, but I’ve noticed a lot of them will tag a version and leave it untouched until a new version is released. But, it is technically possible they could push a change to an older version image.

Build a java application with RHEL-base-jdk:10 and from commit id:[HASH], that is as reproducible as it gets. As is stated you can save all the build dependencies in an Docker image, so you can go back and look at it later. Usually it is enough to have the build logs though.

We set system time as well.

There are some people trying to build alternative toolchains for Docker builds that remove Dockerfiles altogether, e.g. https://buildpacks.io/ and (in a very different approach) Nix-based builds.

My personal approach for Python applications' Docker packaging (https://pythonspeed.com/products/pythoncontainer/) was similar to yours: wrap the build process in a script. I wrote it in Python rather than bash, so it's more maintainable, but a Ruby shop might want to write theirs in Ruby, etc..

Could you elaborate?

Please don't downvote parent. This is true.

It is a fact that everything that happens in a Dockerfile execution via `docker build` can be done with a `docker run` and executing commands and ending with `docker commit`. Even the caching mechanism can be replicated. (Except with more control in my opinion.) It is also a fact that `docker run` has more capabilities than `docker build`, such as being able to mount things in.

docker = bash script + unshare + MANY unneeded limitations

But the nicer packaging and simpler model make that new layer of abstraction useful.

I recommend you to upgrade to BuildKit. Also, the above isn't really true for Docker CLI, mtime was never taken into account in the old builder and Docker CLI never sends uname/gname. With API it was possible in the old builder though (hence the incompatibility with compose for example). In more practical cases I've seen it caused by git directory instead, that isn't stable. In that case, you can put .git into .dockerignore or if you don't use it, BuildKit will discard it automatically.

I think you're not the only one that mentioned this omission, I'll throw in since it seems to be pretty poorly highlighted and has some surprising failure modes:

When you use the USER directive to set up a working user in Dockerfile, be explicit about UID and use the numeric UID.

What works 99% of the time is to use alpha username in the "USER" directive. You can get some surprising artifacts if the container runtime hits this rare bug[1]. There are likely several other great ways this can go wrong, as well. Somewhere deep in the manual, it is suggested that you should only use a numeric UID, even though USER accepts an alphanumeric username most of the time, but if you look at container images built by OpenShift and other pro docker-ers, you will see they always do this with a numeric UID.

[1]: https://forums.docker.com/t/unable-to-find-user-root-no-matc...

You should not include binaries not needed for production for security reasons, not because of size alone.