Comment by chriswarbo | Hacker News Reader

Comment by chriswarbo | Hacker News Reader

I build container images using Nix (actually OCI containers; I don't use Docker at all, although I think it can run OCI containers). Going through these points one at a time:

> Use a smaller base image or libc

By default, Nix will build container images from scratch. No base image (although I guess you can add one as a layer to your manifest JSON, if you like).

> Make the image context smaller

Nix tracks dependencies very precisely. Our container images will only contain the things we asked for, and their dependencies (transitively).

> Fully minimize and tidy up image layers

This only seems relevant for imperative images, built "from the inside" (e.g. the talk about contents being "overwritten"?)

> Splitting up processes and services for breaking up images

The helper-functions in Nixpkgs contain tricks to automatically figure out which content is better kept in separate layers, based on how many times they're referenced (as a proxy for how often they'll be shared across containers). Docker only supports 128 layers, so the remaining content gets combined into the final layer; e.g. see https://grahamc.com/blog/nix-and-layered-docker-images

I recently tried to use nix to build images (docker images though, not sure if k8s can run OCI, or if I can push OCI to a docker registry (artifactory)).

What I found cumbersome though is to «cross-compile» images, I'm on mac/arm but the images need to run on linux/amd64. Ok my code is JavaScript (Node.js), I can compile the sources into plain JS on any machine, and then pack into an image together with nodejs from `pkgsLinux = import <nixpkgs> { system = "x86_64-linux"; };`, and all will work. I can even install node_modules on my build machine since the node_modules folder is (often) just a bunch of .js files. But some dependencies have native code, so I'd have to cross-compile those. And that's where I gave up.

I haven't attempted this yet but I have wondered about the feasibility of using the nixos/nix docker image just to build a linux/amd64 docker image using Nix. Or setting up a nixos/nix docker image as a remote builder for Nix.

> not sure if k8s can run OCI, or if I can push OCI to a docker registry (artifactory)

I push OCI images to AWS ECR, so it would probably work.

> I'm on mac/arm but the images need to run on linux/amd64

I've never used the cross-compilation support in Nixpkgs. My work machine runs macOS (x86_64), so I have a Linux VM as a remote builder. I happen to use LimaVM ( https://github.com/lima-vm/lima ), but anything would work (Qemu, VirtualBox, the VM bundled with Docker Desktop, etc.).

PS: Since containers only work on Linux, and I'm often working on macOS, I use the following assertion to check that I'm not accidentally including Mac builds in a layer:

    assert all (d: ((d.type or null) == "derivation") -> (d.system == "x86_64-linux")) drvs;

Blogpost or it didn't happen! (Seriously though, I would love to read a post detailing this process start-to-finish for the uninitiated)

Here's [1] my favorite thus far using Nix's new flakes to build a Rust project into a container image with Nix.

[1] https://community.fly.io/t/running-reproducible-rust-a-fly-a...

Nixos website has very nice example of minimal docker images: https://nixos.org/#asciinema-demo-example_4

I get why nix is attractive, really, and its cool you can build docker images.

But for people who don't know nix its a bit strange to say that Dockerfiles are tedious, and then proceed to show a much more arcane and verbose way of building them as a solution to that tediousness.

The one thing that makes Dockerfiles attractive (and Docker in general), is that they are so simple and intuitive. You can immediately leverage your basic linux knowledge. That is why a lot of people like it whose main job is not about packaging.

I think the keyword for that phrase is "reliable".

Overtime, I had to go back to fix "working" base images that had become stale, and when dealing with layers and layers of base images this really gets tedious.

BTW, I haven't dealt with nix generated images in production, I'm not aware of possible drawbacks (other than people not being familiar with nix), so I'm not really endorsing it, although I would like to try.

What has this got to do with making Docker images smaller?

Presumably parent's point is "making Docker images with Nix makes them very small [smaller even than the tips mentioned in the article]."

And it does so more or less by default, for free, without having to make any special effort.

Edit: Any special effort beyond using nix, I should say; that is a significant cost if you're not already paying it.

> What has this got to do with making Docker images smaller?

"Docker images" are .tar.gz files containing some binary executable.

The article starts by putting an entire Linux distribution into the .tar.gz (Ubuntu, Alpine, etc.), then goes through several ways to try and make the result smaller.

Nix does things the other way: the .tar.gz will only contain the specified binary, plus its dependencies. It's minimal by default (although there are still tricks to making things smaller, e.g. taking dependencies from `pkgsMusl` instead of `pkgs`, to avoid glibc)

I've started building my personal projects like this but the end result is an image running as root, which is not best practice. Have you been able to build images with Nix that run as an arbitrary user?

There is an example of doing this in the docker-tools examples!

https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...

But I have to learn nix script, which is not easy to learn.

I'd say the common cases, (especially if someone has already done something similar that you can copy-paste from), are reasonably straightforward.

Generally, getting a good understanding of Nix takes some time. It's hard because you need to understand what Nix does, and understand what you're trying to do, and how it constrains what you're trying to do.

I used Nix at work but eventually realized, managing dependencies, compiling cross-platform, and deploying wasn’t actually bad enough to warrant that level of complexity

Reply on news.ycombinator.com