Optimising Docker Layers for Better Caching with Nix (2018) | Hacker News Reader

Optimising Docker Layers for Better Caching with Nix (2018)

grahamc.com

58 pointsTomte7 comments

There was a good talk on the same topic where the speaker gets a Redis image down to 2MB using nix/nixpkgs (see around the 1h5m mark): https://youtu.be/WP_oAmV6C2U

I think the official amd64 image is 40MB compressed, so it’s a pretty impressive feat: https://hub.docker.com/layers/library/redis/latest/images/sh...

I will yell about this Nix feature until my lungs dry up:

https://nixos.wiki/wiki/Impermanence

Boot to tmpfs. No. Hidden. State.

It's bliss.

Does anyone have a good curated list of Docker howtos and best practices?

I'm currently struggling with issues introduced by the Apple CPUs in the newer macbook pros (mbp): Some of our local amd64 dev/test images crash on M1 and M2 based laptops. I only have an Intel-based mbp and I'm trying to figure out the best tooling/process to create images that support both types of CPU.

Currently I'm able to build OK with Docker Desktop creating amd64 and arm64 images in buildx, then using manifest to create the multi-arch image combining the 2. So I feel pretty good about using that in our AWS CodePipeline builds.

But of course it's not that simple: We're supposed to move away from that towards Podman or equivalent. That part is currently puzzling me, because the commands that work with docker buildx aren't working in Podman.

I guess I'm not looking for specific help on that issue, but more of a general overview of how people are tackling similar workflows.

I've had good success running w/ https://github.com/abiosoft/colima as an alternative to Docker Desktop.

I'm on an M1 mac, and it's nice to be able to debug both arm64/x86_64 in the same toolchain (via QEMU virtualization).

The simple solution would be to get an ARM MacBook or Mac mini and debug your builds on hardware. If this is a business then that's a tiny expense vs the collective time wasted by other devs dealing with crashing.

For a more modern iteration of that idea, check out https://github.com/nlewo/nix2container

It has all the benefits of OP but produces the images much faster, and with less on-disk usage.

related to Nixery: Improved Layering Design

https://storage.googleapis.com/nixdoc/nixery-layers.html

Reply on news.ycombinator.com