Hacker News Reader

Comment by vishvananda | Hacker News Reader

Yeah I'm aware of the build mirror, but crashcart builds things under a different prefix so it must build things from source each time. I realize this doesn't affect most users (which is why it takes a while for disappearing sources to be found and fixed). A content addressed mirror for sources as well would solve the problem nicely.

View on HN

I'm sorry, the mirror is actually for sources, not build artifacts. I've updated my comment to clarify.

When was that added? Well fetchurl automatically poke it if it can't find the sources otherwise?

Yes, fetchurl will try it:

https://github.com/NixOS/nixpkgs/blob/733151b7825ee8840f7437...

Edit: It was apparently added in 2007:

https://nixos.org/releases/nixpkgs/nixpkgs-0.11/release-note...

interesting I wasn't aware of that. I wonder why my builds are not using it.

It might be related to using a different Nix prefix for your builds, which is a little poorly supported. Just curious: Why are you using a different prefix?

The point of crashcart is to be able to side load a filesystem with utilities into a running container. It is very important that the location we pick doesn't conflict with any path already in the container. If we used /nix as the mount path it would conflict with any container that uses nix. In order to prevent this (probably rare) conflict, we build our utilities in /dev/crashcart/ instead.

Hmmmm, interesting... That does seem like a pretty good reason, though maybe you could bind mount over /nix? That should be fine from a Nix perspective, not sure how well it works with container technologies.

we could bind mount over /nix or /nix/store, but that means any existing nix packages from the container would not be available. The whole point is to have the whole container file system available along with the utilities. We could alternatively find each package that we need for our debugging utilities and bind mount each directory from the store individually. This would work due to unique paths in the store, but that means potentially hundreds of bind mounts and is an orchestration nightmare.

Sorry, I didn't mean bind mount, I mean union mount, like with OverlayFS or whatever the most-used one is.

we didn't look at using overlay. Might be possible, although that would introduce a dependency on kernel version and/or module. A custom fuse might be an option here as well but fuse in containers is a bit sketchy at the moment.

Try Nix 2.0, it can place your derivations in custom folder by clever use of chroot, read about --store argument.

This looks like a neat feature. Reading up on it briefly, it looks like it changes the mount namespace / chroots when running the binary in order to accomplish this. Unfortunately that defeats the purpose of crashcart, since the whole idea is to have access to the container file system while running your sideloaded binaries.

Unless there is some magic that I'm missing, I don't think this option helps for our use case.

Ok. I will check it out. Thanks.

Reply on news.ycombinator.com