Squashfs binary format | Hacker News Reader

Squashfs binary format

dr-emann.github.io

51 pointsst_goliath12 comments

I wrote undocker[1], which converts a docker "container" to a rootfs tarball. I really wish docker container format were documented as well as squashfs here. Surprising, given Docker Inc should have had enough resources to make their most important format well documented. :)

Anyway, undocker requires extracting the archive before using it. Squashfs wouldn't. Perhaps I will write a docker2squash, so we can mount and run these images immediately after download.

[1]: https://git.sr.ht/~motiejus/undocker

> Perhaps I will write a docker2squash, so we can mount and run these images immediately after download.

Well, if you already have a rootfs tarball, I guess the bulk of the work is done. The tarball can be directly piped into tar2sqfs[1] or sqfstar[2].

[1] https://manpages.debian.org/unstable/squashfs-tools-ng/tar2s...

[2] https://manpages.debian.org/testing/squashfs-tools/sqfstar.1...

Nice, thanks! I won't. :)

> I wrote undocker[1], which converts a docker "container" to a rootfs tarball

I thought a "container" (or, more precisely, a "container image"; or even more precisely, a "container image layer") is a tarball?

I've been creating multi-layer container images using `tar` (plus `sha256sum` and `jq` for the corresponding JSON "manifests"), and it works fine e.g. on AWS ECS.

Can you detail further? - Personally, this seems like insanity so I'm curious what lead you down this path instead of using the more common tools like docker compose or buildah.

> instead of using the more common tools like docker compose or buildah

How are those tools "more common" than GNU tar?

> I'm curious what lead you down this path

I had an executable; I wanted to run it on ECS; so I ran the `tar` command. Pretty simple really :)

A tarball of tarballs, layered using barely documented rules (e.g. you can add a directory, remove it on upper layer, and re-add something else on top).

Constructing them is easier than unpacking. :)

Ah, so you're re-creating the results of overlay-mounting the layers?

I'm definitely ignorant of the exact rules for those; I only use them to split up large images into a "mostly static" layer (e.g. dependencies, which may be large) and a "subject to change" layer (i.e. our code, which will be different for each version)

Plugging this here in case it’s useful to anyone else working with squashfs: I wrote a squashfs reader/writer in Go: https://github.com/distr1/distri/tree/master/internal/squash... (part of the https://distr1.org/ project)

So they are using u32 for their mtime, which is better than i32 because of the y2038 problem...

However that just means they have just another odd 70 years left after 2038, right?

Sure, but it's just for the archive-mod time in second resolution, by 2106 you'll probably have moved to a different filesystem, a newer version of SquashFS, or the ground.

Reply on news.ycombinator.com