Hacker News Reader

Comment by clhodapp | Hacker News Reader

It is sad how much waste will occur because OCI used Docker's flat layer list instead of appc's dependency graph. This will lead to a tremendous amount of unnecessary network traffic and disk usage globally. I think this is a Betamax vs VHS moment on a much smaller scale.

View on HN

There is no real advantage to a dependency graph versus a list of layers. Originally, Docker images were actually fetched in sequence, similar to the ACI approach, but this blocked parallelization on each subsequent step. What has become OCI uses a list of layers to avoid this bottleneck.

As long as images share layers, network traffic would be the same in either scenario.

Reduced transfers will be achieved by formats that have a more granular filesystem representation. Right now, we are really held back by the continued use of tar, but that is another problem.

Yes, I agree that the method that OCI v1.0 & Docker v2.2 has of telling you ahead of time "you must fetch all of these objects and they will be this size" in a compact single metadata only format is the best option. Then it is up to the fetching code to choose the best way to pipeline or parallelize.

I think that lack of parallelism has almost nothing to do with graphs vs lists. It would be pretty easy to either store the entire dependency graph's fetching-critical metadata in each image or to make it possible to fetch the metadata separately from the core data. Both of those would allow you to parallelize downloading of image data in the graph-based system.

Having more granular layers can help a bit but in practice it hits its limits very quickly because higher layers will necessarily get swept up and have to be rebuilt when layers below them that they don't actually depend on get rebuilt. This rules out having a core image and a set of modules that each downstream image may or may not need (to be mixed in or not per-image). Thus, when you use the layer-based system in practice you usually end up with very little layer sharing between images, outside of the core Linux distro layers.

Note that I did not say "more granular layers" but more "more granular filesystem representation". This is a key point in achieving better data sharing. In fact, these use cases are not ruled out, but bolstered by OCIs approach. The introduction of alternative systems will be much easier to do when OCI is broadly implemented.

As far as metadata is concerned, your suggestion is exactly what OCI does. The problem with ACI is that it embeds a large part of the metadata into a tar file, which has to be fetched in its entirety. OCI is mostly metadata scaffolding, made up of indexes, manifests and configs that can all be fetched without large bandwidth requirements.

The compositional aspect that you've brought up has been explored and it doesn't make a whole lot of sense to cram that into images. Typically, such a system requires composing container filesystems through named references, allowing components to be independently rebuilt. Because this composition often relies on details of the target deployment media (orchestration system, container runtime, specific operating system, etc.), especially in how it deals with the security of name resolution, baking it into an image format leads to massive inflexibility.

Moving it up the stack works much better and avoid the technical complexities that come with doing it within images. We can see this image composition in action in k8s PODs, docker stacks and other systems. Such constructions can be distributed through OCI images, through media types, but they are based on the compositional capabilities of the target system.

Images don't need to be stored in this format on disk, the format is for transport/interchange.

But out of curiosity, why do you say that a dependency graph would be better here?

In terms of on disk storage and transport of layers, the layers are already content addressed.

Not the OP so I'm just speculating, but being able to "mount" content-addressable blobs into different locations in the resulting rootfs would enable more incrementality and sharing of blobs.

With a flat list, the same logical set of files will have a different checksum depending on where in the final rootfs it needs to end up, right?

These blobs are already shared. (In Docker) Since each layer is only the diff from the layer before it, the content address is only from the change set.

Technically you can share blobs even between an image that is `FROM centos` and one that is `FROM ubuntu` as long as the layer hashes are the same.

If you have a hierarchical dependency chain, and addressing content based on the content that comes before it in the chain, you can only share blobs between images if they have a common ancestry.

Here's the example I'm thinking of:

One image has a large file at a specific path, added via something like: "COPY largefile /etc/path1".

Another image wants to add this same file to a different location:

"COPY largefile /etc/path2/".

My understanding is that these two blobs will have a different hash and can't be shared because the changeset includes both the file itself, and the destination location in the image.

Is that correct?

Yes, however I'm not sure this would be shared in any format right now.

If we distribute OCI images via IPFS, problem solved.

Reply on news.ycombinator.com