Containers as Kernel Objects – Again | Hacker News Reader

Containers as Kernel Objects – Again

lwn.net

58 pointsTomte27 comments

Two sentences from the article struck me:

> Linus Torvalds once famously said that there is no design behind the Linux kernel.

Reminds of a talk/interview where Bryan Cantrill (of Solaris/Illumos/Joyent fame) said: "You can't evolve your way to containers". Sure evolution is nice and can lead to nice results, but I refuse to think that not designing things is a good idea. From my (limited) experience, a properly designed system helps you. When dealing with poor design, you have to fight whatever abstraction the idea evolved to.

> One could perhaps make an argument that the lack of a proper container object was necessary in the early days, when the community was still trying to figure out how containers should work in general.

I don't think that's an argument, really. The phrasing suggests Linux pioneered containers, which is simply untrue. Containers existed before, in the form of FreeBSD jails and Solaris Zones. Linux suffers from sever NIH in this case IMHO. Was the design of these other systems even considered ? Were there limitations that made them impractical ?

> Containers existed before, in the form of FreeBSD jails and Solaris Zones.

Even earlier, depending on what variant of mainframe virtualisation you are prepared to count.

> Were there limitations that made them impractical ?

Not being up close with LKML workings, I can only speculate that it's due to Conway's Law. Having a kernel-level concept of "a container" requires changes cutting across a number of subsystems worked on by very distinct groups of people.

Cathedral-style development can achieve that kind of a change. The organisation responsible for Solaris saw the vision, worked out what had to be changed to create a uniform mechanism, then changed it. They achieved broadly the same with ZFS.

That said, there is container-as-a-runtime, in which I think having a consistent concept a la Zones has enormous value for consistency, security and performance optimisation opportunities. But there's also container-as-ecosystem. More accurately, _images_ are containers, worthy of that name, because they created such a simple way to take advantage of complex low-level APIs.

Cantrill is quite right. There are many examples where the lack of design hurt Linux, and Cantrill will list many of them for you!

E.g., epoll -(what a disaster). Or inotify. Or containers (the subject of TFA). Or kernel debuggers (never mind). Or tracing frameworks (actually, here Linux has finally evolved something similar to DTrace, namely all the ebpftrace stuff). Or cgroups. Or the kernel keyring. Or...

Lack of design is not a virtue, not now.

Sure, design and design review comes with the grave danger of bikeshedding. (I'm using "design" here loosely to mean what we called "architecture" at Sun.) And it has higher initial cost to not designing. But in the long run it pays (it certainly wasn't the cause of Sun's demise).

Some people really mean “the code is the design” when they say “there is no design.” The point being that it’s easy to come up with a vague “design” that falls apart (or comes together, but as an unmaintainable mess) when you try to implement it.

The fact that containers are built from so many different technologies.. always make me uneasy about the security aspects...

This approach has been very successful to date. People forget but there was actually quite a number of different rival container implementations. LXc and let me contain that for you, for example. The lack of a single abstraction made it possible for there to be rival implementations that each evolved rapidly. Now that Docker has “won” it’s time to simplify by collapsing all of the abstractions down to a single set of interfaces.

How different is the actual underlying container abstraction between all those competing container "end-user experiences"?

For the most part, completely identical. They all work by squirting numbers into cgroups.

The exception that proves the rule are the container runtimes that launch the container in a VM.

Isn't this essentially already happening with CRI-O and the OCI?

While I'm not excited by the idea of having a kernel-wide concept of "container", I do love the idea of being able to create a new detached filesystem namespace, mount things into that namespace, and openat and fooat in that namespace.

It already exists in the form of user namespaces.

kinda sick of all this container nonsense tbh - they are so insecure, un-engineered and propped up only by vc marketing dollars

kernel community is wise to reject this, seriously

Why? Containers are not insecure, some implementation of them have some vulnerabilities. That doesn’t mean that namespaces are insecure, it doesn’t mean that cgroups are insecure, and there is more than one container implementation (such as podman and cri-o).

That sounds like a reductionist argument, unless you have some more concrete issues than just “they are so insecure”. And having VC money is not a problem per se.

"propped up only by vc marketing dollars"

This is just not true. They're propped up by Google's marketing dollars.

i'm uneasy about the security aspects of running under a common kernel too, but what else would you use for deployment? in my case I've been using it as "light VMs" for cross platform development - e.g. toolchains, or handing over a container to a customer which builds their embedded Linux system instead of a full VM. It opens a lot of possibilities.

Isn't OpenVZ/Virtuozzo basically the same idea?

A lot of things are basically the same idea, but none of them are backed by "this is the linux way".

Well, both are containers shoehorned into the Linux kernel via a patchset. Are there other examples of that?

Neither OpenVZ, or the idea posted is currently the Linux way.

Seems like a great way to get rootkits into the Linux kernel ...

How so?

On trusting trust

How would you be able to root out these containers doing seemingly anything sitting in kernelspace?

The code wouldn't be running in kernel space, just the setup/configuration of the environment, which really isn't any different than it is today, tbh. This seems to be more about attaching a kernel auditable id to a container.

It sounds like what you are talking about is already existing functionality, not the proposed design.

The proposed design uses all the existing functionality and gives it a single object in the kernel to control it rather than a dozen different ones. It's not a fundamental change in design to how the containers themselves work.

Ahh, well I stand corrected.

why would it be any more prone to getting rootkits in the kernel than any other process that was granted kernel access to do "this thing you think you need to do"?

Reply on news.ycombinator.com