Hacker News Reader

I think they mean different things to different people. From the problems I have faced with customer-controlled OS installations, the biggest thing that they offer is configuration isolation (or rather independence). I have seen some truly crazy shit done by customer's administrators and even crazier shit done by management software that they install. Taking away that autonomy is huge.

Absolutely, but the context of the article was specifically Kubernetes.

In the context of Kubernetes, what do containers have to do with cgroups? I get that container runtimes typically provide a nicer interface for setting them up, but Kubernetes doesn’t use those interfaces anyway. If it was just about cgroups, they could have just run a regular process in cgroups.

Containers are a construct created by employing cgroups + namespaces together. Namespaces provide isolations in terms of filesystem, network, users, processes, and probably more.

Yes, but those are features of Linux that containers use; containers don’t have exclusive use of them. My original point was:

> Containers were a great advance, but much of their complexity falls into the “accidental” category. There’s no fundamental reason that I should need anything except my compiler to produce a universally deployable unit of code.

I stand by that. We might just be arguing semantics about whether that unit of code would be called a “container” or not. I’m leaving open the possibility that it would be (say) a WebAssembly module or static binary.

A container can also give your application assurance that the file system looks a certain way. But you are right that you can get pretty far with bundling dependencies and assets and configuring all nontrivial paths with environment variables.

However, the choice of containers for Kubernetes is a pretty fundamental one. It simplifies distribution of deployables to a single workflow also for programming languages that don't offer monolithic deployables, to legacy application not originally intended for Kubernetes, or that don't follow the 12 factor app principles.

Containers are a temporary solution while we wait for everyone to realize self contained binaries with static linking are the real solution.

That is only a real solution if everything is running on the same operating system in the same environment.

What is the point of Kubernetes at all in your situation.

Everything already is running on the same operating system with Kubernetes where it matters (kernel, runc/crun, etc). Containers are a band aid to wrap an app with additional files/libraries/whatever. In Go for instance, I can include all this stuff at compile time (even conditionally!).

I'd love to see Kubernetes be able to schedule executables directly without using containers by way of systemd nspawn or similar. You could have the "container feel" without the complexity of the tool chain required to build/deploy/run/validate containers.

The tool chain to build/deploy/run/validate is the whole point to containers. It's something that unifies the deployment model of applications, unless you want to commit to a single programming language ecosystem. And not every ecosystem has tools to create a monolithic deployment artifact.

Even if you have monolithic deployment artifacts: by the time you have solved all the use cases that the image distribution workflow cover, you have basically reverse-engineered it. If you have already done that, kudos to you, you indeed don't need that workflow. And you probably will find it very easy to containerise those artifacts should you ever need to.

Sounds pretty much like containers are the solution to containers.

The best formulation I saw is "containers are a solution to the Rube Goldberg Machine development pattern".

Since dependency resolution problems are hard for many popular languages (Node, Python, Ruby), you are not going to enjoy sharing the installed packages between two apps. Same with shared libraries for compiled languages.

Otherwise something like Nix + cgroups-based runner had a chance to suffice.

The executable piece is nice, but not the whole picture. Configuration and, more importantly, isolation of the runtime, are also huge benefits that come with containers.

No, containers give you things that static binaries just don't. How do you specify the maximum allowed memory for a static binary? The ports it opens? The amount of CPU usage? The locations it will access from the host file system?

Also, how will you distribute this static binary? How do you check that the result of a download is the specified version, and the same that others are downloading? How will you specify its name and version?

By the time you have addressed all of these, you will have re-implemented the vast majority of what standard container definitions and registries do.

The orchestrator can still use cgroups for resource constraints and isolation. Or it could use virtualization - it would be an implementation detail. But devs would not have to build a container.

Binary distribution, versioning and checksumming shouldn't need to be coupled to a particular format.

Obviously docker solves a bunch of disparate problems. That's kind of the objection.

At that point it's not an implementation detail anymore and that's kind of the point.

Static binaries are great but they don't solve every problem and they do to have to.

The vast majority of usecases don't need kubernetes, that's fine too.

You don't have to bash kubernetes because you don't specifically need it for your one use case.

There are plenty of other perfectly valid reasons to bash that dogpile.

> At that point it's not an implementation detail anymore and that's kind of the point.

Not clear what is meant by this. Can you elaborate?

> You don't have to bash kubernetes because you don't specifically need it for your one use case.

It is the platform teams at my employer deploy to, and with good reason. I think we do need it for our use case. But there are ways it could be improved, and the ability to deploy a simple Linux binary feels like one.

AppArmor or SELinux?

These don't solve all those problems. Moreover, it is something already employed by container runtimes.

It's what we did in C in the 80s and 90s and all the young hipsters chose to ignore. This industry loves going in circles.

In the same vein, I also heard that the new data exchange hotness is grpc. Reading a little into that thing it sounds very much like the good, old... CORBA. So many wheels reinvented every decade, sigh.

Many of the reasons CORBA failed are the reasons that gRPC is succeeding. CORBA tried to solve the wrong problem at the height of the object orientation fad - it tried to provide a standard mechanism for remote “objects” across languages which have different object semantics. It was a doomed idea from the start.

As for the industry going in circles, it only looks that way if you’re not familiar with the technologies in question. The idea that statically linked binaries are an alternative to containers doesn’t make any sense. In fact, it’s not that uncommon for a container to contain nothing but a statically linked binary. People don’t do that for fun, they do it for the additional features and standardization it provides over a standalone binary.

Distributed objects had nothing to do with object semantics in a PL. They're a lot closer to what we would now refer to as the actor model.

> Distributed objects had nothing to do with object semantics in a PL.

In a way, that was the problem. In practice for CORBA, there was a problematic relationship between its object model and the object models of its host programming languages.

CORBA was supposed to be language independent, but that meant that native language objects and CORBA objects weren't the same thing. There was a semantic gap, analogous to the one between relational databases and object-oriented languages. In the latter case, you can use an ORM-like solution or even just handwritten data access objects to achieve a native language object layer for your data model. With CORBA, that wasn't really feasible - you had to embed CORBA object implementations in your language of choice, and program to that non-native object model. In some ways it's comparable to .NET, but that has the advantage of using a common underlying runtime across all its language implementations. CORBA didn't have that.

You mention the actor model, but what examples are there of successful cross-language actor implementations? The reason for this is exactly what I'm talking about. The practical consequences of a cross-language distributed object model are problematic.

The closest answers I'm aware of share a common runtime - there's Erlang and Elixir both on the Erlang BEAM runtime, and Akka with Java and Scala on the JVM, with variants for C# that I'm not too familiar with. In both cases, the common runtime determines the shared semantics of the object model, so you don't get the same kind of semantic gap.

It's not a coincidence that what's ended up becoming popular are data-only formats like JSON and service-oriented RPC mechanisms like gRPC or even REST - these don't conflict with native language structures in the same way. As I said, many of the reasons CORBA failed are the reasons that gRPC is more successful - the most critical one being that's CORBA's notion of "object" was the wrong level of abstraction for language-independent distributed computing.

Static linking is even worse. Now you traded a blob containing gigabytes of software making it difficult to audit and maintain with a smaller and more opaque binary blob.

Research shows that containers on docker hub, flatpaks etc are full of unmaintained libraries with vulnerabilities in them.

So no, we need to go back to proper dependency management and shared libraries.

After thinking about this for a few minutes, the author might be on the wrong track connecting this with containers.

But I think there's something really there about "environmental linting". I know deep in my bones I need write access to make log files, but I don't know how many times I've debugged systems lacking this permission.

I know the log path won't be known until runtime, I know the port can be specified at runtime, but I think there's a ton of room for improvement around a tool that says - hey, you're making this set of assumptions about your environment, and you should have these health checks or tests or whatever.

I agree with you about this is what containers give, but I think the author is really on to something about the dev tooling and environment warning about what sorts of permissions are needed to, like, work.

Why not both? “Works on my computer” was absolutely a problem that docker advertised itself as a solution for.

> Containers are not used in Kubernetes or other similar orchestrators because of their support for bundling dependencies - that is a bonus at best.

> Instead, they are used because they are a standard way of using cgroups to tightly control what pieces of the system a process has access to

Well, sure, you get both. But the point is that needing a post-build step just to get that is accidental complexity. I'm by no means a Java fan, but the way JVM gives jars as a compile target and lets you run them with some measure of isolation is an example of how things could be.

With Maven, Gradle etc you can output a Docker container from a single package command.

And I don't understand this idea of using code bundles as the deployment artefact.

They don't specify how the code should be run e.g. Java version, set of SSL certificates in your keystore etc.

Do you really want Kubernetes to have hundreds of options for each language ? Or is it better to just leave that up to the user.

I think the suggestion is to use native Linux binaries with static linking of any libraries and resources in the executable.

Java programs would need to be compiled as a standard ELF. I think GraalVM can do this.

Only some Java/Scala programs can be compiled as a single binary.

It's been a concept that has been around for years but not progressing to the point where it comes close to negating the need for the JVM in a Production environment.

In the meantime containers works today and is significantly more powerful and flexible.

The fallback scenario just embeds the interpreter inside of the binary. That still nets you a single binary, unless I'm missing something?

I just don't follow the argument. You can also use Java tooling to wrap up a container image. Why is it accidental complexity that we settled on a language agnostic target that also wraps up a lot of the process isolation metadata?

Java is an outlier here. For most languages the process of dockerizing a codebase involves learning a separate tool (Docker). As someone who knows Docker, I get the temptation to say "who cares, just write a few lines of Dockerfile", but after talking to a bunch of developers about our container-based tool, having to leave their toolchain to build an image is a bigger sticking point than you might think.

But this is begging the question. If the work was more integrated with the compiler, it would still need to be learned. If your compiler of choice spit out an deployable unit akin to an image you'd still need something akin to the Dockerfile to specify exposed ports and volumes and such, no?

The Dockerfile doesn't specify volumes, those are configured at runtime. In the case of Kubernetes, they're configured in the pod spec.

As for ports, EXPOSE in a Dockerfile is basically just documentation/metadata. In practice ports are exposed at runtime (if using docker) or by configuring services (if using Kubernetes), and these are unaffected by whether a port is exposed in the dockerfile.

IMHO this is how it should be -- if I'm running a containerized database or web server, I want to be able to specify the port that the rest of the world sees it as, I don't want the container creator to decide that.

while I admit I'm being a little loose in what would translate where in this hypothetical, none of it seems so easily cut out.

You're right that 'containers' are essentially a higher-level pattern but the main underlying building block is arguably kernel namespaces, not just cgroups.