I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds.
If there is some additional isolation required, just run the container in a VM.
If there is some additional isolation required, just run the container in a VM.
For anyone else wondering how heavy this is on a MacOS, I ran the install script and it just delegated to brew... brew listed the following packages being installed:
==> Fetching dependencies for unikraft/cli/kraftkit: aarch64-elf-binutils, gmp, mpfr, aarch64-elf-gcc, coreutils, gettext, readline, gawk, gnu-sed, pcre2, grep, make, capstone, dtc, mpdecimal, ca-certificates, openssl@3, sqlite, python@3.12, glib, libunistring, libidn2, p11-kit, libnghttp2, unbound, gnutls, jpeg-turbo, libslirp, libssh, libusb, ncurses, snappy, vde, qemu, socat, wget, x86_64-elf-binutils and x86_64-elf-gcc
Took a few minutes to finish installing... the CLI seems to be based on the Docker commands (build, clean, run, 'net create', inspect etc.), some package-manager like commands ('pkg info', 'pkg pull', 'pkg list' etc.), a bunch of "cloud" commands (I suppose that's the non-free part) and "compose" commands just like docker-compose. Interesting stuff.
Note for the parent commenter: the Lua link in the landing page is broken: https://github.com/unikraft/catalog/tree/main/examples/http-...
I tried to run the C hello world example... I get an error, it wants to run Docker?!?! I thought the whole point was to avoid Docker (and containers)??
Here's the log:
i creating ephemeral buildkit container W could not connect to BuildKit client '' is BuildKit running? W W By default, KraftKit will look for a native install which W is located at /run/buildkit/buildkit.sock. Alternatively, you W can run BuildKit in a container (recommended for macOS users) W which you can do by running: W W docker run --rm -d --name buildkit --privileged moby/buildkit:latest W export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkit W W For more usage instructions visit: https://unikraft.org/buildkit W E creating buildkit container: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?: failed to create container
PS. running the hello-world pre-built "image" worked:
> kraft run unikraft.org/helloworld:latest
EDIT:
A lot of stuff looks broken on MacOS.
For example, `kraft menu` doesn't work (error "no choices provided", even though the docs show it working fine without "choices"?)...
`kraft run --elfloader loaders.unikraft.org/strace:latest ./my_binary` also doesn't work (the docs show it working). Error: "unknown flag: --elfloader".
Seems like the product is still in alpha?!
i don't think devs care if they use containers or VMs, as long as it's easy and they don't have to worry about which version of Python the host is running
The killer app of Docker isn't the container, it's the depth and uniformity of the UX surrounding the container system. When that is broken by something on the host (non x86 cpu was a major pain for a while before popular images were x-built) and emulation gets in the way and is not as easy, or just mildly different (windows behind corporate firewalls that assign ips used by the docker engine for example), the ease of use falls away for non-power users and it's all painful again.
Tech like Docker for windows and Rancher Desktop and lima has largely matured at this point, but somebody could make a new machine and then the process of gradual improvement starts all over again.
Docker? One line possibly.
VM? Afternoon (unless I want to write ansible too)
For Docker, OrbStack does the same thing as Docker Desktop, Podman Desktop, Rancher Desktop, etc., which is set up a Linux VM running Docker and then present a native socket interface on macOS which relays everything it receives to the Docker socket inside the VM.
macOS doesn't have native capabilities for running containers, which is why the nearest thing you can get to containerd on it requires you to disable SIP so it can use a custom filesystem to emulate bind mounts/null mounts: https://darwin-containers.github.io/
If you read the PRs where the principal author of the Darwin Containers implementation is trying to upstream bits of his work, you'll see containerd comparing his approaches to others and complimenting them by calling them 'the most containerish' because real capabilities aren't there.
(I believe I've read rumors here on HN that Apple has those features internally, fwiw. But they've evidently never released them in a public copy of macOS.)
Another clue in all this is to just run uname in any of your Docker containers in OrbStack; you'll see they're Linux machines. Some operating systems have Linux syscall emulation layers (WSL1, FreeBSD's Linux emulation, Illumos' LX Zones) that could perhaps be used to run Linux containers without hardware emulation or paravirtualization in combination with some native containerization capabilities. Afaik Illumos' LX Zones is the only implementation where that's a supported, intended use case but maybe FreeBSD can do it. At any rate, macOS has never had that kind of syscall compatibility layer for Linux, either. So when you run `uname` in a 'macOS container' and see 'Linux', you can be certain that there's a VM in that stack.
PS: Aside from the fact that it's proprietary, I really do quite like OrbStack. It's the nicest-to-use implementation of something like this that I've tried, including WSL2 and Lima. The fact that it makes the VM machinery so invisible is very much to its credit from a UX perspective!
With VMs? You now need a way to get data from within the VM which in most lightweight VM implementations just not possible.
Once again: Containers are not about isolation or security, they are a package format for shipping applications. The packages are easy to build, distribute, multiarch, ...
And requiring a Linux-VM on macOS to run Linux containers, is not particularly surprising.
Take it up with Apple.
This paper argues that when you build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against necessary bits of code to execute _as_ a VM, then you'll get better performance than a container and you'll get that isolation.
This is in fact true based on performance studies, the follow up paper to this shows so: https://arxiv.org/pdf/2104.12721
(Disclosure, co-author of the linked paper.)
We ended up taking this to real workloads if you want to see it in action: https://unikraft.io/
Using the deploy command line tool is the Docker file used to determine dependencies for the hosted VM? What if a developer is using an unusual programming language, like Common Lisp. Is that doable?
It would be nice, but this is really hard to do when modern software has so many layers of crud. Good luck getting say, a PyTorch app, to work doing this without some serious time investment.
The flip side is that if you want something like low-level access to your specific graphics card you may need to implement a lot of additional support. But of course nothing says you have to use this everywhere at the exclusion of everything else. There's plenty of systems in the world that from the kernel point of view are basically "I need TCP" and a whole bunch of compute and nothing else terribly special.
The point of the poster was pretty clear:
“The main benefit is not isolation and security”
It is very useful for managing dependency hell, or at least moving it into "API dependencies" not "Library dependencies", it is handy for pickling a CI/CD release engineering infrastructure.
It's not a security boundary.
(I'm 100% agreeing with parent, in case I sound contentious)
It is a security boundary, just not necessarily the best one.
The snobbery deployed in this "containers vs. VMs" argument really gets out of hand sometimes. Especially since it's almost never deployed symmetrically. Would you make the same argument against using a BSD jail? Do you refuse to run your services in a separate UID because it's not as secure as a container (or jail, or VM)? Of course not. Pick the tools that match the problem, don't be a zealot.
X86 protected mode, processor rings, user isolation in the multi user operating systems, secure execution environments in X86 and ARM ISAs, kernel and userspace isolation, etc. are purpose built security boundaries.
Virtualization is actually built to allow better utilization of servers, which is built as a "nested protected mode", but had great overhead in the beginning, which has been reduced over generations. Containers are just BSD jails, ported to Linux. This doesn't make containers bad, however. They're a cool tech, but held very wrong in some cases because of laziness.
My comment was more of a soft jab against using containers as the ultimate "thing" for anything and everything. I prefer to use them as "statically linked binaries" for short lived processes (like document building, etc.).
But, whenever someone abuses containers (like adding an HTTPs fronting container in front of anything which can handle HTTPS on its own) I'm displeased.
Relevant XKCD: https://xkcd.com/1988/
"Reproducible" is usually defined as "identical output except for the cryptographic signature at the end" (and that should be the only use for a certificate in your build environment, a high-quality build environment should be self-contained and have no network access). That is, once you remove the signature, the built artifacts should be bit-by-bit identical.
If you run a docker build multiple times, and copy a few files into the container, you get a reproducible container image. It is not a hash perfect duplicate, but functionally equivalent.
If builds of your favourite programming language are reproducible or not, is not really related to VM vs. Container.
- https://github.com/firecracker-microvm/firecracker-container...
- https://github.com/kuasar-io/kuasar
- https://github.com/kata-containers/kata-containers
- https://github.com/QuarkContainer/Quark
The isolation could be achieved using namespaces directly. But the API, tooling and registry add a lot of value that would otherwise require a lot of development.
Also last time I looked hypervisors aren't possible on all cloud vendors, unless you have a bare metal server. This matters in my case. Maybe it has changed in the past 3 years.
When docker fits it's great. Same can be said of k8s, where there are a whole bunch of additional benefits.
Swings and roundabouts.
Anyway containers go quite a bit further than just static linking, most people aren't out there linking all the binaries that their shell script uses together?
It's not possible to package half a Linux distribution into a single binary. That's why we have containers.
Personally, I use containers to quickly spin-up different database servers for development or as an easy way of deployment to a cloud service...
That said, it makes sense from a developer POV; if, during development, you don't need the isolation you can run multiple containers (with on paper fast boot times and minimal overhead) on your development box.
There's plenty of cases to imagine where you need the containerization but not necessarily the isolation.
so people use containers instead (even if the container literally only contains a single statically linked binary)
the articles main point still remains, containers are a slow and bloated answer to this problem.
I concede youll need containers for Kubernetes, and Kubernetes on the surface is a very good idea, but this level of infrastructure automation exists already in things like foreman and openstack. designs like shift-on-stack trade simplicity of traditional hardware for ever byzantine levels of brittle versioned complexity...so ultimately instead of fixing the problem we invoke the god of immutability, destroy and rebuild, and hope the problem fixes itself somehow...its really quite comical.
baremetal rust/python/go with good architecture and CI will absolutely crush container workloads in a fraction of disk, CPU, RAM, and personal frustration.
I really don't understand why people do this: I get having a distaste for containers but some people, seeing the massive success of OCI images, mainly seem content on trying to figure out how to discredit its popularity, rather than trying to understand why it's popular. The former may be good for contrarian Internet forums, but the latter is more practically useful and interesting.
I say this with some level of understanding as I also have a distaste for containers and Docker is not my preferred way to do "hermetic" or "reproducible" (I am a huge Nix proponent.) I want to get past the "actually it was clearly useless from the start" because it wasn't...
Check out https://github.com/jrz/container-shell
But reproducible infrastructure as code is just orthogonal to that: everything from Salt to Nix is credible in that role.
Containerizing on top of a Xen hypervisor never made sense to me.
As someone who made a living doing this at scale, where we would build a new datacenter every 2-4 weeks using 100% open source or off the shelf tools, I completely disagree.
I think PXE booting some servers and running a binary on them is 90% easier than most container orchestration engines, Kubernetes control plane, and all the other problems engineers seem to have invented for themselves. I also think it’s almost always much more performant. Engineers don’t have an intuition to realize that their XXLarge-SuperDuper instance is actually a 5 year old Xeon they’re sharing with 4 other customers. Cloud Prociders obfuscate this as much as possible, and charge a King’s ransom if you want modern, dedicated hardware.
No one will ever be considered for a Nobel over Kubernetes.
It's not a technical matter, it's a human, economical matter and actually... Most people are poor, following the largest scale means following poverty not a good thing.
The primary benefit is resource usage and orchestration.
Rather than duplicating entire aspects of an OS stack (which might ne considered wasteful) they allow for workloads to share aspects of the system they run on while maintaining a kind of logical isolation.
This allows for more densely packed workloads and more effective use of resources. This is a reason why the tech was developed and pushed by google and adopted by hyperscalers.
No. Running a container in a VM gets you no additional isolation. Containers share kernel space and as such have limited isolation to VMs, which have isolated kennels. In exchange for this Lack of additional isolation, you’ve added a Bunch of extra Complexity.
Pardon the extra caps I am using iOS voice dictation.
This is similar to how managed container IaaS works. They launch a VM and run your container in it.
It is extra complexity but has a few advantages. 1. People already have a convenient workflow for building container images. 2. The base OS can manage hardware, networking and whatever other low-level needs so that the container doesn't need to have these configurations. 3. If you want to trade of isolation for efficiency you can do this. For example running two instances of a container in the same VM. The container doesn't need any changes to support this setup.
This is the approach we take at kraft.cloud (based on the LF Unikraft project): use Dockerfiles to specify app/filesystem, and at deploy automatically convert to a lightweight VM (unikernel) without the container runtime/layer.
No, you don’t. There is no benefit the container is providing, because The only feature of the container is isolating you from the zero other containers running on the VM.
Using two levels of userland isolation makes about the same sense as using 457 levels of userland isolation.