FreeBSD is growing more and more on me, mostly because of Jails ( also ZFS ).
Call me a boomer, but I still think "containers"/Docker is not a solid, simple and robust solution for isolation.
Call me a boomer, but I still think "containers"/Docker is not a solid, simple and robust solution for isolation.
Nix is a great way to jump over docker entirely. You can then output a container from Nix if you need it, but you aren't limited by them at all.
Can you explain why? It's not something I know a lot about so I'd be interested to learn about
It's similar under Linux's CGroups. You fire the binary inside another CGroup and, that application thinks that it's the only thing on the machine, and may feel a little lonely, with its own IP and isolated network stack and such.
I can download and run anything in any accessible OCI registry with a single `docker run` command. The only thing I've seen that makes jails even remotely close to being as easy and quick to use... is using runj to recreate the same experience.
> It's similar under Linux's CGroups. You fire the binary inside another CGroup and, that application thinks that it's the only thing on the machine, and may feel a little lonely, with its own IP and isolated network stack and such.
Yeah, I know how the isolation features work, my point is that they're not nearly as important. For the vast majority of docker use, you could replace the actual container with chroot and not really make any difference, and the chroot is only to save the trouble of having to modify the application to handle host paths, not as a security barrier. Docker's killer feature isn't containers, it's images that are easy to move around and run, and to slightly lesser degree Dockerfiles that provided a standardized format to describe how to build those images.
I love FreeBSD, but I think this kind of illustrates how the community has really dropped the ball on containers: Nobody cares that jails are technically superior and far more elegant under the hood, or that they've been here longer than on linux, or whether their security position is better. Docker was and is overwhelmingly better UX; all the under the hood stuff is effectively irrelevant to users.
Many of us out there compile their apps, stash into a cgroup or a jail and fire it. Just because mainstream developers using "that one thing", it doesn't mean that the whole world has abandoned other ways to do things.
I personally doesn't pull any big software package from an OCI registry, because I don't trust the installation and don't like the reduced configurability of the packages involved. If I'm lazy for that day, I pull Debian minimal, build the thing I need, push to our local registry and call it a day.
Otherwise I install that thing properly, run it under cgroups, or under its own VM if the workload warrants it, and get a mug of coffee.
> Nobody cares that jails are technically superior and far more elegant under the hood...
Believe me, some shops care.
"Hitler uses Docker":
> If I'm lazy for that day, I pull Debian minimal, build the thing I need, push to our local registry and call it a day.
I'm 100% sympathetic to not using public images, but it really sounds like you've just re-invented docker with extra steps. Or in the second case, like you are using docker just without the public images?
In the second case, the only thing I pull is "Debian Slim" from the public repo. Since I'm using Debian for a very long time, I can actually verify down to last bit (package/binary checksums, etc.). Then built the rest from the source + Debian repositories inside a Dockerfile.
I push the container to local registry, and Dockerfile to our local Git server.
There has not been a lot of documentation on how to do service jails. Maybe because a full jail (with a full and reasonably sized OS) is so trivial to setup.
But there is no denying that Docker have had great marketing and a good use case.
I do however think the real reason no matter how much of a FreeBSD Jail or Solaris Zones fan boy you are - you know that Linux is the elephant in the room. That brand name alone without arguing distributions.
Linux already had a huge market and mindshare compared to FreeBSD or Solaris. And when Linux got CGroups you had (within reason) feature parity. For anyone who are not daily distro switchers changing your base OS is no minor feat. So even if CGroups was not perfect the incentive to switch is not huge.
But it does make me sad that Docker has become so prevalent that a Docker image is the only way some projects make releases. All too commonly without documenting the build steps. Do not be so hard on your posix friends, please!
That's the great thing with Docker, all you need is documented in the Dockerfile.
Rather than using plain language and reason about dependencies, limitations, workarounds and more or less informed choices you need to infer this from the Docker file. Pray that they have left even a single little comment regarding non obvious issues.
Akin to saying everything is documented in the Makefile. Why not a quick glance a main() to see if we parse any args.
You can give people step by step instructions on where to go. But you empower them with a map.
Documentation is hard. Quality documentation even harder. A dockerfile is a very poor substitution for me.
I do not mind projects who prefer Docker and they get all my love and appreciation if they document the manual building steps. And I am totally fine by them telling me I should probably not do that. But I am seeing more projects skipping this and instead spend effort and time on using and debugging with Docker.
But I am old and my beard is getting grey. I have learned to fear the programmer who tells me to read the code to understand the system. I have been told that it is concise and is good for velocity. Not everything old is good and we should move on. I for one miss the days when documentation was considered a priority.
Docker, as you know better than me, is just a wrapper over CGroups, nothing else, plus a container image format, basically.
In the end, CGroups provided what jails provided with the same and better performance, and someone leveraged that subsystem with a tool, before everyone else.
I have no qualms with neither Linux nor CGroups. Great technology in itself. No reason to spread FUD.
Well, OK.
> No performance impact running inside the jail as it is basically just a jail id managed by the kernel.
Thanks for letting me know.
> No reason to spread FUD.
I was just relaying what our system and BSD guy said back in the day. I have nothing against BSD. That quote was before hardware virtualization days, BTW.
x86 virtualization came along in the 1990's. VMware Desktop everybody knows today was released around 1999. Other players have been around.
This is the same year jails was introduced with FreeBSD 4.
The funny thing is that jails are often referenced as "virtualization without the overhead".
Jails are really fast as it is the same shared kernel. Hardware VMs have additional security advantage as a guest kernel vulnerability will (normally) not affect ofter guests. But it comes at a cost.
However, I remarked that the quote is dated before hardware virtualization, namely VT-x/AMD-V, VT-d, etc., hence hardware segmentation of processors and I/O devices were not possible on that era's x86 hardware.
I have no qualms against jails and/or BSD. I'm not a flamewar person. As I said, this was a quote from our System/BSD guy, nothing else.
Maybe the hardware was saturated, maybe that kernel had a regression, maybe without hardware virtualization/offloading there was some kind of overhead in these days, I don't know.
All in all, they are nice technologies, and I like/support them all.
1. The ability to create services in a clean slate environment. Unix processes tend to inherit by default most environment attributes--working directory, open file descriptors, etc.--and having a way to create a process (or group of processes) without inheriting all of that stuff is easy.
2. Virtualization/isolation in a lighter weight solution than a full virtual machine.
3. Checkpointing that allows you to consistently restart an image with a given configuration, even if you go on to horribly destroy it in tests or whatnot.
4. A language for designing how to construct these images (the Dockerfile).
I suspect the popularity of Docker mostly comes from the latter, and I'd definitely like to see something like that but utilizing jails and ZFS features instead.
Docker minus the giant well-populated well-maintained image library would be almost useless to me.
[edit] one daemon improved immeasurably by Docker is Samba, of all things. The invocations are a little arcane, but once you’ve got them figured out it’s one extra option line to add a user, one line to add a share, repeat as needed. So very much better than relying on distro-magic to make it work, or, god forbid, trying to configure it manually with some config file that ends up inexplicably having no portability, or is silently ignored despite the claims of the docs, or doesn’t work at all because there’s some option commented out by default that definitely shouldn’t be. Docker forced them to finally make the “I just want to share a damn directory, to these users, either read only or read-write” use case, which is probably the vast majority of use of Samba, straightforward, concise, and reliable to configure.
I’m not a boomer, but I’m considered old in this industry.
You can do this with ptrace() which is in POSIX, but its going to be slow for i/o heavy applications. Unfortunately POSIX doesn't provide anything else (seccomp would already be a step up). See also proot[1], gVisor[2] and User Mode Linux[3] which all use ptrace().
[1] https://github.com/termux/proot
[2] https://gvisor.dev/docs/architecture_guide/platforms/#implem...
[3] https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linu...
What? Did you think the P stands for POSIX?
Uh, I guess time to do binary translation ala qemu-user. Webassembly anyone?
This much simpler solution could be replicated in basically any decent OS and wouldn't require shipping an entire Linux kernel and deal with slow IO and the network hoops to jump through on anything that isn't Linux.
The same directory of ready-made solutions that Docker enjoys today could be rebuilt uppon this simpler stack.
When you realize that linking to the right libraries gets you most of the way there already, you begin to think there's a gem of a solution hidden in plain sight among all this mess.