Docker: Not Even a Linker

Tech is strangely ahistorical. Not just practitioners not reading the literature, but it seemingly being forgotten entirely. Possibly this is a side effect of so many of us being self-taught.

In another thread I've just been arguing with someone who thought that the DOOM code should have been thread-safe.

> it is sometimes hard to me to see how new generations fail to find such information, even though it is available on the web

As someone who has attempted to go back and find old articles and concepts for which I remember only the generalities, I understand this completely. The problem, IMHO, is that search engines put too much emphasis on the publication date of an article - you'll get recent musings on the topic on the first ten pages, and then if you're lucky you'll get the original material on the eleventh.

However, you're frequently unlucky in this regard, since the search engines will devolve into poor-yet-recent matches before looking further back in time.

Ironically, the internet is forgetting, because we've too finely tuned the algorithms to value newness and community guesses over authority.

> I guess the main cause is thatbone needs to know what to search for.

It is common misbelief (particularly among yang people), that all in IT changing so fast, that any technology created more than 20-30 years ago is outdated and don't worth to study. The wheel reinvented many times.

So I've recently reimplemented some of our applications service dependencies in docker for the main reasons of mitigating dependency issues and allowing for for developer local consistency.

I fully admit this is deferring solving these issues fully (solr 1.4.1 really?) but it has allowed for minimizing the issues the existing backlog was causing us. Arguably its also brought more consistency to our environments vs the chef/puppet/hand rolled management tools. We could have used these tools with a different process to achieve similar results though.

Maybe you old dogs should've barked a bit louder because the way we used to manage systems and system environments before docker was plain ridiculous. No one ever told me to use containers, I don't even know of any pre-docker technology that allows me to do easily what docker allows me to.

We used to do Chef scripts that are fed json configuration files that render configuration template files that installed loads of software into a completely bloated environment.

Now we just have a simple bash script that provisions the machine, which means it sets up the network, firewall, sets up docker some monitoring, a configuration daemon (etcd) and then runs the orchestrator (in a docker container of course).

And then from that point on, all operations are in the container world. You build a new production container, push it to the private registry, and tell the orchestrator to redeploy a service.

Was that workflow even possible before docker? Using FOSS? I get angry just thinking about how we used to do it.

We have dependency management built into the package managers which hides that from us these days. Unix and Linux before package managers was kind of a pain. Now, I will totally give you that it was nothing like the "DLL hell" of Windows.

What people mean is Python (and others scripting languages) dependency hell. Unix dynamic linking very rarely has any issues as static typing makes it relatively easy to reason about stable interfaces. The raise of containers, is consequence of raise of dynamic scripting languages, and rather un-unixy and "get it ready ASAP" culture, IMO.

You can have 100 daemons written in C/C++ running on one system just fine, and that's how Unix systems used to work, but as soon as you start using software written in Python, you can't get two running system-wide, without recompiling every dependency into separate `virtualenv`. So it's easier to just throw them each in their own lightweight virtual machine, based on the recipe on how to rebuild the thing.

Nix and Guix are purely functional package managers, meaning that software builds are treated like a mathematical function: Input the same source code + dependencies and receive the same build as output. They have features such as reproducibile (often bit identical) builds, transactional package upgrades and rollbacks, and unprivileged package management. They solve the dynamic linking problem by allowing each package to refer precisely to the dependencies that it was built with. With this mechanism in place, it becomes very easy to use applications that require different versions of some C library, or a different Ruby/Python interpreter, or whatever else. Furthermore, it can do this without relying on a specific type of file system, and without requiring that applications be run inside containers or virtual machines. This makes it very composable and general-purpose.

https://nixos.org/

http://www.gnu.org/software/guix/

What it doesn't do: handling cpu quota on per "stack" basis, no builtin security isolation. That said, both use container technology for that.

By solving the issue at a layer below (instead of adding one like docker does) it makes things much cleaner, more powerful making obselete puppet and the like. FWIW describing a containers/vm's/os'es in guix is much more easy than using docker.

Have a look at https://github.com/NixOS/nixops too.

Thank you very much

To a certain degree I don't want to know what's in them. If I want to add search to my stack - initially I'd rather not have to have an intimate knowledge of Elastic Search, a task queue and whatever other moving parts there are. In many cases a black box that just works would be a fantastic option.

The reason hosted services are popular is for exactly this reason.

A wide understanding of different technologies is a wonderful thing but sometimes you just need to ship.

Yeah, having to rebuild every app that uses OpenSSL when a new advisory is issued... wow, that would be expensive!

There is also Gobolinux, that do so in a less stringent fashion.

This in that, as i understand it, compiling a lib with a feature enabled or disabled will produce different tree branches in NixOS/Guix, while Gobolinux will happily replace one compile with another if it is of the same lib version number.

So you can't have lib foo 1.0 with feature X enabled in one compile sitting side by side with the same foo 1.0 with feature X disabled when using Gobolinux.

Then again, Gobolinux has never been meant as a server distro...

Edit: thinking about it, Gobolinux could perhaps be extended with checksum dirs inside the version-subdir to separate different compiles. Not sure if this has been considered or attempted.