Docker 1.11: The first OCI-compliant runtime, built on containerd | Hacker News Reader

Docker 1.11: The first OCI-compliant runtime, built on containerd

blog.docker.com

153 pointsah3rz61 comments

You know, standardization is a great thing. But, fuck, another major release with huge changes... What ever happened to stable architectures with hardening burn in periods? What new bugs are going to creep in due to these huge changes? How is this going to break our work arounds for previous versions?

It's a great time to be in operations, containers are a huge step forward, but how are we supposed to be confident in a technology with so many major changes every other month? Even Node.js has LTS releases.

And yes. Get off my lawn, too.

If you want something stable I'd stay away from containers and use well-tested technologies like VMs and configuration management.

In my experience many of the tools in the container ecosystem are in the early adopter phase, promising a lot of convenience for some use cases, but too buggy and feature incomplete to really deliver yet. Not to mention the breaking changes and occasional migration to a new tool.

We were using docker up to about a year ago, and ripped it out for just this reason - we didn't have the time available to keep coding the workarounds. Great if you have the time, not so great if you don't.

We were too deep in already unfortunately, but 1.10 introduced a whole new barrage of bugs and regressions, wasting hours and days working around it. If they don't devote some time getting the basics right and getting the core stable, we need to really consider alternatives. I'm already scared what 1.11 will break.

Please make sure to report any issues so they can be fixed.

Stuff like this is what makes me wary of any technology that's been hyped.

I too had a production deploy using Docker and but droped it because of changes like this and some instability.

Started using Ansible (ok, maybe comparing oranges to apples here) and its better for my use case.

Or simply use containers technology that's been around for a decade and is battle tested and stable, like FreeBSD Jails - https://www.freebsd.org/doc/handbook/jails.html

Linux is not the only OS around, and Docker is not the only container technology available, that's important thing to remember.

Thing is, on top of being stable, it should be useful. If you're using containers just for your own code - fine. If you mix and match other repos and ecosystems like we and many others do, good luck with FreeBSD. 60% of the time, interesting repos already have a Dockerfile and if they don't, it's usually pretty easy to write one. With FreeBSD containers, I'd have to do all the wiring myself - if and only if it even compiles and works out of the box on a non-Linux system.

Well, sure, but how is that different from writing workarounds for all these Docker quirks? At least you end up with self-baked solution based on stable technology, instead of self-baked solution for unstable technology. And when it comes to software, its hard to find one that doesnt work on FreeBSD (there's over 24k of precompiled packages available!), not to mention FreeBSD's Linuxulator, that is Linux kernel emulation, built in, so that sentence may be considered spreading a FUD.

You are mostly right but devops people should still play around with the newest tools to form their own opinions.

It's complicated.

I wouldn't stop considering containers per-se (SmartOS zones/FreeBSD jails are fine!) but the whole tooling surrounding the management of Linux containers and corresponding images is still in the "cambrian" phase. Security issues and the general over-selling of the technology (Docker and CoreOS especially are almost too good at marketing their products..) shouldn't discourage you to play around with "those toys" though. Even for certain production scenarios there is a bunch of setups which work fine already (RedHat's OpenShift platform comes to mind).

There's also these:

- https://runc.io/

- https://linuxcontainers.org/

And I'm not sure if they're the same thing, compatible, or what. I'm going to let someone else pick the winner for me. I'll come back to containers in like a year or so and see what the lowdown is.

Plus, there's FreeBSD Jails, Illumos Zones, and SmartOS' support for Linux apps (Dockerized or otherwise). SmartOS is based on Illumos.

I was reading the recent HN discussion + article on the recent network namespaces, and wondered at how much ahead networking on Illumos is.

This is a huge upgrade that is super welcome.

Recently (on 1.9) we have seen quite a few cases where we had "zombie" containers, these are container that can no longer be started or stopped due to cgroup misconfiguration or something along those lines.

The new architecture means that for weird cases like this all we need to do is kill off runc without forcing every container on the box to restart (by restarting the docker service which is what we do now).

It is also really nice that you can now launch apps on runc direct without needing a docker intermediary.

Sounds likely to be https://github.com/docker/docker/issues/18180

If so, it's a bug with a particular (newer) version of AUFS that has been fixed in most distros.

most likely that bug, been tracking it, but I think this goes beyond this specific issue. It makes docker a lot more robust when bugs like this pop up in future.

We hit 18180 in all our images with Java applications. Workaround was to upgrade the kernel.

I particularly like that in addition to refactoring the architecture, they worked on stabilization.

> With the containerd integration comes an impressive cleanup of the Docker codebase and a number of historical bugs being fixed. In general, splitting Docker up into focused independent tools mean more focused maintainers, and ultimately better quality software.

> DNS round robin load balancing: It’s now possible to load balance between containers with Docker’s networking. If you give multiple containers the same alias, Docker’s service discovery will return the addresses of all of the containers for round-robin DNS.

wait wait wait. I'm kind of new into docker world. And so far i've been struggling in understanding how to replicate a container in order to scale. For example i want to run the same Django project 3 time as web1, web2, web3. If i do so now i've to expose 3 different ports, one for each. Plus, to make it working, I need to have a load balancing, thus i've to use HAProxy and do a load balancing to point to each of the container. Anytime I add a new machine (e.g., web4) i've to change the conf of HAProxy and restart it. This brings down the system for a moment. (is this the right approach btw?)

Going back to the cite paragraph. Now i can create many containers and dockers automatically does the routing for me? Am I right? or I misunderstood the meaning of the cited point?

I just need to create all of them with a --alias web ?

> (is this the right approach btw?)

This is why I use hipache for load balancing / routing - it is the only solution I've found where you can change the routing or add new backends in a live system without any downtime. Here is the main problem though: Its load balancing isn't exactly smart, for example it won't keep the same client IP on the same replica, thus creating problems when a client writes something on one replica, then gets switched to another that didn't get the update yet.

I'd love a better solution for this btw. Is noone working on Hipache anymore? Other than this problem I find it very elegant.

you might want to take a look at fabio:

https://github.com/eBay/fabio

I love Fabio. The Consul stack in general is fantastic.

Been watching #consul IRC for a while now though, and the vast majority of the setup problems are due to Docker's weird networking and security. Fabio/Consul run like a charm but Docker throws a wrench in the machine.

> Anytime I add a new machine (e.g., web4) i've to change the conf of HAProxy and restart it. This brings down the system for a moment. (is this the right approach btw?)

You might find this story from a year ago interesting:

"True Zero Downtime HAProxy Reloads" http://engineeringblog.yelp.com/2015/04/true-zero-downtime-h...

HN discussion (with some answers from the post author): https://news.ycombinator.com/item?id=9369051

I'm curious as to how much of an actual issue you experience though. Barring an error that prevents HAproxy from starting, it should be pretty quick? Maybe not quick enough for streaming media/realtime audio-visual communication though.

You can use nginx to have zero-time redeployments without gross hacks haproxy requires to do that.

I would also recommend nginx. For pure proxying it's rather convenient, and offers some nice high level caching if you desire it.

But to use nginx i've to recreate the container to link the new "web4" container. don't I? and then to make it reload the conf. or is there anything else that i can do?

Why load balance three copies of a container on the same server? Just set your cpu and memory constraints to what you need for one.

you got a point indeed. My idea was to handle more things in parallel for services that are not heavy in resources. such as a service that checks the auth of a user from a ID/Key. anyway, with the swarm I could put the containers in various machines.

Yes, for ex: docker run -d --name web1 --net prod --net-alias web <webapp> docker run -d --name web2 --net prod --net-alias web <webapp>

resolution for 'web' will return IPs of both the containers. You might still have to watch out for the DNS caching at the application level.

The Open Container Initiative is an important step towards some form of standardization for the future of containers, and I think the Linux Foundation will steward the project well.

Standardisation at the format level will help with portability across different platform implementations.

This "standardization" is pretty kludgey if you have to docker pull then docker export.

I think that's because OCI currently only has a specification for the runtime, not the distributable image. But it seems like as of a few weeks ago, work is underway to standardize the distributable image as well: https://github.com/opencontainers/image-spec

I was almost excited! From the FAQ on the project you linked [0]:

> Q: Why doesn't this project mention distribution?

> A: Distribution, for example using HTTP as both Docker v2.2 and AppC do today, is currently out of scope on the OCI Scope Table. There has been some discussion on the TOB mailing list to make distribution an optional layer but this topic is a work in progress.

I really hope CoreOS manage to get distribution into the scope for OCI. We need to move beyond Docker images. Standardizing on-disk layout in OCI is only mildly useful in my opinion.

[0] https://github.com/opencontainers/image-spec#faq

There are other tools that can get data from a docker v2 registry.

It is standardisation of the file describing what parameters were used to build a container, ultimately every "implementation" has been a tarball of a linux filesystem which is usable with any linux container system (and possibly soon Windows).

"As an example, 1.11 is a huge step toward allowing Engine restarts/upgrades without restarting the containers, improving the availability of containers."

This is great.

ELI5 the OCI & why it's important? I follow Docker from a distance and find it fascinating.

Disclaimer: I work at Docker.

Docker has become a de facto standard for executing programs in a portable sandbox (aka "container"). There were increasing demands for making it a "proper" standard, so last year we donated a spec and reference implementation for a universal intermediary format - a "PDF of containers" if you will, and partnered with the Linux Foundation to manage it. The majority of the industry followed.

Now that the Docker container engine supports this intermediary format, and other providers will soon follow suit, it reduces the risk of depending on one provider. If you want to switch away from Docker, you can run your containers elsewhere.

Now everyone can focus on building better tools, instead of trying to make "their" format win. The result is better tools.

As ELI5s go, this seems not well-targeted at 5-year-olds :)

As is pointed out repeatedly on /r/ELI5, the explanation doesn't need to be for literal 5 year olds.

Correct. It's not literal, and I didn't mean it literally (I don't think anyone actually does). I would probably have not been happy if that was the only type of answer. Maybe, we will say ELI16? Kidding. No, don't do that.

I'm aware, but I felt like shykes was simply repeating the copy from the blog post without actually breaking it down into simpler pieces. A sibling post did much better.

Disclaimer didn't say you are the CEO.

I hope this guy does this again here, https://news.ycombinator.com/item?id=11379475

> Disclaimer didn't say you are the CEO.

Correction: not CEO, "Founder, CTO and Chief Product Officer"

https://www.docker.com/company/management

Solomon's comment and disclaimer seemed perfectly reasonable to me. Why does it matter if he's the CTO or random employee?

I am sorry if this comes across as mean-spirited, but, I think it is quite disingenuous to claim that Docker is a "de facto standard".

OpenVZ for instance, has well over 77K host-nodes which run over 840K containers.

These are not ephemeral setups on someone's laptop but servers that companies are spending money on each month to have in a datacenter.

IMHO (because the reporting method requires a small stats program to run, which is optional and many don't run it) there are 2 to 10 times more in actual existence.

Source: https://stats.openvz.org/

'a' defacto standard, not 'the' defacto standard.

You don't come across as mean spirited, it's a fair point.

I didn't mean that Docker is the most widely deployed software in a particular category (that would be hard to measure accurately anyway). But what Docker introduced is the use of containers to deliver software, not just boot servers. Developers use Docker to package and publish their runnable artifacts, in very large numbers. For example Docker Hub passed 2.5 billion image downloads. Many more images are certainly transferred between private nodes every day. OpenVZ does have a concept of templates (I was an avid user in past life) but they were never intended as a means to distribute applications. In that regard Docker is undisputably the de facto standard.

I hope this clarifies my meaning.

PS. OpenVZ is awesome. I remember when we made the switch from ovz to lxc in 2010, when it became clear which would get merged mainstream, and we could finally ditch our double-patched xen+ovz kernels. The transition was very painful! What we missed most was the "bean counters" feature, ovz stats were unmatched for quite some time.

Am I correct in assuming that containers specified by OCI can be exported and managed by other tools like lmctfy. You see your competitive advantage being in docker.

As an industry, we hadn't put down on paper what it means to build, execute or discover a container.

If you wanted to sit down and write software to build or execute a container, there wasn't a spec to follow or a checklist of the features you needed to support. Basically, you had no idea if what you wrote today would continue to work in the future. Or if someone else constructed a container, that you could run it.

(CoreOS employee)

Garbage collecting for the registry is a long overdue feature that is quite critical. I'm glad to see that it's now possible.

I'm pretty happy about being able to add labels to `docker build` - now it should be possible to easily associate a git hash of the code that was used to build the container.

> A few months ago we added the ability to sign images with hardware Yubikeys in the experimental channel of Docker. This is now available in the stable release.

I understand that you used to have to install a separate version of Docker to access this feature, but does the change in 1.11 also mean you no longer have to set the DOCKER_CONTENT_TRUST environment variable, or will that be made default at a later time?

Still need to either set the envvar or pass the equivalent flag.

Correct link to v11.0 change log: https://github.com/docker/docker/blob/bump_v1.11.0/CHANGELOG.... The article links to master changelog which is 1.10.

Somewhat off topic, but docker-compose finally works on the old iMac I use at work:

https://github.com/docker/compose/issues/271

As someone who spent too much time with Oracle, this headline is quite confusing (OCI => Oracle Call Interface). Computing has a serious TLA issue.

ha, googling gives me OCI Open Catalog Interface, some SAP standard for SRP software, which makes no sense in docker context, at all :D

Does this mean we are now at a point where LXD/LXC & docker will play nicely with each other?

I don't see any movement towards the OCI spec by the LXC/LXD team. Canonical isn't shown on the list of members (https://www.opencontainers.org/about/members).

This answer from Mark Shuttleworth seems to sum up their position: https://answers.launchpad.net/ubuntu/+question/268502

Reply on news.ycombinator.com