Hacker News Reader

While it is scary when a company takes its initial success in the technology scene and uses it to expand into its verticals, this is not quite the same case as the Microsoft analogy...Docker is open source, after all. I think the backlash that CoreOS started with its Rocket / AppContainer announcement ahead of DockerCon has caused some knee-jerk reactions.

Does Docker's expansion into orchestration and clustering endanger CoreOS and maybe other players? Certainly. Is it the same approach? Definitely not. The CoreOS approach to containerization using Docker, and now Rocket, is novel. I haven't considered it for any of the architectures I'm presently building for companies - I prefer to let the operating system do what it does best, and build my services on top using containerization.

I was worried about what Docker's plans for clustering entailed leading up to today. I have been using Mesos for the better part of a year, and see it answering many problems that reach beyond the container model. I think people should do as you suggest, and re-evaluate this on two fronts: 1) Swarm is separate from Docker, so we can continue to deploy Docker through Mesosphere Marathon onto Mesos as we've been doing. 2) Because scheduling is pluggable - as the source code validates for us - we can work in Mesos as we see fit.

As for the proposal, I vote to keep Compose (and the fig codebase ;) ) separate. It will be much easier to add the hooks necessary to do cluster management between Compose and Swarm without risking regressions to Docker.

Also - what has become of the Docker Governance Advisory Board? I think the ratification of a standard / specification would go a long way to solidify trust in the industry (much like CoreOS is trying to do with their App Container proposal).

- Paul

The advisory board is just that -- advisory. It has no teeth. All approval of anything goes through Docker.com/Solomon.

It is worth reading through the notes from the last/only meeting: https://docs.google.com/document/d/1JfWNzfwptsMgSx82QyWH_Aj0...

Have you participated in any other open source projects? All of them with any success in the community (that I can think of) have a monarchy or oligarchy at the head. Anyone may contribute, and a group of committers review and add if the contribution fits the project... who determines what the project and its roadmap? Usually a select few, driven by even fewer or just one. So I don't think it's necessarily unusual that Solomon heads it.

As an aside - I've watched headless projects devolve into anarchy... here's a good read from years ago - http://ariya.blogspot.com/2008/03/common-characteristics-of-...

You are right -- perhaps I'm being unfair. It is Docker's right to run their project as they see fit.

I'm glad to see the new projects (swarm, at least) layered on top of Docker. It is good to see those sink/swim on their own. I was uncomfortable when Docker was (last week) going down the path of building everything into their existing binary.

You're more than fair. They made the rules, and I guess they can break the rules. But really, the value of the project lies in the community's continued support and backing of the project.

The good thing is that this is open-source Apache-licensed code. Easy enough to fork it if necessary, even if just temporarily while we wait for another project to mature and replace it.

  Hudson -> Jenkins
  Mysql -> MariaDB
  Docker -> ???

Even if they were in the main binary, do the technical consequences really outweigh the politics?

Joe I can appreciate the drive to aggressively monetize a specific business model ;) but if you really feel what you've seen at Google should win, you'll have much more leverage taking a pedagogical approach. Appreciate that you have certain duties.... perhaps you might have had more agency at Facebook?

FreeBSD has no monarchy. Nor does the Apache process.

OpenStack????

The perfect example of a headless project achieving a lot less than it should.

Benevolent dictator can work. But here Docker has said they want to make it possible for the project to be community led and run over time. Actions on the ground speak otherwise. Hard to restore trust once lost.

Agreed. I'm very confused about how the concerns from the advisory board are (not) being addressed in an open manner.

From reading the minutes, it seemed to me like one of the clear conclusions (based on concerns raised by big stakeholders) was that Docker Inc. was going to be firewalled from the Docker open-source project and that development work going forward was going to be done in a more open fashion.

All of these things have happened. Do you have specific evidence of the contrary?

I gave a presentation on this topic at Dockercon today, it talks about how the project is organized, how the company is firewalled, and recent improvements we're working on since the DGAB. I will share the link with you once the video is available.

Thanks for the comments.

The advisory board met last month. Here is a writeup http://blog.docker.com/2014/11/docker-governance-advisory-bo...

There is also a link to full minutes.

Speaking for the DevOps, a large part of the job is being responsible for building trusted systems from trusted components. I want to continue considering Docker to be part of that trusted stack, but from looking at the minutes, it seems that you are willing to mislead even large partners like Google while on the record.

  JB (Google): Docker is an anchor of a large ecosystem of projects. Docker Inc 
  bought Fig, is it a separate project or is it going to be in Docker?
  SH: it should be explicit. The Fig guys make proposals and those are reviewed
  as every body else's.
  JB (Google): The idea is we will be more comfortable if we know how to 
  contribute both to Docker and the larger ecosystem. 
  SH: The day after Docker Inc. acquired Fig nothing changed at all, everything
  is still in the Open. The reason for the acquisition was to make Docker more
  user friendly. The experience should be awesome. The more people we can throw
  at the problem the better.
  VL: Understanding what is off limits or at least where the limits are would
  be helpful. This is entirely driven by technology? We can collaborate "here" 
  but maybe we can compete "there".
  BG: It's absolutely right, there is a large number of projects competing with
  Docker Inc. and that's fine. Some folks want Docker to remain a packaging
  format. We should add a statement there on clarifying this.

Source (both as github gist and google doc):

  https://gist.github.com/inthecloud247/00eb3130f728b635db9e
  https://docs.google.com/document/d/1JfWNzfwptsMgSx82QyWH_Aj0DRKyZKxYQ1aursxNorg

Just today, the "Host management" proposal initiated by a Fig developer was closed (by him) without any discussion after he released the new Docker Machine project (which AFAIK was initiated without any public discussion or warning.) https://github.com/docker/docker/issues/8681

I definitely haven't scrutinized this as much as you have, but I don't see the scandal here.

Docker Machine is separate from Docker. Once Machine was released, what was the point of talking about integrating the functionality into Docker?

They've repeatedly stated that they want Machine and Swarm to be totally separate tools.

Joe Beda is just trying to reduce the amount of work and risk his GCE team takes on supporting Docker. Solomon is re-asserting the Docker project's autonomy. There's no scandal here. Tension, yes, but Joe Beda is playing textbook strategies in the interest of Google's business model, and Solomon is giving a mostly textbook response.

The Docker team needs some space to execute on the opportunity before them. Space to try, fail, innovate. The end product will be better if they have that space and don't have to achieve equilibrium with too many stakeholders or worry about political microfriction when attempting big leaps.

The bleeding edge is bloody; if you need full-featured orchestration TODAY, your best bet is probably something older than Swarm, Fig, etc.

A container, while it can be many processes, in the case of Docker is just one process. So its just a better Operating System process...

Why then is there all this fuss about how these projects are organized? If its a good idea (which it is) won't Linus just reimplement(conform) all of these projects and move them into the kernel anyways?

> in the case of Docker is just one process.

That is not true. It can be just one process. It does not have to be.

> So its just a better Operating System process

No. That's just a tiny bit of what Docker provides, and even that part of what Docker provides is largely "packaging" of cgroups functionality. That was largely there with tools like LXC

The real value Docker provides is a means of building and managing images, and mediating things like capabilities and port mapping in a user friendly way

> won't Linus just reimplement(conform) all of these projects and move them into the kernel anyways?

Not a chance. There's no reason why these things needs to be done in kernel space. Some features that they may want, such as improvement to filesystem overlay support, or low level "plumbing" to make overlay networks more efficient, could end up in the kernel if someone comes up with something good, but there'd still be plenty to do for the userspace tools.

I think the concern from third-parties is not so much about bloat or whether everything is bundled in the same binary but that by defining "the way" of doing things, you are taking the oxygen out of other ecosystem partners that have alternate approaches for doing so. You are making it pluggable but by having your official implementation it becomes the "de facto" standard.

By defining a standard way of orchestrating you make life easier for users that can deploy with confidence the same environment across multiple clouds, etc. BUT the tradeoff is that you alienate another set of parties (providers are commoditized, ecosystem partners offering alternate orchestration tools feel marginalized). Secondarily, there are concerns whether your approach is better than others and that is too early to tell.

I am curious to see how all this plays out, lots of things at stake here and a very thick fog of war :)

Anyone who has used Docker for more than a trivial app sees the need for orchestration. If the requirement is that obvious then it clearly needs to be part of the core.

Many many solutions have been meeting that need just fine and many new ones are still in the works. Adding a new solution will not help things.

Yeah it will.

At the moment I have to choose between multiple competing solutions. Competition is good when there are strong differentiating features.

In the "Orchestrate Docker" field the requirements are so clear after using it for your first project that it isn't at all clear what benefit competition, or solutions outside of the core Docker product suite gives.

As someone who has followed Docker since it was part of dotCloud, I can tell you that some standard practices will be great. I've seen dozens of different ways of doing similar things with Docker.

By creating these orthogonal solutions, Docker isn't taking the air out of anything. Do these solutions hurt Amazon's proprietary use cases? Or Google's? Or [insert managed Docker platform here]? No.

If anything, they just make it possible to use Docker without third parties. Maybe that's scary to third parties, but there's no reason a company shouldn't compete in its own ecosystem as long as it doesn't unfairly punish competition.

So if Docker acts like modern-day Microsoft for enterprise, where solutions are sold from many vendors with many mixes of products, then great.

If they act like Apple, where they crush and eliminate overlapping services, then I guess we'll all be switching to the CoreOS containers someday...

We are using Docker to package an on-prem product.

As the ecosystem is recent and evolving quickly, we are looking at ways to handle multiple containers and deploy them at scale. And often, there are several solutions.

Having docker provide the set of components (and it's great to have multiple components for that) helps in defining a standard direction.

And as long as all the components are open source and can be deployed on our end, we are glad to use them. We would consider switching if somehow the docker tools become tied to a proprietary service or tool, which is unacceptable for our customers.

I'd also lean towards keeping compose a separate binary, particularly given the community's concern over Docker's feature-creep, and the fact that it uses a separate configuration file, and doesn't need to be the same binary for any particular reason.

Also, I filed an issue on the Github repo for this, but "machine" is already used by a command line tool on OS X and other BSDs.

Perhaps "docker-machine" and "docker-compose" are good compromises in both cases.

Maybe docker could use subcommands a la git.

So with docker-machine binary on the path, "docker machine" would just delegate to that.

Similarly, if I had a docker-flynn binary, "docker flynn" would just magically work.

This means that the main binary is composable from multiple parts and its easy to write plugins for other systems.

I actually just suggested the git plumbing/porcelain model in another comment, which would fit nicely with this suggestion: https://news.ycombinator.com/item?id=8701085

Why not just "flynn"? Should I really be typing "docker" before every command for the sake of someone's branding?

Imagine if git, svn, cvs all installed their own "checkout", "clone", "commit", etc. executables. It would become chaos if you had more than one installed. I prefer when various software suites keep their executables in their own namespace, if only for sake of future-proofing, to prevent naming conflicts.

There are always symlinks and/or shell aliases if you want to override that locally.

That's nice, but it's irrelevant because there are not going to be many "flynn" executables.

Hi shykes,

Well, first of all, thanks for Docker, it is a great product, love using it!

My only criticism is:

I find it scary and kind of shortsighted that docker (and even more so rocket) seem to neglect basic networking features. From what i can tell everything is based on the approach that you write your webapp with docker in mind, but i for one, would like to use my legacy applications with docker and i would like to write applications with full networking in my control. I really, really don't want docker to choose some IP for me. And i really really don't want my apps to talk over some random ports. I want control.

So far everything is still based on some random interfaces and ports that are passed around.

Bingo! We looked at Docker for VoIP a while ago and it didn't appear like it was a good fit for such applications. To be fair, Azure had the same odd-sighted flaw, although they have a weak workaround with public IPs now.

(disclaimer: I'm one of the Docker maintainers)

We hear you and we already started working on addressing just that. The goal is to define a networking interface (a set of functions) that networking plugins would satisfy. Again: "batteries included but removable". There's a lot of work to be done and details to be figured out, but I can assure you that networking is not left out, and is going in the right direction as well.

We will send out a proposal on Github as soon as we can. Stay tuned!

FYI today we had a design session on this topic with several members of the ecosystem (Weave, ClusterHQ, Socketplane, Tutum, VMWare) during Dockercon. As soon as there's something worthy of a proposal, we'll open it up for comment on the repo. I'm pretty excited about this.

This is a completely new product and really should be differentiated as such via new packages:

  apt-get install docker.io-platform

Just don't put any of this stuff into the docker.io or lxc-docker packages and mostly things will be well in the world.

People who don't understand how much utility these improvements provide can complain all day. Personally you've delivered exactly what I have needed quickly and, from my use, bug free.

My vote (not that it gets counted here) compose should definitely be merged.

Great job on all the rest, ignore the haters, they just prove you've made it!

> Swarm and Machine are separate binaries.

They appear to enjoy first-class support in the docker CLI. Is that right?

No. There is a question on how to get machine to cleanly change which machine Docker talks to. The current answer is: standard dynamic config system in docker core, which machine plugs into to change values.

I think I got swarm and machine mixed up. Swarm seems to be tied to the CLI, because the command is "docker run", with additional settings. I think I am skimming too skimmily.

Pluggable scheduling and placement is a good move. It all looks like a straightforward operations research problem ... until you discover that you built a SPOF subject to the network fallacies.

> because the command is "docker run", with additional settings

The `-m` flag was already present in docker run. Swarm makes use of that flag to decide on which host to put a container. The `constraint:` setting is indeed new, but (afaik) the values behind it are not pre-defined, so if someone develops an alternative clustering-backend they should be able to make use of that for their needs.

In any case, Swarm will be pluggable, so other back ends are possible and can be developed by other parties.

Thanks.

I upvoted you just for using the word "meme" correctly.

The supposedly incorrect uses of the term are also correct -- even with the blessing of the original creator of the word:

"""In 2013 Dawkins characterized an Internet meme as being a meme deliberately altered by human creativity—distinguished from biological genes and Dawkins' pre-Internet concept of a meme which involved mutation by random change and spreading through accurate replication as in Darwinian selection. Dawkins explained that Internet memes are thus a "hijacking of the original idea," the very idea of a meme having mutated and evolved in this new direction."""