A new upstream project to break up Docker into independent components
github.com
github.com
Here is an updated version with some clarifications:
<<<
Docker is transitioning all of its open source collaborations to the Moby project going forward. During the transition, all open source activity should continue as usual.
IMPORTANT NOTE: if you are a Docker user, this does not change anything. Docker CE continues to exist as a downstream open-source product, with exactly the same interface, packages and release cycle. This change is about the upstream development of the components of Docker, and making that process more open and modular. Moby is not a replacement for Docker: it's a framework to help system engineers build platforms like Docker out of many components. We use Moby to build Docker, but you can use it to build specialized systems other than Docker.
You can learn more at http://mobyproject.org
We are proposing the following list of changes:
- splitting up the engine into more open components - removing the docker UI, SDK etc to keep them in the Docker org - clarifying that the project is not limited to the engine, but to the assembly of all the individual components of the Docker platform - open-source new tools & components which we currently use to assemble the Docker product, but could benefit the community - defining an open, community-centric governance inspired by the Fedora project (a very successful example of balancing the needs of the community with the constraints of the primary corporate sponsor)
>>>
EDIT: I'm happy to answer any follow-up questions here, if it helps clarify.
Also, isn't it kind of ironic that you built your company on OSS and then invited a well known destroyer of OSS onto your main stage?
Plus, I want my DockerCon money back. What exactly did you announce besides multi-stage builds at the general sessions that is actually benefiting Docker users? I don't care about your plumbing and constant rebranding, other than finding it very discouraging, I want to know what you're actually doing with your product this year. I guess nothing.
This.
I was a very early proponent of docker, and have been using it continuously since the very early versions. I read the release notes and most weeks I read the weekly email newsletter, and I still don't really have a clue where Docker is going, what its current status is (as far as how the various pieces fit together, etc) or what half the new features even do.
The docker team's communication is really poor, and their documentation seems even worse, unless I'm missing some oracle somewhere that fills in all the details that the actual documentation leaves out...
if you don't currently use any of these new features, and can't work out what they do, perhaps you really don't need them. Therefore, ignoring it and continuing on is the best cause of action. It's not docker team's responsibility to educate you!
(Or you could make the assertion that I'm just an idiot, but my career suggests otherwise.)
I was annoyed at the fact that I paid $150 extra for a workshop, "Docker for Publishers" that while useful in itself, I could have done in large part for free as a Hands On Lab: "You'll then deploy both a certified Docker image, as well as a certified Docker plugin."
I think I missed this; who are you referring to? I scrolled through the list and didn't recognize anyone that I would consider a "destroyer of OSS" so I'm not sure if I missed something, if you're just exaggerating or a little bit of both.
Incentive structures be damned, the evil M$ must be fought, here, I brought this knife to cut my nose off with! (+5, Insightful)
https://blog.docker.com/2017/04/oracle-database-dev-tools-in...
(The other posted list talking about Microsoft is from 2016)
Edit: See my response below
https://blog.docker.com/2016/06/dockercon-general-session-vi...
Guest speakers for the last (second) general keynote were:
-Guest speaker #1: Docker and Microsoft demo by Mark Russinovich, CTO Microsoft Azure
-Guest speaker #2: Docker at ADP by Keith Fulton, CTO, ADP
And just somehow I sense that the parent poster isn't objecting to the CTO of ADP. /s
(What makes the objection doubly silly is that Mark Russinovich isn't the usual corporate drone one might expect. He's the guy who did the Sysinternals tools and several editions of the "Windows Internals" reference on the architecture of the Windows OS.)
The fact that they’re constantly brought on stage to be paraded as a symbol of enterprise maturity and stability just tells me that the product is not mature or stable enough to speak for itself, and that the original OSS community really has no say in the direction of Docker while enterprises are sold the ship and wheel for their rubber stamp approvals.
From Sysinternals to the Azure CTO... what a weird transition. That guy has had an interesting career.
https://en.m.wikipedia.org/wiki/Sysinternals
(Interestingly citation needed; people still circulate the code on BitTorrent of very old copies.)
Ironically a MSer told me he was famous for his infamous condition he be a Senior Engineer with an unorthodox, exceptional contractual or understood conditions: no painful managing other MS people. This only changed when made the CTO of Azure.
No idea if that's rumor's true. He's a personal hero and I'm yet to read his novel about cyber war (yes, you heard me).
Today images are flat. You have a proper thing that is an image and and image is composed of layers. Some images may share layers (so they only need to be downloaded/stored once) but there is no hierarchy. You don't see layers anymore except by inspect an image (and even then only the content hash). There are quite some reasons for this to be the preferred method.
But they're certainly doing a lot with the product this year. It starts with cashing in the open source goodwill and rerouting the eyeballs to their enterprise offerings.
Docker is, and will remain, a open source product that lets you build, ship and run containers. It is staying exactly the same from a user's perspective. Users can download Docker from the docker.com website.
Work has been ongoing to break Docker into modular components for some time, with runc and containerd as examples. Containerd for example has been donated to the CNCF. We are now completing this work with the goal being that the monolithic docker repo eventually ceases to exist, instead being assembled from a set of components.
Moby is a project which provides a "Lego set" of dozens of components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
Docker the product will be assembled from components that are packaged by the Moby project.
The Moby project provides a command-line tool called moby which assembles components. Currently it assembles bootable OS images, but soon it will also be used by Docker for assembling Docker out of components, many of which will be independent projects.
As the Docker Engine is split up into multiple components the Moby project will also be the home for those components until a more appropriate location is found.
Can I call my tool moby-something?
For Moby, there is no need for a strict trademark policy because Moby is a "meta-platform". In other words downstreams don't run Moby: they build their own specialized platform with moby. So they don't need to worry about the Moby trademark anymore than they need to worry about the trademark of gcc or make. Anyone who uses Moby to build their platform will be more than welcome to say "built with Moby".
- Who will be able to use the Docker trademark? If someone built a fork of Docker CE from Moby, can they still call it Docker? What about distribution packagers?
- Right now a single binary ('docker') is used to build images and runs clusters (among other things). As the monolith is broken down, will the new binaries still be named/branded 'docker'? Will there be rules about packaging only some of them? Will Docker CE/EE still have a single binary for all purposes?
Docker.
> If someone built a fork of Docker CE from Moby, can they still call it Docker?
No.
> What about distribution packagers?
We're still working this out, with the guidance of a few distro packagers. There will most likely be a change of some sort, but don't want to rush anything until we are certain that it won't break users.
> Right now a single binary ('docker') is used to build images and runs clusters (among other things). As the monolith is broken down, will the new binaries still be named/branded 'docker'?
Yes. The same 'docker' binary will continue to exist. We are going to spin it out into its own repository under the docker github org. It will only act as a client, and we will encourage the development of a healthy ecosystem of alternative clients.
> Will there be rules about packaging only some of them?
Basically only Docker can create something called "Docker". Everything else will be regular open-source project rules: build, patch, package any way you want.
> Will Docker CE/EE still have a single binary for all purposes?
It depends on the target platform. Docker for Linux is a deb or rpm package with several binaries. Docker for AWS is a Cloudformation template. Docker for Mac is a Mac application with a very complex array of components built in. etc.
Could you explain the reasoning behind the need for a new name?
I ask this because would it not have been a better choice to name the other releases differently.
For example:
Docker (docker/docker) = open source development.
Docker CE (docker/docker-community) = free product release
Docker EE (docker/docker-enterprise) = commercial product release based on Docker CE (private).
docker/component-1
docker/component-2
docker/component-3
It's a common concern in the free software community that having an open-source project too tightly coupled to a single company's product is a bad thing. Moving the upstream open-source project into Moby, and keeping the downstream open-source product into Docker, solves that problem.
I think a lot of the confusion comes from the downstream product split between Docker CE and Docker EE.
Many people are waiting for the other shoe to drop, because the point of releasing a separate binary version for 'CE' and 'EE' versions is very vague if the products are identical, with the extra EE functionality not really being necessary since it can be added via external services like DDC or plugins for LDAP authentication.
Is Docker EE also open-source?
Breaking out core Docker into components sourced in an external project like Moby makes it easier for you to replace the OSS components of Docker EE one piece at a time with your own private, closed-source versions where the paid development is focused.
So you get Docker CE which pulls all components entirely from Moby, then you get Docker EE which starts off identical. Then at some point you fork an individual Moby component, add some enterprise feature, and Docker EE now pulls from this. Docker EE is now a variant that's "mostly" open source with that one closed component. This component is focused and optimized for enterprise for a bit, then the same thing is done with a different component.
Rinse, repeat. "Docker EE" drifts away from its open source roots and over time loses resemblance to the original project while still taking advantage of the brand that's been built and maintained by thousands of contributors. Essentially, it's a way to co-opt an open-source product.
That's the source of my initial misgiving, anyway.
I get it, you're a business and you want to make money. Just stop being coy about Docker EE and Docker CE continuing to be identical except for the 'support and certification' mentioned in the Docker EE announcement.
I want to address your concern but don't really understand what it is. What exactly are you worried will happen, and how does it affect you?
Really?
It's a question of ethics. You must surely get that?
Correct. Moby uses containerd because Docker uses containerd.
> which I guess is not related to systemd
You guessed correctly.
> even though systemd also has its own container system
Correct I believe systemd uses a tool called nspawn to isolate processes in containers. We don't use systemd/nspawn, but I think it would be a cool customization to build a Moby assembly that uses systemd.
> Are there any components to moby that are OSS
All Moby components are open-source.
> and a part of the linux stack
All Moby components run great on Linux.
> or is this whole endeavor to break away from linux dependencies?
Moby works great with Linux and will continue to. In fact you can build a complete custom Linux system with Moby, thanks to LinuxKit. (But you can also target an existing Linux system, LinuxKit is optional).
> Moby works great with Linux and will continue to. In fact you can build a complete custom Linux system with Moby, thanks to LinuxKit. (But you can also target an existing Linux system, LinuxKit is optional).
I think the question was more along the lines of "Are these changes being made to add support for other Operating Systems?" and less "Are you abandoning Linux?"
I don't think anyone was questioning your commitment to Linux, though I kind of am now because of how defensive that non-answer was.
OCI doesn't have an equivalent to discovery yet, but presumably it'll be something similar to the appc spec (https://github.com/appc/spec/blob/master/spec/discovery.md)
I also recently talked to some CoreOS devs at Dockercon and have started considering extending rkt to better support OCI runtimes (and images though images are "supported" at the moment). Exciting times.
I wish the market of container images had more competition. Right now there's Docker Hub/Quay (expensive, lots of features) and Amazon ECR/Google Container Registry (cheap, few features).
I see. Looks like I had misunderstood.
Yes, being able to target more different platforms is one of the reason for the change.
- RedHat eating Docker's lunch, and dominating Linux development
- the somewhat questionable practice of using Docker as a GPL-circumvention device (eg. commercial images pulling a Debian/Ubuntu userland on first load), though I'm unsure about the legal implications
Here is a summary we wrote earlier for Moby https://mobyproject.org/#moby-and-docker
see https://www.openhub.net/p/docker for statistics on that.
To answer your question: we think this move will accelerate the growth of the project, because it addresses the two most common concerns from contributors: 1) that the project is too monolithic, and 2) that it's too tied to the Docker product and company.
That's good news.
So you're setting up some sort of foundation so that Docker is no longer a gatekeeper for the upstream stuff? I didn't see that in the announcement.
This has pros and cons. Red Hat and Mozilla both learned they had to be aggressive with trademarks, as people were distributing sabotaged builds under their product names.
That's why Firefox in Debian is called IceWeasel, for example; any Firefox build that incorporates external patches (or even one that is built with unofficial options, iirc) must not be called Firefox (barring a waiver from the Mozilla Foundation). This allows Mozilla to pursue people who are distributing contaminated builds of "Firefox".
Changing posture on the Docker trademark will make the arguments of Docker Inc. more persuasive should they find themselves needing to make use of the same sort of remedy. It will also make a separate upstream so that Docker will now be based off the "Moby Framework". If Docker Inc. becomes defunct, "Moby" will be an separate project that lost its primary sponsor, not a defunct project bearing the name of a dead company.
The cons are that it's probably a signal that Docker is going to continue working to monetize aggressively, potentially including spurious legal threats that limit the discoverability of competitors and vendors of alternate tooling. For example, it's possible that Kubernetes will no longer work "with Docker" but "with Moby", and the only orchestration for "Docker" will be docker-swarm.
It's also a potential signal that Docker plans to gear more of its work toward its souped-up closed-source distribution available to paying customers only.
The other pro is that Docker as a product totally sucks, so their alienating people in the community will drive users to other solutions that suck less. I keep hoping the containerization fad will go back in the bottle, and while unlikely, this diminishes faith in it since Docker == containers to a lot of people, so that's good.
Is this because containerization in of itself should go back in the bottle, or do you think that people are just taking it too far?
Kubernetes is massively overcomplicated and has bitten off an offensively oversized problem space. The arrogance is staggering.
Ultimately, VMs and containers are attempting to solve the same basic problems, and while containers do have some cool features, there is no way that they justify the gross tradeoffs people are making in service of the fad.
What the flip did you do? They held off the Great Firewall!
/impressed
Once the componentization is complete, Moby will be an integration repository, with the tools necessary to combine upstream components into a complete system. We will use it to assemble Docker, and other system builders can use it to assemble alternatives to Docker.
This redirect will probably prove disruptive, confusing, and frustrating to a lot of people. I assume it's too late now, but you might consider asking github if you can revert to allow github.com/docker/docker to remain as it was for a time, and allow moby/moby to gradually pull in the actual code.
Perhaps there are technical reasons this would be infeasible. The github landing page serves as valuable marketing, branding, and social-proof (stars, etc) even if most folks don't end up looking at the code there.
What will happen with hub.docker.com? Is store.docker.com the replacement? Or will both coexist? Which one will be hosted on mobyproject.org ?
If you're a Docker user, nothing changes for you. Docker Hub and Docker Store are also unaffected.
Such as "docker reputation for security became too terrible" or whatever.
Docker only got root privilege isolation (i.e. don't run every container as root on the host) a little over 12 months ago.
Their entire approach seems to be "build whatever buzzword batshit crazy idea cool kids want", with "hey is this shit stable, or even remotely secure?" way down the list of questions even considered during development.
Also: if someone is an OSS maintainer, and their response to bug reports is as bad as Dockers have been, sorry but they deserve to feel bad. The "you" I refer to below is a mythical OSS maintainer, not you specifically, @andrewguenther.
Maintaining software doesn't automatically remove any responsibilities you have to maintain the community. If there are huge issues with the software, you have two choices:
- acknowledge and identify a plan to solve the issues. In this case, assuming the issues are resolved in a timely fashion, I would expect people to remain courteous.
- treat the community as shit, ignore security concerns, and never fix fundamental bugs. In this instance, you should expect people to call your project out for the piece of shit it clearly is.
Let's also be clear here: we're not talking about some tin-pot project with 2 guys hacking away on their lunch breaks to find a cure for cancer, who are being told "this is crap".
Docker is developed by Docker Inc, a company that received nearly $170M in venture capital investment just in 2014-2015, and apparently makes several million a year in revenue.
So, do as you say, not as you do, eh?
Also, how on earth is suggesting a piece of software has a terrible security reputation, "assholish".
Exactly.
Moby is an open framework created by Docker to assemble specialized container systems without reinventing the wheel. It provides a “lego set” of dozens of standard components and a framework for assembling them into custom platforms.
Audience
Moby is recommended for anyone who wants to assemble a container-based system, this includes:
Hackers who want to customize or patch their Docker build
System engineers or integrators building a container system
Infrastructure providers looking to adapt existing container systems to their environment
Container enthusiasts who want to experiment with the latest container tech
Open-source developers looking to test their project in a variety of different systems
Anyone curious about Docker internals and how it’s built
Moby is NOT recommended for:
Application developers looking for an easy way to run their applications in containers. We recommend Docker CE instead.
Enterprise IT and development teams looking for a ready-to-use, commercially supported container platform. We recommend Docker EE instead.
Anyone curious about containers and looking for an easy way to learn. We recommend the docker.com website instead.
I see this kind of cynicism / snark all the time on HN (especially when it comes to Docker) but is it really warranted? Surely it's the rare exception? I've never met senior people who couldn't understand the naming difference between the open source version of a thing and the enterprise-branded version. Or a CEO or President. They're not morons.
This seems like cheap shots for upvotes. Or am I wrong?
The conversation usually goes like this with my folk:
I get it, Moby is the OSS version and can be easily stood up and you don't see the need for extra cost. I'm glad you're trying to reduce expenses, but I don't mind paying for 24x7 Enterprise Support and it gives me piece of mind and priority escalation support.
Basically, I don't mind paying extra for a get out of jail free card if a whole swarm is offline.... I can tell the CEO we have the vendor partner engaged with a critical ticket and she can have piece of mind too. So, let's quote out the Enterprise version of Docker and evaluate the TCO of both solutions.
It's not a "cheap shot", and it's not for "upvotes", it's just an observation about the nature of this change being non-technical, related to messaging.
- a CIO
You might not care about this, and many don't. You're running the risk that the maintainers won't pack up shop and stop maintaining the project, and that you & your team will need to inherit the project.
Other way of looking at it, is that if you're a business, you always should pay for your software, even if it's open source. Because you will one way or another: through your team, or through the risk that free riding brings.
Platinum support means you can get C-level executives on the line.
Reputation and counter party risk all factor into evaluation and cost of support as well
That's "on the _premises_".
It is a FACT that individuals are unable to use the term "Docker" on anything they would like to create and share with others. This has been enforced with takedown notices and legal entities getting involved in "protecting" the Docker brand, on Docker's behalf. This, in and of itself makes sense, given Docker is a private company with a significant amount of private investment behind it, with expected returns on protecting that investment. It has used that investment well to further build the brand, as you indicate here.
However. Docker is also an idea which is one of taking a "software container" and starting it somewhere, running it for a bit, twiddling on it in a variety of ways, then moving it somewhere else where someone else can do that too, without too much operations headache as you do so. Again, it's the IDEA of this that matters, not whether it is technically true or not. The idea simply leverages the concepts of writing, deploying and running code and smears them across a the vast resources of which the Internet is comprised.
I've seen the gleam in people's eyes when they talk about a ubiquitous computing infrastructure, of which it will run their code, without question, and never, ever breaks. Irrational beliefs are still beliefs, and sometimes it is necessary to also believe they are rational ones in order to get things done. We're all busy, and the Internet requires a lot of us to get more done in the future. People need the idea of Docker to get project's interest raised, approved, and put into production to get more things done.
This is where Docker's dissonance comes in, and does so fairly heavily handed. Docker is here to make money. They will do that at the expense of all other things, given their purpose is to preserve shareholder value first. That includes changing the idea that everyone holds about ubiquitous computing. That's not to say that shareholder value isn't tied to consumer value and completely rational, but it can also become a vicious circle very quickly when company values are prioritized over consumer values at any expense, or vice versa even.
Personally, this is why I'm not a big of traditional investments in infrastructure technologies. While I do think that traditional investments have been a great help to the growth of technology in the past, it is my belief we are going to continue to run into increasing challenges attempting to rationalize arguments for shareholder value vs. user's best interests when it comes to infrastructure. And, to make it even more challenging, I think most times the users don't realize they are trusting these offerings in an implicit way, which makes their view on matters highly irrational. That is to say, irrationality indicates additional work must be done to resolve the truths of who and what to trust. This is made difficult by applying that irrationality to the very systems on which we base trust.
I don't want to live in a reality where we are having to constantly question the trustworthiness, or irrationality, of our infrastructure.
I don't have a great suggestion for how to fix these types of issues other than proposing a radical change in infrastructure business models. I do have a suggestion on how to do that, but it will make it completely irrational for traditional investors to consider, which would limit the adoption of it, even if it got going. A catch 22, if there ever was one.
This seems to be a much higher level construct , like PaaS or Lambda, than what Docker provides.
Docker was more of a bottom-up wake up call to those upper layers that lower layer technology usability still matters.
So to me it seems fair but depending on your experience I could see it as a cheap shot for upvotes as well.
Here is an old saying: "Nobody ever got fired for buying IBM" - neatly illustrating the issue. I have met _many_ senior people that don't understand these naming differences. They tend to be the bosses of the CEO and CTO's
Basically I wouldn't ignore it.
If you were already migrating to Docker, now you have to explain that the project name changed overnight without explanation, and it's hurting your credibility pretty bad. It's also gonna bring a lot of confusion onto everyone for a very long time.
The three points seem to be, cheaper, more secure, and the other guys are dumb.
I'm sure they do. If I were involved in a situation like that I would definitely check this out, just to make sure that their recommendation is not just a way to protect their income stream. 'Don't look there' is the forbidden fruit theory at work, of course everybody will look there.
More recently, I was part of a discussion where Kubernetes has decided to roll its own logging infrastructure, while leaving important parts like log rotation still undecided.... driving me even closer to the Docker ecosystem.
However, it seems that Swarmkit is a big casualty of the whole CE/EE (and now Moby) split. Docker EE (https://www.docker.com/enterprise-edition) still does not mention Swarmkit anywhere. In fact, there is NO top-level page on Docker's own site that actually mentions Swarmkit [1] and even Docker "Swarm-mode" is part of some second level documentation and help pages.
It makes me really worried about how you are seeing Swarm-mode/Swarmkit as part of the long term future of Docker Inc. You guys really dont talk about it - not even on twitter.
I'm assuming this is your handle as well (https://twitter.com/dockerswarm), which has not seen a tweet since Jan 2017. However, you guys do tweet a lot about "Docker Datacenter".
What's going on ?
[1] https://www.google.co.in/webhp?sourceid=chrome-instant&ion=1...
Can you please refer to this particular discussion and link to it here, I would like to make sure that this indeed is what happened.
The last known proposal is here https://github.com/kubernetes/community/blob/master/contribu...
You can check the rotation part. Btw, yes I am aware there was increased journald support added in 1.6 . And I'm hoping that more reasonable minds prevail.
I get very worried when people say "the format+kubectl cli is more important. we will figure out the mechanics of easy stuff like logrotation later. until then you can use a cronjob. systemd is obviously a monopoly."
The ideas of decoupling logs from lifetimes of the the things being logged, desynchronized log-watcher services that ship logs to central off-machine log stores, logging chains, shims for things that talk to syslog over AF_LOCAL or UDP sockets, shims for things that talk to the systemd journal using its idiosyncratic logging protocol, compartmentalized logging services that are protected by the operating system from compromises to the logged services, and persistent log pipes that do not lose log data; have all been designed and implemented.
* http://jdebp.eu./FGA/daemontools-family.html#Logging
* http://jdebp.eu./FGA/do-not-use-logrotate.html
* http://jdebp.eu./Softwares/nosh/guide/logging.html
* http://skarnet.org/software/s6/s6-log.html#diesyslogdiedie
> Docker is transitioning all of its open source collaborations to the Moby project going forward.
Should I understand that the core team wants to keep the brand "Docker" but use it in a commercial way, while Moby will be the underlying open source code?
Is it Docker/Moby = RHEL/Fedora ? or Docker/Moby = Mongodb.com/Mongodb.org?
Moby = open source development
Docker CE = free product release based on Moby
Docker EE = commercial product release based on Docker CE.
Nothing is dead; and everything that was open-source remains open-source. In fact we are open-sourcing new things.
The difference between them is like
Redhat Enterprise Linux (Costs Money, Enterprise Supported)
vs
CentOS (Community managed, community supported, Libre Free)
https://www.redhat.com/archives/fedora-list/2003-September/m...
This move is going to confuse many users, and could entirely derail their growth. What the hell were they thinking, to execute this decision at such a critical point in time? Docker was already confusing enough to get started with, and now there's an additional barrier to entry just to understand what it all means? People right now at least understand what "Docker" is; now people are going to be redirected to something called "Moby", and many will give up out of confusion.
Really bad timing for such a move.
tldr; Am I even supposed to call it "Docker" now? Years of growing recognition of that name, thrown out the window.
> defining an open, community-centric governance inspired by the Fedora project
I don't think the same applies to Docker Inc. so my initial reaction is step on the brakes here.
If Moby is the name of a collection of subsystems that are drawn upon to build user-ready container platforms like Docker, isn't GNU userland/Linux kernel->(Fedora/Red Hat)->Red Hat Inc. as Moby->(Docker CE/EE)->Docker Inc.?
Perhaps a better way to explain it is that Moby is a "containerization kernel/core" a la Linux or an "open containerland" a la GNU. The term "framework" may be a little more accessible at the cost of diminishing street cred by appealing more to web devs than old-school system devs. These fancy words probably shouldn't be a problem since it seems only developers would be interested in Moby anyway (instead of end users who would want to use a Moby distribution).
With a little bit of finesse, this could've probably been construed as the boon for the open-source community that it seems you originally intended. "Docker Inc. donates its core technology to the open-source community and invites competitors to use it as the base of their own products." Too late to right the ship?
Docker CE will remain Docker CE. In the Fedora/RHEL analogy, think of Docker CE as the recently introduced "RHEL free developer edition". It's still a commercial product - it's just free and open-source.
> With a little bit of finesse, this could've probably been construed as the boon for the open-source community that it seems you originally intended. "Docker Inc. donates its core technology to the open-source community and invites competitors to use it as the base of their own products." Too late to right the ship?
You mean like this? :)
https://blog.docker.com/2017/04/introducing-the-moby-project...
http://www.cio.com/article/3191344/linux/why-docker-created-...
http://windowsitpro.com/cloud/docker-brings-open-collaborati...
https://jaxenter.com/docker-containers-moby-project-133404.h...
Outside of the Hacker News bubble, this is a non-issue. 500 people read the title of a pull request, didn't even read the contents of the change, and clicked on the "thumbs down" emoji. Meanwhile the vast majority of Docker users don't scan the repository for pull requests. And the majority of those who do actually take the time to read the code, and usually have enough context to understand why we are doing this: modularization, openness etc. In between these two groups, you have Hacker News basically.
That pull request was confusing, yes. And we clarified it. But it's an implementation detail and definitely not representative of how the Moby launch was understood by the community.
Moby = open source development
Docker CE = free product release based on Moby
Docker EE = commercial product release based on Docker CE.
Nothing is dead; and everything that was open-source remains open-source. In fact we are open-sourcing new things.Good thing it's all OSS. Don't try to solve problems with a pitchfork, when a simple fork will do.
There's nothing viral about using GPL code. Your docker containers are probably running on Linux, which is also GPL, and that has neither infected nor affected you, has it?
It's viral, any connection whatsoever, not sure where they draw the line. GPL is just not enforced except when it comes to selling software licenses.
Would you not think all these huge companies would think twice before using Linux if it meant being dependent on the good will of everyone who ever contributed to any of thousands of packages that come with a typical distribution?
"I predict that the term "micro" will be replaced by the term "modular", and that docker will create a replacement for the microcontainer named the modular loader/container."
By creating an independent project commercial actors in the Enterprise space can share resources, components, and standards without having to explain why they`re better than Docker even though they`re built on top of Docker.
This is a great move towards industry and container infrastructure orchestration standardization. Common APIs will go a long way to widespread container adoption.
quite to the contrary I'd say. Docker will be their commercial enterprise offering and moby will be the open source project.
Dockers underlying components are quite usable for many solutions in other projects and platforms. Now, instead of having "Docker TM" components in their stack, OSS projects can connect the tools without seeming to support a specific commercial provider.
Presently there is a lot of confusion and OSS projects are hesitant to use Docker components because of the perception of a closed eco-system. Now the project has clearer commercial branding and a more appealing package set for OSS consumers.
Remember: many of the most important actors we need to collaborate in this space are direct commercial competitors to Docker Inc.
But those projects aren't true OSS in the way that something like Docker / Moby needs to be. Your Vagrant example is one of them; Vagrant is a useful tool but ultimately too narrowly-focused. Docker's biggest coup was seamlessly integrating orchestration tools with deployment tools -- and now they're separating the two (which is completely fine, because they need to be separate tools anyway to support complex use cases).
Docker has built a lot of valuable IP that they want to protect (branding, UI, API, etc.) That makes a lot of sense. It also makes sense that they want to open source a big chunk of the "glue" between everything because 1) it's generally useful to lots of people / projects and 2) it gets other companies to kick in dollars for maintenance.
You're not going to get non-customers to fund the maintenance of an open source project that is wholly run and owned by Docker. They know that, and hence they're creating a governance board. But once you do that, you lose full control of your brand (which is attached to the project), and thus you have to divorce the two.
Docker is still going to drive the development of Moby. But over time, other companies will enter the fold to build other complex system orchestration projects on top of Moby.
Not the first time this is done either.
If they had renamed the commercial stuff to Moby they'd have to start from scratch, now there are thousands of articles and links that say 'go get docker' intending to point you to the open source stuff, instead directing you to the commercial stuff.
A very effective bait-and-switch.
That happens ALL the time. Except they (managers) don't read tutorials, they read trade magazines saying that X is the new hotness.
So off they go and buy X Enterprise Edition.
This happens in the corporate world, not everywhere, but that's where a company gets the big contracts.
Not picking on Docker here, it's a great product and good luck to them; just a commentary on corporate fashion driven IT culture.
That's not Docker. That's Moby.
By splitting up the open-source Docker CE codebase into smaller projects, each smaller project is more approachable to outside potential contributors. If Docker can succeed in increasing the number of contributors to the Moby projects, compared to the Docker CE project, then they'll have succeeded in increasing the relevance of their product in their target market, which translates to more customers, sales, and profits in the long-term.
Of course they run the business risk of a fork taking off, but in the real world, forks are arms races. They're easily won against dead or severely mismanaged projects, but against a project with a well-funded corporate sponsor? Few, if any, will have the resources to put into making a better fork than the original.
And like they said, for the vast majority of their customers, nothing is changing. Their customers bought a Docker product yesterday, and they're buying a Docker product tomorrow. That branding isn't changing.
Docker. Ok, works with containers I get... wait. Your logo is a WHALE with a bunch of shipping containers on it. You know that whales DISAPPEAR underwater for hours at a time, and shipping containers are NOT MEANT TO DO THAT?
Ok ok maybe it was an honest mistake hey whats this new thing moby.. wait. Whales. Moby. Moby dick? Your project is named after a mythical white whale that sank boats, the very thing that actually carry CONTAINERS.
Having said all that, based on this [1] summary of Docker's usefulness in production from February, maybe both names are absolutely on-point for the reality of this abysmal 'product'.
1. https://thehftguy.com/2017/02/23/docker-in-production-an-upd...
Funny rambling backed up by a crazy well detailed article and you get downvoted by the HN hive mind. Docker is the ...ahem... whale in the room that you shouldn't criticize ever!
Just updated the titles to include moby.
Maybe I should change the links as well, then it can get reposted on HN and reach the first page along the announcement :D
Just like the stuff that relies on Docker
Moby = open source development Docker CE = free product release based on Moby Docker EE = commercial product release based on Docker CE.
Moby is a set of standardizable tools that underly that product which might be shared with OS suppliers, Kubernetes, or other virtualization solutions.
If you want containers for your apps, you want Docker. If you're bundling containerization components into your Linux distro/orchestration platform/custom hybrid cloud solution then Moby will be your swiss army knife :)
A lot of the community understands that "docker" has gained a lot of features, but they may not understand it is because Docker is the name of an integrated product suite a company wants to sell.
These features are added because Docker, Inc. wants to have commercial offering containing a full technology stack around containerization, a community offering to attract people to their stack, and open source backing the community offering so that enterprises don't get fearful of vendor lock-in.
But other companies want to use containerization. And Docker doesn't mean anything in their context, because they only want to use a few pieces and swap the rest for their own. In fact, they are resistant against using Docker because 1. It is iffy whether they could use the trademark, even if they were ok marketing a competing product 2. They don't have the same voice in the technical decisions of the open source project
Moby is an attempt to solve these issues: - It is a project with a vendor neutral name - Presumably, it will be managed in a way that gives vendors somewhat equal footing - It makes it clearer that the pieces are containers, etc, the project that develops all the open-source pieces is moby, and one of the products that integrates all the pieces is docker.
It is similar to how RedHat split out Fedora, except that RedHat didn't see the value in having a RedHat-branded community edition.
The intention is to have a clearer split between the Open Source community project - which is and will always be the heart of Docker (or now Moby) and the "product" pieces. The Docker product itself is not changing at all from an external perspective - you will still call "docker run" as before. However, Docker will be an "assembly" of pieces from the moby project plus some under the docker namespace, probably including "docker/docker-cli". Going forward, this will allow the docker-cli to make more product-centric moves like further Hub integration that would be too controversial for the community moby project. It also allows other organisations to take the moby code and reassemble it in different ways without offending the Docker "product". Furthermore, the intention is that in the long run, various Moby pieces will move to community foundations such as the CNCF and OCI.
Separating out the shared commodity container stack points towards plug-and-play cross-vendor container stacks based on standardized APIs. Puppet, Chef, Ansible, Azure, AWS, VmWare, Kubernetes, etc etc: todays market is a divergent mess. Being able to mix-and-match configuration tools, clustering tools, and management interfaces will get us to best-of-breed solutions. It should enable plug-and-play components that today, amidst a dizzying array of incompatibility, generally force you onto wholesale stacks.
Props to Docker for investing in the shared eco-system and risking the brand confusion to push the industry towards the next step.
This doesn`t impact users of Docker or containerized app devs, people won`t be setting up a Moby stackexchange site any time soon. This is big news for the devs of container platforms though :)
This seems absurd from a branding point of view.
I personally find this a little bit vindicating. I'm 39, which is like 120 in programmer years. I feel like one of the advantages of being an "old" (in quotes!) programmer is that I'm pretty good at spotting a fad. Unfortunately whenever I point one out I get mocked and down voted to oblivion, so I've learned to just sit on the sidelines and watch the fads go past and just work to avoid them in my own projects.
Programming is very faddish, and I've seen fads come and go. Here's a few of the ones I've seen in my tenure:
- "Design patterns" heavy "enterprise" programming, such as commonly seen in the older work in the Java and .NET ecosystem.
- XML for everything.
- Agile. Oh god, agile.
- Test driven development.
- Dynamic languages (as productivity magic pixie dust).
- Service oriented architecture (SOA).
... I could go on.
Not everything in these fads is bad. Fads often contain good ideas and some fads contain non-fad elements that stick around. But they're fads insofar as they are over-hyped as magic cure-alls.
The key characteristic of a fad is this:
It's heavily hyped as a cure to some set of very hard problems in programming or system administration, but all it really does is move the problem somewhere else or hack around it in some trivial gimmicky way. There is no real innovation. Meanwhile the fad often introduces new problems that nobody thinks about until the shine wears off.
My simple heuristic for recognizing a fad is to ask "where's the innovation?" A real innovation is a conceptual leap forward. It has a certain "meaty" feel to it and seems worthy of at least one solid CS paper. It often reduces complexity, since when deployed you can now dispense with all the mountains of hacks you used to work around the problem prior to the innovation.
Here's some current things that I very strongly think are fads:
- Microservices, which is just a reboot of SOA. The idea is not inherently bad and often results in more scalable systems, but the faddish part is the idea that replacing local API calls with RPC API calls or event queues is going to make some major class of programming problems go away. No, and you've also just introduced a new set of problems around network unreliability.
- "Serverless" cloud, a.k.a. total lock-in to a proprietary mainframe. Everyone doing this is going to regret it in 5-10 years except Amazon's shareholders. It's a roach motel. Compute will get another order of magnitude cheaper and prices for everything else will drop accordingly, but these prices will not since you drank the kool-aid haha.
- Containers.
... yes, containers.
Like most fads, containers are a response to a real set of problems. These are mainly:
- System/VM configuration drift and variability.
- Dependency and DLL hell.
- The fact that Linux/Unix has devolved into a single-tenant operating system where it's hard to deploy more than one thing on one "server." (This debacle is deserving of a whole very long blog post.)
Containers are a fad because they don't address any of those problems with real innovation.
System configuration drift, dependencies, and DLL hell are are addressed by the gimmicky hack of basically tar'ing up whole Linux images and treating them like gigantic statically linked binaries.
If you're going to do that, why do you need Docker/moby/whatever? Just use a static Linux distribution like http://sta.li and run every service in its own home or chroot. Keep your service trees in git and manage systems with Chef, Puppet, or f'ing shell scripts.
I fail to see why orchestration (a.k.a. provisioning) systems like Kubernetes could not work in such an environment without the container cruft.
Containers don't solve multi-tenancy much in practice. Their security profile is not good enough for true multi-tenancy, and if you try to run too many on one server you're going to run into stability problems that derive from all the bugs that exist in all that extra complexity they add.
Real solutions to these problems would... you know... really solve them. A real solution would make it possible to deploy stuff easily and forget about DLL and dependency hell. A real solution would restore true multi-tenancy to Unix, allowing users (not root) to install and run services on commodity identically-configured Unix systems. A real solution would reduce complexity.
Edit:
I do wonder a little if the idea of throwing out dynamic linking, which both containers and some newer compilers like Go do, is not a bad idea. Maybe it's obsolete. Maybe the tiny memory savings of dynamic linking are no longer justified by the complexity overhead of dependency management.
Any language, as it grows and takes on larger tasks, its dependencies will also grow.
Build time dependency management, distribution, and run time dependency management are different phases that would still need to be managed
I'm 36, and have seen most of what your described and more.
And, their competition at the higher level is starting to abstract away from their runtime. Like this: https://github.com/kubernetes-incubator/cri-o
Docker is the name of a company and a container management product (Enterprise and Community Editions) assembled from the pieces in Moby.
1) It's not conflicting with anything major (Unless it does, short 4 letters names are way too common).
2) It will transfer along the URL redirections
3) Docker has enough credit and following to gather SEO on any new domain and product name that comes out
The new pages will be top links on google very quickly. No worries about that, machines are fine!
The trouble is with humans, they can't keep up. All the existing content -documentation, community, articles- will stay under the Docker name, forever splitting the knowledge and bringing confusion onto mankind.
Docker isn't being renamed to Moby, but rather all of the open source components are being spun out into a new umbrella OSS org called Moby.
No need to s/docker/moby anywhere.
This is going to end up being a case study in how not to make announcements, because the news is otherwise big/interesting.
https://github.com/docker/docker
This all seems so unnecessary. Why didn't they just keep docker docker and have docker-xxx for whatever other brands they want to introduce? Reminds me of CoreOS rebranding as Container Linux.
They'd lose 40k+ GitHub stars, I guess.
You appear to be disagreeing with something I said, so I have to assume it's based on the assertion that naming a company AND product the same thing is usually a bad idea, at least for marketing purposes. The confusion that has arisen with other products in the past is real, and it's clear that people are confused with this move. That's why I said it. I didn't say it to be completely precise about what happened, because people aren't precise in understanding how companies market to them. It's a common mistake in marketing, of which I am moderately skilled, or so I hear.
Anyway, I would say, in general terms, that reality is perceived differently from person to person. This means that some may conceptually map a concept to a word in a completely different way than another. That some are confused about this particular change in mapping is a good indication that my recommendation is a sound one, where the product, company, project and repo should be called different things to avoid branding confusion[1] when the consumer is making a decision.
I would also add that this behavior of companies "being confused" in return to people's confusion resulting from a change in branding of something is also an indication of irrationality. And, it's also blaming in tone to assume people should "just get it" when these things are executed on poorly, at least marketing-wise.
This is not nice.
There is no animosity towards kubernetes, quite the contrary. We are working with the kubernetes community to integrate the individual components of Docker, so that everyone can reuse each other's code and ideas.
For example:
- we're actively working on a containerd+CRI integration.
- there's great technical discussions about supporting CNI in SwarmKit
- we are participating in the early CSI (common storage interface) discussions along with the Mesos and Kubernetes communities.
- we are discussing the possibility of participating in the Kubernetes secrets implementation work, since we invested a lot in SwarmKit secrets and some of that work could be reused, or at least discussed for reference.
In short: no animosity. Not sure what happened to you at Dockercon but we will look into it.
Thanks for responding. There really isn't many more details other than that. It was in DockerCon Seattle in the space needle. I don't remember who the employee was. I did generally get the feeling that Docker employees felt at least wary or worse about k8s. That was from more than just that once situation, but from multiple interactions I had over the conference with employees. I only pointed out that one interaction because it made me certain that at least that employee was more than wary.
I'm happily using Docker and it was a good conference otherwise.
I'm no k8s fanboy (and I have the comment history full of replies from annoyed k8s contributors to prove it), but it's because they know that Kubernetes is going to end them. It relegates Docker to a swappable implementation detail.
Docker is out on a limb with the dynamic mesh, nobody else is using the p2p gossip protocol that I know of and there are problems, and Docker paid-for support is just dropping tickets related to networking issues making this work. The service outages caused by mesh gossip problems are simply measureable by NewRelic free ping synthetic monitors. Docker is getting paid--they are falling down on support.
These large companies are developing open source as a strategic asset but most people have no idea and just think they're being altruistic.
The reason this really sucks is because as a result, all the large projects backed by large companies kill of the alternatives which have very pure motivation. Just like react is replacing a pure open standards based way of web development. Sure, it's "open source", but this is not right.
It's fascinating how far we have come in the past 20 years. First the introduction of using opensource at work, then businesses starting to opensource things themselves, and here we are complaining that the top opensource projects are in fact backed by companies.
I understand your point, but it's also a luxury to be able to complain about this.
Docker, on the other hand........
If they can drive adoption, it makes it much easier to move customers away from AWS and on to GCE.
I'm looking forward to them accepting defeat on this one and just natively supporting K8s like Google.
Docker used to be the open source part. Now that's being moved out of the way so when you search for Docker you get the paid offerings. The goal is to confuse the people who haven't been following this into thinking Docker is a thing you pay Docker (the company) for.
It's brilliant.
Brilliant! Brilliant! Bravo!
Following the first thing they google without actually understanding what's involved, all the way to handing over money sounds about what I expect from someone who thinks Docker is a viable platform to use for anything worth handing over money for.
- Decision maker googles and finds there's this Docker company that will solve their problems. Heard good buzz about Docker in the news, how open it is, and how many companies use it.
- Decision maker to engineer: "Hey can we use Docker software to solve problem X"
- Engineer: "Sure, the software is actually called Moby now though"
- Decision maker: "Meh, I don't care what it's called, I just want problem X solved. Docker solves it?"
- Engineer: "Yeah I guess"
- Decision maker: "Ok here's your budget. Pay Docker to solve problem X."
- Engineer: "Alright"
What fucking problem does Docker solve that a "decision maker" can understand it enough to get a result telling them to use Docker, yet s/he can't understand the concept of "open core" software.
Docker's status seems entirely driven by me-too cool kids cargo-culting the bejeezus out of it because they heard it can run their nodejs react app better.
If you work for a company where your described situation could happen, fucking leave.
:thumbsup:
- Boss: here's the business-level problem I want solved
- Engineers: here's what we think it'll take to do it, and the technical path forward
- Boss: either (1) cool, what can I can do to help you get started? or (2) yikes, that's a bigger problem than I thought. What's the 80% solution?