Why Docker and CoreOS’ split was predictable | Hacker News Reader

Why Docker and CoreOS’ split was predictable

danielcompton.net

201 pointsdantiberian33 comments

Great post. I completely agree, and was making this same argument to someone yesterday. There are a few important trends and facts:

- Containerization allows more tenants-per-server than virtualization

- The entire business model of cloud computing is maximizing tenants-per-server

- Because containerization has a direct cost benefit to cloud providers, they will adopt it quickly

- Quick adoption of containerization will trigger a "new wave" similar to when Linux gained mass adoption

- Cloud companies are scrambling to invest in hot containerization rather than build their own

- Docker has developer street cred and a solid team

- Post-dotcom, post-web2.0, companies invest in open source software now. This is new.

- Cloud companies compete with each other

- Different cloud companies invested in Docker, CoreOS

- Docker, CoreOS are competing

My logic is a bit fragmented, but all the necessary facts are there, and the conclusion that logically follows from them is that it makes rational business sense for CoreOS to diverge from Docker. There's billions of dollars at stake in the latest "platform war" (previous wars include desktop, server, mobile) -- the battle for the cloud. Containerization directly reduces costs and the "metric that matters" for cloud hosts, tenants-per-server. Therefore any open source company with a good containerization solution is going to attract a lot of investment. But bigcorp A does not want to rely on strategic direction of bigcorp B. Therefore they diverge their open source offerings.

It makes sense from a business perspective, and it's kind of sad from an open source perspective, but also good if you believe in free market economics. The question is, how much do free market economics apply in open source?

Containerization directly reduces costs and the "metric that matters" for cloud hosts, tenants-per-server.

the flip side of this coin is that it reduces the cost for every cloud provider. In essence containers moves the locus of competition further up the stack, commoditising the layers below. I'll bet there were a lot of people who were upset when shipping containers were standardised too.

I'll bet there were a lot of people who were upset when shipping containers were standardised too.

There's a GREAT book on the subject:

"The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger"

http://www.amazon.com/Box-Shipping-Container-Smaller-Economy...

This is a fantastic book. I reviewed it last year: https://www.petekeen.net/book-review-the-box

What I found most interesting was how long it took to come to agreements on "standard" sizes. Lots of infighting and politics.

>The question is, how much do free market economics apply in open source?

This is something I ask myself a lot (as someone focused 100% on open source at a company whose core business is not open source support).

For all the hype that open source has enjoyed as a business model, it is much smaller/untested/too early to tell compared to the proprietary model. The most financially successful open source company to date is Red Hat, which, at 10B USD+, is a great success by anyone's measure. But that's the best any open source pure play company has done. If you look in the other direction, Oracle is worth 180B USD, and even a fledgling BI company like Tableau is in the high single digit billion.

As much as I'd like open source companies to be an Oracle-grade blockbuster success, the brief story is not exactly promising.

As well as Red Hat has done, they are dwarfed by many other companies. This post about open source compressing the market size by 10x made a lot of sense, and was also a little depressing. http://kellblog.com/2014/11/18/it-aint-easy-making-money-in-...

But isn't that exactly how innovation and disruption works? Taking a market and saying "we'll do it for 10x less".

It depends what kind of innovation. Disruption often involves business or marketing innovation combined with technical non-innovation. Shrinking a market by 10x can be a good way to ensure that there won't be enough profit left to invest in R&D.

I guess it's a question of what the company is selling. Google for example uses huge stacks of open source to run their business. And even though they are far from being pure open source, they do contribute back to the community in multiple ways, for example with contributions to the Linux kernel and open source projects like Chromium and Go. And then there are projects like Qt, that mix open source tools with enterprise support.

But I think that the field of IT is still extremely young and that in the future larger companies will discover that collaborating on open source platform and infrastructure projects will give them much better synergy effects and lower IT costs by removing the license cost and at the same time lowering the support cost. Compared to history, the 20-30 years that mainstream computing has is really just a spark and there is so much change coming that will be directed by market and competition.

re: your Google example, that's why I said open source "pure play". Of course, we wouldn't be where we are without all open source projects that sprang up in the last few decades.

And yea, it's too soon to tell. Software & software business are evolving rapidly under our feet. I am curious to see what will happen next.

I think that's the wrong comparison, because companies like Redhat are pretty much by design forcing down margins in the spaces they are operating in by lowering the bar to compete. E.g. to compete with Redhat, you can start by learning their products and offering consultancy services on them, and then gradually forking each one of them to offer specialised products; but conversely so can everyone else.

We should not expect to see open source companies get to Oracle sized valuations; instead we should expect to see them chip away at the market share of companies based around proprietary software until said companies margins shrink, and either their valuations shrinks with them or they expand into new areas in time to delay it.

But ultimately we should expect larger and larger parts of the software space to be commodified and margins driven down for core features like this.

I consider Twitter to be an open-source company. The amount of open-source work they sponsor and incubate surpasses that of most traditional vendors.

Their business model is also a lot better. They have a nice unrelated service they make money off of instead of trying to charge for enterprise support or licenses.

conversely, look at companies like facebook, google, and walmart who have all made the decision to open source their code. software and non software companies alike recognize that contributing their work to the world adds value to their own product.

i think in the future you will have more non technology first companies, like walmart, with strong software divisions promoting and funding open source software. they dont need to make money directly off selling the software when they can make money off what the software does.

I think valuation is the wrong thing to measure here. Open source companies do not seek to maximise value capture. The surplus created through open source by its nature are spread throughout the user base and ecosystem. Perhaps Red Hat has produced 100B+ of value but only is valued at 10B or whatever because that is the part they have been able to capture effectively with their business model.

> - The entire business model of cloud computing is maximizing tenants-per-server

this is not exactly true. the business model is to maximize the revenue per server, not the number of tenants. this can be done in a number of ways.

Good point. Of course, one of those ways is by charging the same price-per-tenant but increasing the number of tenants-per-server. If multiple companies do this, we should expect the price-per-tenant floor to decrease.

I liken containers to adding an extra row of seats on a commercial aircraft.

Right now the airline (host) operates one aircraft (server) and can fit X number of seats on it (split due to both consumption but also overhead from the seat's depth (hypervisor overhead)). If containers have lower overhead than hypervisors then it would be akin to an airline buying thinner seats and getting an extra row in.

However it does remain to be seen how much more efficient containers are over hypervisors, and also how secure they are (there are lot's of unanswered questions on the security front).

I don't think most people are comparing containers to hypervisors (VMs); they're comparing using just VMs to using containers on top of VMs. Especially, they're comparing running one-VM-from-some-IaaS per workload, to running several of those workloads (possibly from multiple tenants) on top of the same VM-from-some-IaaS.

If it wasn't insane to run a hypervisor within a hypervisor, this platform war would have started years ago, with Heroku taking AWS VMs and splitting them into tiny Heroku VMs.

My understanding of Amazon's new container service is that it lets users spawn a bunch of instances that the service manages (job scheduling and so on). This is not maximizing tenants per server nor maximizing revenue per server, IMO, as it mainly lets users increase the workload per instance. Without the container service, unless they run the similar infrastructure themselves, users would have to run additional instances to have the same isolation and ease of deployment (if they use Puppet, Chef, Ansible, or whatnot instead of some Docker cluster management software).

This analysis is spot on. There is nothing wrong with what Docker or CoreOS are doing. As the OP pointed out, stuff like this happens all the time in computing.

What makes it awkward is that you can't say this stuff and hope to convince the critically important group of people for open source software/companies: user communities.

Users care a lot about their software and community around it growing and improving, but they won't be convinced by the needs of venture-backed, growth-oriented corporate strategy. (I am not saying this is good or bad but stating the fact).

I appreciate the author's insights of taking previous examples of technology shifts, extracting a generic pattern from them, and then applying it to the new Docker direction.

But I don't fully buy into it.

To me, the new emphasis on "orchestrating & managing" containers is an obvious and unavoidable progression of taming new layers of complexity as they appear. The lower level technology of containers solved one set of complexities (app deploy onto 1 operating system), but they also create new complexities: tame the explosion of a thousand containers in the data center.

How is this different from MS System Center, HP OpenView, and IBM Tivoli as managers/watchdogs of Windows & Linux boxes or KVM-over-ip to remotely reboot/powercycle racks of servers? Yes, those "enterprisey management console" products are "value added" and successful. Yes, there was competition between OpenView & Tivoli and competition between KVM vendors. But they weren't the kind of products that captured mindshare as glamorous; it was just something the IT department picked to "keep the lights running". Those management products also did not mint billionaires (millionaires yes, but not billionaires.) So the recent analysis from blogs that pits Docker vs CoreOs as a "strategic-winner-take-all" platform war like Internet Explorer vs Netscape doesn't feel quite accurate.

It seems like the more interesting software fight is datacenter cloud o/s which is even higher level than Docker's orchestration-of-containers focus. Example of this would be OpenStack[1]. Proprietary ones would be Facebook internal cloud stack and Amazon AWS. This "datacenter-as-a-platform" would subsume the container standards fight.

There may be something I'm missing in the Docker vs CoreOS that makes me discount this competition while others are playing it up. For example, is there some plugin architecture that would lock in developer mindshare?

[1]http://en.wikipedia.org/wiki/OpenStack

It seems like the more interesting software fight is datacenter cloud o/s which is even higher level than Docker's orchestration-of-containers focus.

I hadn't thought about it, but this makes a lot of sense as the next place of competition where value can be added. It's also where CoreOS is trying to be. It would also suggest that the level above that is abstracting datacenter's away to just manage a single global system made up of multiple DC's.

There may be something I'm missing in the Docker vs CoreOS that makes me discount this competition while others are playing it up.

Docker and CoreOS aren't strictly competing with each other, CoreOS wanted to use Docker as a commodity component (as it was designed), Docker eventually realised that becoming a commodity businesses wasn't going to be very profitable and moved into CoreOS' territory.

> It would also suggest that the level above that is abstracting datacenter's away to just manage a single global system made up of multiple DC's.

I work on Cloud Foundry. We already have this tool, at least for the IaaS layer. It's called BOSH[1], and it allows no-downtime deploys and updates of very large PaaSes (such as our own Pivotal Web Services).

In our reference Cloud Foundry configuration, cf-release[2], we define a deployment manifest that can install a Cloud Foundry instance that spans two AWS availability zones.

We dogfood this system every week. Sometimes twice per week. Our entire Pivotal Web Services system is updated, live, online. And basically, it's very rare that anyone notices. Which is awesome.

[1] https://github.com/cloudfoundry/bosh

[2] https://github.com/cloudfoundry/cf-release

The interesting fight is actually the next level up from OpenStack (or AWS, or vSphere). It's in the PaaS space: Cloud Foundry and OpenShift are sprinting to be the full-service cloud an enterprise can run in a private datacentre.

I have a dog in this fight -- I work on Cloud Foundry. Naturally I think CF's tech is better, especially our next generation of controller/staging/execution/health management (Diego).

When working with FreeBSD jails a decade ago, I always thought they were enough to build my servers and needed just more polishing. Then hypervisors caught on and it was the virtualization era.

Now soft-virtualization is coming back and it's becoming popular to say "hypervisors are overhead".

It's funny how we could have taken a different direction ten years ago but I think I understand. We needed to go through the "VMs will run any of your existing software, don't worry". With mass adoption, new software can start to be written to run inside soft-virtualization (with not that many modifications, if any).

If only.. oh well.

Click here if you get an 404: http://danielcompton.net/2014/12/02/modular-integrated-docke...

Back up now. Sorry about that.

Author changed the post title I think, the link is now dead. http://danielcompton.net still shows the article in first position on his blog.

Some post management didn't go quite right, thanks for the tip, it's back up now.

I have a different theory.

Originally Docker was going to be a container and CoreOS was a platform to run containers. They were both pieces in a traditional software stack.

Docker had a round of funding and the investors pushed Docker in a different direction: App store. Docker was no longer just a piece in the stack and CoreOS could see the writing on the wall: libOS. Docker would marginalize CoreOS the same way LXC was marginalized.

CoreOS is going on the offensive because they were tipped off on Docker's hand.

Occam's Razor.

So which platform can one use to deploy docker cloud? We already have a set of CoreOS servers running docker containers... with rocket replacing docker and automagic updates in coreOS this seems kinda dangerous.

Great analysis, thank you.

Reply on news.ycombinator.com