> Kubernetes is not really a container orchestration platform.
Since Kubernetes schedules, I'm curious what makes you say this. Kubernetes without Mesos works perfectly well for the use cases that both CoreOS and Google envision.
Since Kubernetes schedules, I'm curious what makes you say this. Kubernetes without Mesos works perfectly well for the use cases that both CoreOS and Google envision.
And yes, as noted above I work for Mesosphere. YMMV.
Don't get me wrong, I really like mesos. I think it's a fantastic piece of technology and I really want to see it do extremely well. I just think Kubernetes is something a little more accessible to the wider developer community. Plus we get to see it grow from infancy and help contribute to that growth. AND its written in Go, which is a major plus.
Mesos and Mesosphere talk a lot about scale, with the implication that the technology you're modeling powers the entire Google fleet in one big harmonious system. It doesn't. There's a fairly hard upper limit on the system you mentioned in terms of scale, which is one of the motivations for Omega. The system you mentioned also fits into an ecosystem from a DLM[0] to a RPC system to a build system and on up. Pieces of that are gradually being open-sourced, too, like Bazel. Google is clearly becoming aware that their infrastructure being generally available will benefit the industry enormously, and regardless of my personal feelings on their business model and exploitation of user data, that is a good thing. If you're telling me that Mesos can do better than all of that expertise, better than John Wilkes, better than all of that, to the point where you don't even consider Kubernetes on the same path as Mesos, I don't know what to tell you other than come back to reality. This seems to be a rut with Apache projects that model Google systems in some way: Zookeeper, Mesos, Hadoop, Flume. All of them look like if I described the systems in question over a very blurry fax, and the open-source community did its best. That's not outright criticism, it's just having the unique experience of having seen their Google counterparts; it's a good thing they exist, but there are absolutely opportunities to do better, Mesos included.
I get what you're trying to do with Mesos, but I strongly disagree with your characterization of Mesosphere as a datacenter operating system. It's disingenuous, and positioning Mesosphere as the only company who can do hyperscale in containers is a tough sell. Last time I used Mesos, in 2012, it could barely handle 100 machines and I had to write shell scripts to even get a simple static application scheduled. I then threw 10,000 jobs at it and the scheduler hung and took down the entire cluster. Clearly you have come a long way since then, but it's apparently easy for you to forget that given the age of Kubernetes.
[0]: You'll pardon me if the thought of Zookeeper doesn't excite me as the backbone of the entire system.
I'm biased, of course. I work for Pivotal, we're the lead builders of Cloud Foundry. Our current generation of software can spin up hundreds to thousands of machines without too much fuss. Our next generation[0] is ... better.
And again, this is my bias showing, but I think our design for container workload scheduling is better than Mesos and Kubernetes.
We don't get much buzz on HN because we sell a complete PaaS to F500s. It's not very approachable on a personal level.
Lattice is a selection of Cloud Foundry components, principally the Diego runtime system, Loggregator log streaming system and Gorouter HTTP routing system.
If how software is written matters to you, then you might like the way we work. Cloud Foundry is Pivotal Labs DNA, scaled up. 100% pairing, every new line of code is test-driven.
We dogfood everything by running Pivotal Web Services as a public PaaS, which is always within a week or two of HEAD across the board.
Where we follow is that Kubernetes is extracted from real experience and Mesos has a head start on implementation.
In terms of design, I like ours best. Tasks and long-running processes are distributed using an auction mechanism, which means that there's much less global resource-status chatter required to make the scheme work. Diego is "demand-driven", meaning activity occurs when new demands are made. Mesos is "supply-driven", meaning state-affecting activity occurs when resources become available. They both solve a critical problem by pushing intelligence out to the edges, but to different edges.
I am much less familiar with Kubernetes, so I will avoid being any wronger than usual.