Deis 0.9.0 – Dockerfiles, Domains, and HA Routing
deis.io
deis.io
As a maintainer of Docker I think it's a good thing that you preserve the freedom to run your platform on any number of distros, and are not stuck, for example with having to install systemd everywhere.
I agree a nice aspect of using Docker is that you have the flexibility of using whatever distro you want for each application component... however a heterogeneous container platform doesn't really provide much of a win, and to make that work you just end up implementing your own solution for each of the components that abstract you from the platform.
The whole point of Docker is to introduce simple interfaces to important components of distributed systems, and get a critical mass of people to agree on it, producing interoperability. So far we did this for packaging and sandboxed execution of individual components, with some success. We plan on doing this for other areas in the future, including clustering, stack composition, application networking, service discovery, authentication and access control, etc. This will allow rival implementations to compete in each of these areas, without creating fragmentation. I don't want a World where "CoreOS containers" can't talk to "Atomic containers" or "Mesos containers" because we couldn't agree on a common interface - we can do better than that.
We are pretty open about that plan. The Deis guys know this and are making the bet that embracing "Docker-native" interoperability is good for them.
I think you definitely want to be very open about what the inside of the container looks like and even what the interface is between the containers and the rest of the world.
I was thinking about the platform that the containers run on top of. You don't gain anything by using a given host platform, but building alternate components that handle deploying and wiring up those containers. You're better off either using the existing platform's components or contributing patches to it, and the latter really only makes sense if you're adding something distinctive.
Well of course! It is your own project.
> If you think of CoreOS as an end-to-end monolithic platform, this may seem strange.
Yeah, but I'm not looking at CoreOS as an end-to-end monolithic platform. I'm looking at it as a bunch of components that happen to be bundled together, each of which is pretty nice all by itself. It doesn't mean you have to use them all, but on the other hand, you would hope that if you were going to replace one of them entirely you'd go with something with a significant advantage (and I'd argue in some cases it has gone the opposite way).
> What you want is a vast ecosystem of small composable lego bricks.
Understood. I just don't think you help yourself much if you take out one brick and replace it with one of your own, particularly if the new brick isn't any better than the old one (and might be worse).
Deis was architected before the CoreOS integration. While we love CoreOS, architecturally we need to remain scheduler agnostic. Probably a good opportunity for a blog post...
> the use of rsyslog in lieu of systemd seems like really going against the CoreOS grain
We view platform logging as separate from application-level logging. Aggregating logs through the systemd journal is possible, but it would tie us a little too closely to systemd. Using log sidekick containers (as we do) is more portable and easier to funnel to end developers.
Okay this makes sense. If I were in your shoes, I might be tempted to radically reexamine the architecture in light of CoreOS's design. Otherwise you end up duplicating work or working at cross purposes, and CoreOS's components (systemd, etcd, fleet, etc.) are pretty cleanly separable from both CoreOS and each other already, so you shouldn't end up being married to CoreOS should there be some kind of problem.
Given that you're maintaining a level of agnosticism, what do you see as the main thing you are gaining by integrating CoreOS?
> We view platform logging as separate from application-level logging.
One can certainly take that view, but I'd argue it does lead to more complexity for managing logging.
> Aggregating logs through the systemd journal is possible, but it would tie us a little too closely to systemd. Using log sidekick containers (as we do) is more portable and easier to funnel to end developers.
Fair enough, but if you really want to be agnostic, I'd say have the contract that in the end everything goes in to a syslogd of some kind, but have the default approach to getting that job done be via systemd (with /run/systemd/journal/syslog to get it the rest of the way). If someone wants to use rsyslog or something else to do the job, so be it.
While using the log sidekick approach is popular, it's actually quite inefficient and clumsy. Particularly on magnetic disk, you've got this nice, efficient, append only per-event log writing that doesn't even have to be cached and then you periodically scrape through the old stuff as if it were just a blob of bytes.
The big win though is that you no longer need a sidekick for managing logging from each container. No muss no fuss about where the logs are, etc. Instead you just direct logging to systemd (or if need be, the syslogd socket). No sidekick required!