preillyme

And then?

Distributed computing is now the norm, not the exception, and we need a data center operating system that delivers a layer of abstraction and a portable API for distributed applications.

Well the main idea behind Mesos is to create an aggregation layer that spans server capacity in local and remote datacenters. This aggregation does not tightly bind the servers into a shared memory system, but rather treats them like a giant resource pool onto which applications can be deployed. It is a bit like the inverse of server virtualization really.

While not a complete data center operating system, Mesos, along with some of the distributed applications running on top, provide some of the essential building blocks from which a full data center operating system can be built: the kernel (Mesos), a distributed init.d (Marathon/Aurora), cron (Chronos), and more.

Interested in learning more about or contributing to Mesos? Check out mesos.apache.org and follow @ApacheMesos on Twitter. We’re a growing community with users at companies like Twitter, Airbnb, Hubspot, OpenTable, eBay/Paypal, Netflix, Groupon, and more.

It’s still far too early to tell whether Rocket’s container technology will catch fire the way Docker’s did. Regardless, the pressure is on Docker to distinguish itself now that it has a competitor that seems to be coming out guns blazing.

I'd love to see a Windows version for sure.

I'm really glad to see that I don't need a DNS server inside the VM in the new release.

Yeah glibgil I hope that everything ends up back in Docker. I don't really like having so many different approaches to solving the same thing.

Docker has a new competitor (wired.com) https://news.ycombinator.com/item?id=8682794

CoreOS is building a container runtime, Rocket (coreos.com)

https://news.ycombinator.com/item?id=8682525

Some of Docker’s earliest supporters now believe that the company behind the technology, also called Docker, has strayed from its original mission, and they’re exploring a new project that aims to rebuild this kind of technology from scratch.

I like the main features of LXD; secure by default (unprivileged containers, apparmor, seccomp, ...), image based workflow (no more locally built rootfs), support for online snapshotting, including running state (with CRIU), support for live migration and a simpler command line experience.

Docker versions 1.3.0 through 1.3.1 allowed security options to be applied to images, allowing images to modify the default run profile of containers executing these images. This vulnerability could allow a malicious image creator to loosen the restrictions applied to a container’s processes, potentially facilitating a break-out.

Up to and including version 1.3.1, was vulnerable to extracting files to arbitrary paths on the host during ‘docker pull’ and ‘docker load’ operations. This was caused by symlink and hardlink traversals present in Docker's image extraction. This vulnerability could be leveraged to perform remote code execution and privilege escalation.

I love that in https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2014-6408 Trevor Jay says, "Red Hat does not support or recommend running untrusted images".

Besides the above CVEs, the 1.3.2 release allows administrators to pass a CIDR-formatted range of addresses for '—insecure-registry'. In addition, allowing a cleartext registry to exist on localhost is now default behavior. This change was made due to user feedback following the changes made in 1.3.1 to resolve CVE-2014-5277.

Docker versions 1.3.0 through 1.3.1 allowed security options to be applied to images, allowing images to modify the default run profile of containers executing these images. This vulnerability could allow a malicious image creator to loosen the restrictions applied to a container’s processes, potentially facilitating a break-out.

Docker 1.3.2 remedies this vulnerability. Security options applied to images are no longer consumed by the Docker engine and will be ignored. Users are advised to upgrade.

The Docker engine, up to and including version 1.3.1, was vulnerable to extracting files to arbitrary paths on the host during ‘docker pull’ and ‘docker load’ operations. This was caused by symlink and hardlink traversals present in Docker's image extraction. This vulnerability could be leveraged to perform remote code execution and privilege escalation.

Docker 1.3.2 remedies this vulnerability. Additional checks have been added to pkg/archive and image extraction is now performed in a chroot. No remediation is available for older versions of Docker and users are advised to upgrade.

Related vulnerabilities discovered by Florian Weimer of Red Hat Product Security and independent researcher, Tõnis Tiigi.

Docker support in Mesos: * Users now can launch executors/tasks within Docker containers. * Mesos now supports running multiple containerizers simultaneously. The slave can dynamically choose a containerizer to launch containers based on the configuration of executors/tasks.

Yeah it should be pretty nice for sure. Try it out and let us know what you think for sure.

Frameworks are now able to reconcile task state through the mesos API, to ensure that they remain eventually consistent in the face of failures.

The TaskStatus associated with a status update has relied on a single enum and free-form string to communicate the status. There are benefits to adding more data to the status update, and in Mesos 0.21.0, we have introduced the notion of a Source and Reason.

The source of a status update is, broadly speaking, the actor that changed the status. This can currently be Master, Slave, or Executor. Similarly, we have added an enumeration of reasons for the status update. The list can be found in include/mesos/mesos.proto and is fairly extensive.

Our hope is that frameworks can use the source and reason to better communicate status updates to end users, and make better scheduling decisions.

We have also introduced the notion of a TASK_ERROR state, distinct from TASK_LOST. The semantic difference is that tasks that are updated as lost can be rescheduled and should succeed, while tasks with status error will continue to fail if they are rescheduled. In Mesos 0.21.0 we do not send TASK_ERROR but it has been defined so frameworks can prepare to receive it. We will start sending it in Mesos 0.22.0.

Mesos 0.21.0 also introduces an optional pid namespace isolator for the Mesos Containerizer. The isolator runs each container in its own pid namespace such that processes inside a container will not have visibility to processes running on the host or in other containers.

The Mesos Containerizer Launcher has been updated to use the pid namespace to terminate all processes when destroying a container. This avoids known kernel bugs and race conditions when using the freezer cgroup. If the container is not running inside a pid namespace (started with an earlier slave version), the launcher will fall back to the prior behavior and use the freezer cgroup.

This isolator is only available on Linux.

Mesos 0.21.0 introduces an optional shared filesystem isolator for the Mesos Containerizer. The isolator does not provide full filesystem encapsulation like Docker. It is intended for deployments where containers share the host’s root filesystem but certain parts of the filesystem should be made private to each container.

One example use-case is a private /tmp directory for each container. Processes running in the container will see a normal /tmp directory but it actually refers to a subdirectory of the executor work directory, e.g., the relative path ./tmp. Processes will not be able to see files in the host’s /tmp directory or the /tmp directory of any other container.

This isolator is available only on Linux.

Mesos modules provide a way to easily extend inner workings of Mesos by creating and using shared libraries that are loaded on demand. Modules can be used to customize Mesos without having to recompiling/relinking for each specific use case. Modules can isolate external dependencies into separate libraries, thus resulting into a smaller Mesos core. Modules also make it easy to experiment with new features. For example, imagine loadable allocators that contain a VM (Lua, Python, …) which makes it possible to try out new allocator algorithms written in scripting languages without forcing those dependencies into the project. Finally, modules provide an easy way for third parties to easily extend Mesos without having to know all the internal details.

I really enjoyed the line, "Don’t let old assumptions determine how you live your life today".

This is really good news for the Phacility folks... for sure. Nice work folks.

With Docker, in regards to PaaS support, we are expecting developers to bring their own pre-built Docker images. This unfortunately means that we are going backwards and now telling developers that they must create their stack themselves outside of the platform. The low-level system dependencies within the container are once again in the domain of the developer. Time spent figuring this stuff out is redundant, not the best use of this engineers expertise and therefore prone to error.

The ability to statically type in the most recent version of HHVM, via their internally developed Hack “language”. Hack is a derivate of PHP, but supports static typing. You start files you want to type check before compilation with <?hh instead of <?php, and the compiler does the rest. This approach further increases the quality of compilation, making native code that much more efficient, and HHVM’s PHP that much faster.

Docker Containerizer

Mesos 0.20.0 adds the support for launching tasks that contains Docker images, with also a subset of Docker options supported while we plan on adding more in the future.

Users can either launch a Docker image as a Task, or as an Executor.