Scaling Mesos: A Perspective Featuring Mesosphere, Airbnb, Ericsson and Others

Schedule:

6:30 - 7:00 Networking and Light Dinner

7:00 - 7:10 Announcements

7:10 - 7:40 Mesosphere: Deploying Docker Containers at Scale with Mesos and Marathon

Abstract: The norm these days is to operate apps at web scale. But that’s out of reach for most companies. Marathon is an open source project that makes it considerably easier to deploy Docker containers at web-scale using Apache Mesos.

See how Mesos and Marathon can help deploy and manage Docker containers at scale and how the Mesos cluster scheduler builds highly-available, fault-tolerant web-scale apps.

Bio: Sunil studied Computer Science at the University of Cambridge, where he wrote his dissertation on delay-tolerant networking systems. After four months spent cycling across Africa he returned to London to join a major investment bank and then Last.fm as a Data Engineer. Last year his interest in robotics brought him to UC Berkeley, where he completed a Master of Engineering degree with a focus on distributed computing and unmanned aerial vehicles, working in his spare time on an UAV startup. After leaving Cal, he joined Mesosphere as a Distributed Applications Engineer and is currently working on easy deployment of Mesosphere clusters on Google Cloud Platform.

7:40 - 8:10 Airbnb: How we're using Mesos at Airbnb today

Abstract: Mesos is the operating system for Airbnb's data platform. We use Chronos, Marathon, Hadoop, and other frameworks for operating a complex system of data warehouse and analytics tools. In this talk, we'll discuss how some of the day-to-day operational problems are solved using these systems.

Bio: Brenden Matthews is a software engineer at Airbnb, working on the data infrastructure team. He is an Apache Mesos contributor, the primary maintainer for Chronos and Hadoop on Mesos, and has also contributed to related frameworks such as Storm, Spark and Marathon. Previously, he has given talks at about Mesos at Twitter, QCon, and MesosCon.

8:10 - 8:30 Ericsson: On the use of Mesos and Marathon for Multi-Datacenter Service Orchestration

Abstract: With the availability of numerous geographically distributed datacenters, there is an increasing need for highly distributed service orchestration. Apache Mesos and Marathon are valuable starting points for further exploration towards a system enabling such service orchestration while achieving high datacenter utilization and providing high availability. However, both frameworks are designed and used for (resp. on) a single datacenter.

In this talk I will present prospectives on leveraging Mesos and Marathon as building blocks to orchestrate Docker-based services across multiple datacenters.

Bio: Eugen is a Senior Research Engineer at Ericsson Research Silicon Valley Lab. He currently builds systems which leverage real-time big-data analytics for improved resource management in large-scale datacenters. Prior to that, he was a postdoc in the Data Intensive Systems Group of the Advanced Computing for Science Department at the Lawrence Berkeley National Lab (LBNL). He holds a PhD in Computer Science from the University of Rennes 1, France which he received while working in the Myriads INRIA project-team. During his PhD he worked on systems and algorithms for autonomic and energy-efficient resource management. Eugen received his Bachelor and Master of Science degrees in Computer Science from the Heinrich Heine University Duesseldorf (HHU), Germany. He was a student research assistant in the HHU operating systems group, interned in the Myriads INRIA project team and was a summer intern at the LBNL.

8:30 - 9:00 Common Q&A

I'm very interested in how Mesos can serve up http in a Highly Available way to all these services that run internally in the cluster. Has that problem been solved and will that be on topic at the meetup?

Awesome, curious to see how Airbnb used this with Hadoop