DigitalOcean partners with Mesosphere | Hacker News Reader

DigitalOcean partners with Mesosphere

techcrunch.com

140 pointsneom33 comments

This seems completely insecure. I just spun up the small cluster, and it does nothing to firewall off or isolate the instances from the internet?

Eg, Zookeeper is listening on port 2181 on the public internet?

How is this OK?

Use iptables. I know it's a glib answer, but that's how hosting always worked until AWS came along and introduced filtering from an interface outside of the control of the OS.

That is a horrible glib answer.

I was supposed to get a EASY TO USE, READY TO GO, Mesos development environment. So now I have to ssh to 5 boxes and manually add iptables rules? Why don't I just install Mesos myself then?

Honestly this whole thing just reads to me as a marketing releases, not something I would recommend anyone to use.

If the answer is 'use' iptables, fine, but then I would expect the automation that builds my cluster to do it for me, not leave me with a $200 a month cluster just waiting to get hacked.

Digital Ocean has never, since their release, provided anything impressively technical with regard to their infrastructure. It's not surprising that this is half-assed as well.

>If the answer is 'use' iptables, fine, but then I would expect the automation that builds my cluster to do it for me, not leave me with a $200 a month cluster just waiting to get hacked.

Did you confirm that the tooling is indeed missing?

Yes, iptables is installed as a dependency of Docker, but is not configured to do anything:

		$ iptables -L -v
		Chain INPUT (policy ACCEPT 2172 packets, 459K bytes)
		pkts bytes target     prot opt in     out     source               destination

		Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
		pkts bytes target     prot opt in     out     source               destination
		0     0 ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
		0     0 ACCEPT     all  --  docker0 !docker0  anywhere             anywhere
		0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere

		Chain OUTPUT (policy ACCEPT 2350 packets, 461K bytes)
		pkts bytes target     prot opt in     out     source               destination

Linode doesn't do this either. Nor do I want it to. It's my cluster and I should be responsible for firewalling it.

Given Linode's atrocious security record who cares what they do.

The default should always be to listen on internal interfaces or have firewalls pre-configured.

I would love to see someone quantify the value in this through a tutorial or real world use case. I am Having a little bit of trouble wrapping my head around it.

Does this have enough security baked in to run a production web app? Still trying to get through the marketing speak on on the page...

You know that mesos runs the majority of twitter's production web services, right? Take a look at this for an example:

https://www.youtube.com/watch?v=Hal00g8o1iY

Why is this awesome?

Production ready Mesos hosted in the cloud.

Now I dont have to run a $500M company to cut server costs and save dev ops time. Apache Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.

The traditional way of organizing your servers requires a certain number of servers per cluster. For example: 50 webservers for your web app in one cluster, 5 servers for your database in another cluster, and 10 servers for redis, etc. On average, you'd utilize 50% of each cluster, but have the extra capacity for spikes.

Mesos makes your servers way more efficient by treating them as raw power, allowing any server to run any kind of app/task. It chops a server up into many linux containers and can shuffle around your tasks, so that a web app, redis server, and database server could all be running in different areas of the same "physical" server. I read that one company saved ~40% of their server costs and served the same load just by switching to Mesos.

Marathon is made by Mesosphere and acts as the brains or controller for your servers. You can allocate what percentage of resources go to certain apps/tasks, and it also handles deployment, scaling, failover and restarting.

Mesos supports Docker, so converting an existing app to run this on DO with Mesosphere is simple. Some applications are built for Mesos already, like Spark (MapReduce replacement), and will be inherently faster without the Docker overhead.

I found Mesosphere's tutorial useful for setting up a dev environment and trying things out - https://mesosphere.com/docs/getting-started/playa-install/

>Apache Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.

I love Apache Mesos as much as the next person, but this isn't true. It's predominantly used by Mesos devs. It's one of the up and coming hot technologies, but it hardly has enough users to say it's predominantly used by anyone more than enthusiasts and devs.

It's powering huge infrastructures as we speak; Twitter, Airbnb, and many others: http://mesos.apache.org/documentation/latest/powered-by-meso...

Last time I checked very few of them relied critically on Mesos (Twitter, some parts of Netflix, and Airbnb did IIRC). Most are just test deployments or non-critical applications. (e.g. I know Ebay and Paypal are also OpenStack users because they try all the new things).

Other than those, none of those are really huge companies. I have nothing against Mesos, but I hate overhyping something when it's still clearly in an early adopter phase. The side effect of overhyping is that the tooling isn't mature/simple enough for less ambitious people so they get a bad association with the project because they tried it too early after someone implied that it was mainstream already.

>>> Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.

Mesos is used by companies of all sizes in their own datacenters and in public clouds, and in production.

URX and Sailthru are merely two examples of growth stage startups that are operating on Mesos. It's not just about improved resource utilization, it's also about fault-tolerance, high availability, ease of operations and developer workflow.

Sailthru doesn't operate on Mesos. While it does utilize it, it is in a very specific part of the company.

Security baked in? Infrastructure services generally require that you configure and deploy your own infrastructure, including security measures. AWS still definitely allows you to very easily deploy very insecure things, and you still need to do a lot manually to tighten things up.

So it does not peer the machines in any type of virtual private cluster similar to AWS security?

How does Mesosphere differ from CoreOS? I haven't worked with either one, but as a casual observer it seems like there's a bit of overlap between the two platforms. Is that a fair assessment, and can anyone comment on the maturity of one solution vs the other?

Mesosphere is based on Apache Mesos, Marathon and other technologies. Mesos is an abstraction of the underlying hardware - exposing APIs to allow running tasks in a cluster ~ think of just launching threads on your linux box - with Mesos it's similar, you can launch processes on your servers across your datacenter or cloud. It's production software that powers Twitter, Airbnb and other companies at scale. Mesos is more than just orchestration - it's really a fabric for writing distributed systems. Apache Spark was a sample app written for Mesos. Marathon is another application that runs on top of Mesos and orchestrates containers. It can handle tens of thousands of containers and more in a single cluster. Chronos and Storm are other systems that can run on top of Mesos.

Marathon: https://mesosphere.github.io/marathon/ Chronos: https://airbnb.github.io/chronos/

In addition to what others replied, you might be interested in this presentation by Benjamin Hindman (who co-created Mesos) to see how it all fits together: https://www.youtube.com/watch?v=F1-UEIG7u5g

CoreOS is Linux distribution. It has been tuned to be lightweight, secure, and highly compatible with Docker among other things.

Mesos is a job scheduler for large distributed systems based on a quick read.

edit: and to the point, it is very natural to consider Mesos slaves as running inside of Docker containers (which may be based on CoreOS)

CoreOS contains a tool called fleet, which abstracts systemd into a cluster init system. fleet has scheduling rules such as "don't schedule more than 1 foobar on this box" or "schedule this app on a box with metadata region=foo" built in to it.

fleet doesn't aspire to do complex resource scheduling or high density bin-packing. The best way to accomplish these tasks is to use fleet to bootstrap more complex and higher-level services like Mesos, Kubernetes or Deis. fleet will make sure the components of those services are running in the cluster, even if machines die or are partitioned.

(CoreOS employee)

Or, more robustly, run Kubernetes, Deis, and Marathon — along with Spark, Hadoop, Cassandra, and so on, all Mesos, all on the same cluster. Mesos is the only technology capable of doing that at any scale.

It'd be very interesting if you know anyone who is doing this sort of thing (fleetd for bootstrapping a more "heavyweight" scheduler) in production, and could prod them to do a write-up on their system. It might inspire some more interesting PaaS products. :)

Deis has been doing this successfully for a while now. Their work is all open source. Here's what the units look like to bootstrap all of their components: https://github.com/deis/deis/tree/master/deisctl/units

More info on the components: http://docs.deis.io/en/latest/understanding_deis/architectur...

The Kubernetes project has also documented a getting started guide for CoreOS: https://github.com/GoogleCloudPlatform/kubernetes/blob/maste...

The current options are $80 development and $200 for production a month. I would like to see a $60 option for starter, but I would imagine the benefits of the mesosphere deployment are better seen on a higher number of instances.

You can create your own custom cluster[1] with the specs and number of instances that you desire, which would allow you to easily create a $60 option.

[1]: https://digitalocean.mesosphere.com/clusters/new/custom

You can customize the cluster ... if you drop down to single core machines you could get a development cluster (1 master, 3 slaves) down to about $40/m I believe.

Will there be Mesos-specific pricing? The full benefit of Mesos abstraction would remove the concept of a physical machine, and allow DigitalOcean to compete with Heroku-like services even better!

This is great. The tools are still pretty buggy though.

How so? What would you like to see handled differently?

"Internal error while provisioning. We're looking into it! In the meantime, try creating and launching a new cluster."

Tried launching like half a dozen clusters (both custom and default)

Would also be nice to see a post-process set up tutorial on security (iptables and whatnot that people here are talking about)

The execution on GCE was much better imho with VPN support and generally better security (albeit the cluster cost was significantly pricier).

Reply on news.ycombinator.com