Production Quality MongoDB Setup with Docker | Hacker News Reader

Production Quality MongoDB Setup with Docker

medium.com

42 pointsarunoda15 comments

Docker is awesome and I've totally redesigned my development workflow around it. Instead of running a monolithic dev server with dependencies for every project I start (mongodb, mysql, ruby, a giant mess of gems, node, memcached, jvm, etc etc--at a certain point these things start stepping on each others toes), I have a docker image for each dependency and configure a cluster for each project. Source version control is handled independently by simply mounting a project's root folder on each container and passing around config files. File system layering means the container branches for each project only take up as much disk space as whatever gems or node packages they don't have in common. For me, the moment I "got" docker felt like the first time I really understood the git workflow. Starting a new project is now a matter of minutes instead of hours, and the reduced friction allows for tons of experimentation that I would have avoided before. It's also fun to be able to spin up a 12 server mongo cluster on a single VM and watch how the sharding and replication works with different key schemes. I really can't say enough good things about the project, with the caveat that I've only used it for local development so far. I'm super excited to see where it goes once it hits 1.0.

We're using Docker in production (even though it's not 1.0) and it's been great so far. We manage a large number of small images on behalf of the end goals of our users at DevTable, and managing the images themselves was becoming a pain. To address this we built Quay.io, which we're using to store our images and feed them out to our servers in production. With your large number of apps and containers, you might find it useful too. We're basically trying to be the Github of Docker images.

Docker is awesome, but specifically NOT ready for production.

From Docker's website "we make a point of saying that we’re not production ready".http://blog.docker.io/2013/08/getting-to-docker-1-0/

While I love your enthusiasm, I worry that you'll mislead others into thinking that its a good idea to replicate your setup and run something important.

I never said Docker is production ready or not. What I show you is how to run(and use) mongodb with docker.

BTW: We don't need version 1.0 to use it in production. See NodeJS.

1. You're not running your db on node. It's unlikely that your data will disappear if it turns out node has a bug -> the failure case is much more serious for mongo on docker.

2. You run the even versions of node (noted as stable) in production - they've literally set things up so there's an acceptable way to run node in production.

3. You're showing people how to make a production environment on a system who's creators do NOT want it to be in production. Sounds like a bad idea to me.

I've been checking out all of this new stuff recently, like http://www.packer.io and http://www.coreos.com. It all seems cool, but I can't help but think it's too little too late.

I want to deploy at the application level, not the OS image level. The abstraction that I want is:

Servers: Something that runs applications. Could be a physical server, an instance on EC2, or a local process, etc.

Applications: My code. Black boxes.

I want to be able to boot up a new server and throw some applications on it, like a new MongoDB shard or whatever. Then tear some applications down, or tear whole servers down.

And I want it all to be managed by a RESTful API over HTTP so that I can write a script to automate deployment, and even automate the scaling of my service.

I think you can do it with just Docker, a bare Linux distro, and a few scripts.

Does this exist yet?

You pretty much just defined the goal of Docker. And it's already incredibly close to what you describe.

Right. So when can I write:

    shards = []

    boot 'ec2', 'us-east-1', (server) ->
        shards.push server.start 'mongod_docker_image'

    boot 'ec2', 'us-west-1', (server) ->
        shards.push server.start 'mongod_docker_image'

    setTimeout ->
        shard.stop() for shard in shards
    , 5 * 60 * 1000

Does this exist yet? I started writing my own earlier today with Docker because I haven't found anything yet.

The plan is to cook a single ami that can be deployed ad infinitum. The image only has a slave application on it that receives and runs Docker images.

Pretty much.

A Docker AMI Generator exists. DigitalOcean offers an out of box Docker host. OpenShift will replace cartidges with Docker. OpenStack can provision Docker containers. Other PaaS/IaaS are in the works.

Running a registry is as simple as `docker run samalba/docker-registry`.

Pushing to that registry is very simple, especially if you have a reverse proxy ( I use hipache. Mine is registry.stinemat.es )

So, assuming those first 3 are in use in a 'convenient' way (as defined by you) - your request exists today. That matter of convenience will grow.

I would like to test out clustering and distributed computing on the cheap with a group of friends. So I would like to replicate an IaaS/PaaS/SaaS stack with vagrant-mesos-docker. I can then update my friends applications, OSs, and use their extra storage for a VPN service?

I've been playing with docker, mongodb, hipache and meteor app a month ago, and I'd like to say that it was great experience. Here is my collection of scripts of Dockerfiles which allow you to run multiple instances of Meteor app behind hipache reverse proxy. MongoDB data is persisted using docker volumes too. https://github.com/vlebedev/docker-test

Nice.

BTW: I'm not sure about idea of using Docker to store the data.

I just read about it, Hipache does not supports sticky sessions yet. Sticky Session support is really important for Meteor.

Hope it would be great, If you use HaProxy or Nginx.

Great summary - especially spinning up another container to use as the client - I was wondering how to do that : )

Docker is seriously another planet of cool!

Yes. It is the start of the next gen DevOps.

Reply on news.ycombinator.com