How I migrated from Heroku to Digital Ocean with Chef and Capistrano | Hacker News Reader

How I migrated from Heroku to Digital Ocean with Chef and Capistrano

matteodepalo.github.com

100 pointsmatteodepalo34 comments

I'm planning to move away from Heroku too, partly because of the routing system issues you mention in your article, but also because I want to sharpen my sysadmin fu. Have you considered https://www.cloud66.com/ because it seems like a nice tradeoff between going "bare metal all the way" and Heroku's comfy environment.

I've tried their service a few weeks ago, but they had a problem related to my particular Gemfile so I abandoned it for the moment. Their product is great and I'm sure they will iron out these small bugs, but for now it's not a valid option for me.

DigitalOcean recently became fully integrated with Cloud66.com just last week. It's a great way to easily spin up a new server, and echo's our mantra of simple cloud hosting.

Cloud66 is awesome I should say, they take the pain of having to do all the groundwork yourself, which is so freakin awesome ! Are there any similar providers to cloud66, apart from Heroku? (and Engineyard ofcourse)

I love the idea behind this software. I just wish it worked with Django.

So how does this resolve the Heroku routing problem for you? You have one server that internally routes efficiently - but what happens when you scale past one server?

If you're not scaling past one server, why does Heroku's routing cause you issues?

On Heroku I was using thin that forced me to scale horizontally to handle more than one request at the same time. With this setup I can set a number of unicorn workers bigger than 1 and I'm done. I can scale vertically until I can and then set up more machines with the same methodology and use a load balancer like HAProxy to manage requests.

Why not use Unicorn on Heroku?

There is a limited amount of workers you can run with 512mb of memory (max 3 for my application) and the random routing can overload one dyno while the others are doing nothing. Basically you have to scale and pay more when with Digital Ocean I can buy a 2gb machine and run 10 workers spending 20$ per month.

He can use a 2-core VPS with plenty of memory and he will have much more headroom than a single Heroku dyno.

How many people really have to worry about scaling past a single server? To me, the issue is more about uptime so that where PaaS's get extra credit as far as I am concerned. It is great to be able to go on vacation and not much worry about admin hassles.

Using unicorn on a vps is the equivalent of many 'servers' or dynos on heroku, and unicorn is taking the place of the heroku routing here, so this does address the problem as you'd have more control over things like timeout and routing strategy, and have other options too for load balancing. If you do need several servers there is software out there to do load balancing, but with caching you can easily scale a single vps to serve a lot of users. For example HN still runs on one server I think, there's a pic of it somewhere that pg posted...

Interesting article I think (as someone who already uses Chef).

That said like I commented on your post, create a proper user instead of running as root, really.

I totally agree and it's next in the list of improvements I want to do. I just wanted to share this asap until it's fresh in my mind.

Edit

Done! :-)

Sure :-) I will definitely forward this to people new to this, I would have loved to have such a tutorial a couple years back!

Every time I see someone doing something cool with Chef, I think of this: http://progopedia.com/language/chef/

And then I think, "Wow. This person must be a mad genius."

This was a very helpful and practical tutorial. I don't know why sysops is a source of perpetual confusion for me but it's good to have as many examples as possible. Could you elaborate on the actual hoops that creating a non-root user was creating for you?

First of all I didn't know whether to provision my machine with root or not. Scraping through sysadmin resources I found people installing Ruby with the "deployer" user and people using root for that task. When I tried installing Ruby with deployer I got many permission errors, but I realize that was caused in large part by my lack of experience with sysadministration.

Also, running processes with runit as non-root requires extra [care][1] and I just wanted to get everything up and improve my scripts AFTER I knew that my setup was viable.

Edit

One more thing: I couldn't install ruby-shadow, which is needed to handle passwords with chef, due to some errors with the version of ruby I had.

[1]: http://smarden.org/runit/faq.html#user

With rbenv you can install ruby directly to your deploy user, no sudo required as it installs to your home directory

https://github.com/sstephenson/rbenv/wiki/Deploying-with-rbe...

Thanks, I'll try it as soon as I can :-)

You can deploy the infrastructure with root (or a sudoer), and run the app as a regular user (which is quite often called "deploy" in capistrano recipes).

If I am deploying an application with "deploy" (i.e. non-priviliged), it is ok to also run Unicorn as deploy? Or should it be run as nobody or www-data ?

With https://github.com/rmoriz/knife-digital_ocean you can launch and bootstrap an instance using knife in a single command.

Works with knife-solo and traditional chef-server environments.

This is great. Two weekends ago I did the same type of migration using Linode and Capistrano. Heroku taught us what we should expect from a platform, and it seems like, as a community, we're moving towards getting those great features ourselves.

This is one of the fundamental differences between a platform and an infrastructure provider. As a platform you build a single tool that is used by everyone and with it come inherent limitations. If an infrastructure provider can give you the building blocks to create a platform service on your own, you maintain full control. So long as your time investment into growing the infrastructure doesn't outweigh the cost savings of a platform you get much more predictable growth, better pricing and maintain full control allowing you to develop a product that's customized to your needs exclusively.

This is great. Thanks for sharing. I've been toying around with mina, but I may give Capistrano another shot. Right now I have a bunch of build scripts that would be much better suited for Chef and your setup closely mirrors my own.

I would be very leery of hosting any serious business on Digital Ocean. Their terms of service are simply too strict.

For example, you can't send email to your users without double opt-in.

Yikes!

Not to mention the ambiguous language banning 'offensive' and 'inappropriate' content.

Hi - It's Mitch from DigitalOcean. We are still ironing out our TOS. Given that we are still a startup we are always constantly looking to improve the language.

We don't consider ourselves too strict. As long as you are not trying to run an attack or spam/flood our network, we are totally cool. :)

Mitch,

I appreciate the sentiment, but I can't stake my livelihood on promises made on an internet forum.

I want to be a Digital Ocean user, but I'm going to base that decision on the legal agreements first.

I do hope this will encourage you guys to remove some of the ambiguity in your TOS, and to remove that double opt-in requirement.

Best,

Jacob

I totally hear you Jacob. Know exactly where you're coming from.

And yes, we will continue to work on our TOS. :)

Best of luck! Mitch

Did the same thing 2 days ago! And cannot stress enough how rewarding the experience was. Although I never heard of Digital Ocean so I chose linode, but I'm also very happy with them so far.

Linode have stackscripts, so you actually could do without Chef with a well structured bash script.

I have asked Digital Ocean about the possibility for Stackscripts, and I don't think they have planned for it, sadly.

It is nice to order a server, choose a stackscript, and have a server to deploy to without even logging in on the instance.

Anyone who's interested in migrating from Heroku go do a lot worse than giving Cloud 66 a try. All the lovely deployment goodness without the lock in.

Author here, I updated the article; now a user called "deployer" runs the app. Turns out it was easier now than when I started with the migration.

I hate the term cloud computing and all the stupid hype that comes with it.

It's called mainframe computing, the exact same clusterfuck your grandfather used in 1970. With bigger hard drives.

Reply on news.ycombinator.com