Chef 11 released

SSH + shell scripts are pretty good for one off things. But they're rarely idempotent. Chef and Puppet will say "you asked for nginx version x, you have it, so I won't bother installing it". You can also handle dependencies and roles, so you end up with things like "this hosts a Rails application, so you need Ruby and bundler, nginx configured a certain way, and the upstream load balancer needs to point to the new server". So using shell scripts ends up either taking longer or as you grow it gets more complex because you have to check for different things that a specialized tool just handles natively.

On the deploy side, ssh + shell scripts might be a better solution (IMHO stuff like Capistrano and Fabric can be seen as a fancy wrapper around that). So the procedure for a deploy might be "check out source, copy to a certain folder, create these symlinks, restart these daemons". Deploys, by their very nature, aren't idempotent, as each deploy is intended to change the state, you either finish it or abort and roll back.

I interviewed community managers from Puppet and Chef over at http://linuxadminshow.com, there are some interesting comments from both about the need for it even at small scale. Nathen from Opscode had a few comments about erchef. He also runs a podcast called "Food Fight" where they very recently talked about using Chef for the deploy process (http://foodfightshow.org/2013/01/application-deployment.html)

That said, I prefer Puppet because its abstractions make more sense to me, and are pretty easily extended using Ruby.

Puppet is also trivial to bootstrap (at least on Debian). Unlike other Ruby stacks, I can install it using the package manager and be up and running in no time. With Chef you have to maintain additional services like CouchDB and RabbitMQ.

The real reason not to use ad hoc scripts or a tool you wrote yourself is that things like cfengine, Chef, and Puppet are maintained and used by communities -- your successor(s) will thank you.

I can't talk about chef but puppet basically gives you a declarative language to specify the end-state in which the server config should end up as. So you say something like "the apache server must be enabled and running" and then when the client runs it makes sure the state matches that. So if for some reason apache gets disabled (say a new version of the package disables it) the next time the client runs it will right that.

I have even used Puppet but I could not get over the feeling I was placing a lot of trust in an agglomeration of system in provided recipes

You don't have to use the recipes. I had the same worry so what I did with my own puppet usage was set a rule that I won't copy recipes from anywhere and just use puppet to do the same things I would do directly on the server but describe them using puppet so I can replicate them later. This has worked great for me. I still have to learn all the details of how to configure everything on the machine but I now have a version controlled log of how I've configured everything and I can very easily apply the same recipe between two machines.

And even though I don't use copied recipes puppet itself abstracts away a bunch of stuff. So for example "make sure apache is installed" is implemented with apt in ubuntu and rpm in redhat. So if I wanted to repurpose my recipes from ubuntu to fedora I'd have to recode paths and package names but I wouldn't have to figure out how to interact with the package manager, what the command to create a new user is, etc.

With fabric it's easier to say - do this, do that. Both have their own place and you can use both for different tasks.

It may not matter at the first deployment stage (it's always install this, install that), but it does make a difference when you do continuous deployment for example. That are many dependencies that you'd have to implement in details and manually with systems like fabric.

I wrote a firewall cookbook that takes advantages of Chef's search capabilities to dramatically reduce the maintenance of our firewalls, while keeping the rules ultra specific (inbound and outbound). Check it out: https://github.com/jvehent/AFW/ & http://jve.linuxwall.info/blog/index.php?post/2012/11/14/AFW

But practically speaking I'd say Ruby is essential. It'd be a disaster trying to use others' cookbooks without being able to read the recipes, and almost certainly you'll want to do something they don't cover.

On the other hand, perhaps Chef is an opportunity to learn Ruby? Certainly you can be productive with a very superficial understanding of the language, so you could get started pretty quickly. And a Chef recipe is about the flattest, simplest code imaginable.

As one of the original Chef contributors before Opscode, and then becoming one of the lead people working with the open source community for Opscode, the vitality of the open source Chef project is very important to me. However, Opscode's support for all of our open source projects also hinges on the company being successful enough to continue to do so. Thus, a balance must be struck.

Opscode doesn't respond to every ticket on that open source tracker, just like we don't write all of the code in the open source project. I personally try to at least see every issue, but with over 1.8 million installs of the Chef gem alone, there are a lot of users out there. If you want a guaranteed response, we do have various paid support plans which will provide you with that, but it doesn't come for free because we have to pay the people who research and write those responses.

There are multiple ways to install the Chef 10 Server and many ways to start the processes. We have tried to support the most popular, and I think most people have found success. We're proud that we can do things like release our rewrite of the Chef Server in erlang back to open source. We want it to be easy for people to use Chef. We initially wrote it to make our lives easier, and we want your lives to be easier too. That's why the Chef 11 Server ships in our Omnibus packaging, which provides a running Chef Server in three commands or less.