Comment by hugoduncan | Hacker News Reader

Comment by hugoduncan | Hacker News Reader

Parent Full thread

Pallet was actually born out of frustration using chef to automate a website at my previous startup. The initial versions actually used chef solo, and added jclouds for node provisioning (we actually demo'd this to Opscode, back before Fog was integrated in knife). I am by no means an expert in Chef, but Pallet intentionally makes a very distinct set of design decisions.

For instance, the reference to "copy/paste" you mention, and which is easy to misinterpret. Recipe reuse in chef is very much at the source code level - checkout a recipe and edit it for your specifics. Pallet places much more emphasis on using recipes (or crates as we call them) as library functions, in cleanly versioned packages (jar files). This allows you version your recipes, add them as dependencies to your projects, and cleanly separates the configuration data from the recipe code.

On a perhaps more fundamental level, the push model seems to be much more suited to automating configuration across nodes - eg. getting your haproxy pointing to your web frontends. While this is somewhat possible in Chef, it is left to the user to achieve. In Pallet there is first class support for this sort of cross node configuration. It makes it easy to have mixins, that say ensure a node is monitored, or it's log files collected.

A couple of additional advantages of the push model are that it leaves you free to use API's without having to push credentials for those api's to all your nodes and allows you to use Pallet for command and control.

Finally Chef and Puppet are both great frameworks. Pallet is a library.

Hopefully this outlines some of the differences in approach between Pallet and Chef/Puppet. The use of clojure was certainly not the driving force, and I hope that wasn't the main message that came across.

    Recipe reuse in chef is very much at the source code level 
    - checkout a recipe and edit it for your specifics. Pallet 
    places much more emphasis on using recipes (or crates as 
    we call them) as library functions, in cleanly 
    versioned packages

This is huge; when I was using chef re-use was an absolute mess. Ruby has all this great existing gem infrastructure for first-class dependencies and distribution, but chef ignores all that and just sends you back to cobbling together git repositories.

Opinions of Rubygems differ greatly and I would consider cargo culting gems as a distribution and dependency management infrastructure for Chef cookbooks to only be great for those who are strong with Ruby.

Each community has its own requirements and those of the Chef cookbook authors in general are different than those of Ruby library authors.

Lets say that I right a cookbook for apache2 on Ubuntu. What should I name the gem? Many other people will write this cookbook, particularly since mine only works for Ubuntu. Perhaps 'chef-apache2-ubuntu-btm'? Where do you store such metadata so a user can find, use, and contribute to a cookbook?

This isn't a simple solution and is a bit unique, which is why Opscode has been crafting a solution at http://community.opscode.com. It is easy to get a copy of a cookbook from the repository using 'knife cookbook site install COOKBOOK_NAME' and you can search the site through the web interface and using 'knife cookbook site search' to find the cookbook you're looking for.

The community site development has been slow, but continuous. In December we added source browsing so you can review cookbooks without downloading them.

http://www.opscode.com/blog/2011/12/14/chef-community-site-c...

There's still much to do, but I think we're (I work for Opscode) headed in the right direction for the project and community.

I've never used Chef either, but those don't strike me as unique challenges. Building a Chef-only distribution mechanism to support discovery, versioning, and source viewing is fine and all, but such things do already exist in any sane dependency management infrastructure (whether we're talking about gems or maven jars or cpan or whatever). In particular, the dev side of me is baffled by the notion of having a separate toolchain for downloading source dependencies. git submodules and such are bad enough, but mucking around with a special workflow to download sources seems like it's a long path with few benefits and perhaps some big downsides (i.e. build/process repeatability?).

That said, perhaps none of that really matters; I wonder if all this comes down to hidden assumptions re: constituencies and heterogeneous skillsets, which I think is what you were pointing at in the beginning of your comment. i.e. "you need to learn the Chef toolchain" may be a more sale-able/scalable message than "you need to learn the Ruby/gem toolchain".

If package management was as a happily solved of a problem as you imply, I think we would see fewer solutions out there. Each tends to solve for the needs of its own environment, users, and sometimes the whims and bias of its developers. When you ask, why not use "gems or maven jars or cpan or whatever" I think you have to stop and ask, "why are there so many options already?"

There are many views out there for what the answer should be for Chef cookbook workflow. What you are "baffled" by, another person uses daily and argues for.

When using knife to download a cookbook into your git managed chef repository, it creates a 'vendor' branch to store the upstream version. You can utilize this to create diffs and patches to contribute back to that cookbook project. It is up to the author exactly how to do that.

Some really like git submodules, others prefer one cookbook per git repository, some just have one company-wide repository and ask you to fork and apply branches against.

I think you're right about the 'Chef toolchain.' There are [large enterprise] users of Chef who don't care that Chef is written in Ruby nor particularly want to learn Ruby. They want to use Chef. Opscode continues to strive to provide tools to make the workflow easier for Chef users as a whole over preference to a particular toolchain, especially one tied to and solving problems for a single language.

> Pallet places much more emphasis on using recipes (or crates as we call them) as library functions, in cleanly versioned packages (jar files)

This is something that I can certainly appreciate and something that I've discussed at length with colleagues. In general dsl's require that you perform some impressive acrobatics when you want to extend them and this becomes particularly frustrating when you consider that a programming language like Ruby already has many well thought out facilities for said extensions (mostly directed at chef here). I'd like to think there's some happy middleground between a dsl that's easy to dive into and making use of the language constructs we've come to depend on everywhere else.

There does seem to be a bit of a shift coming to Chef with the use of LWRP (Light Weight Resource Providers) as a way to have a library like interface to cookbooks. But its not as first class as a Clojure library.

There is also a new way of managing cookbooks more like the Ruby Bundler called Librarian https://github.com/applicationsonline/librarian

Also, Pallet is sweet - I (Adam Jacob, one of the authors of Chef) have talked with Hugo, looked at the code, and played with Pallet. It's good sauce, and it's not quite the same kind of beast as Chef/Puppet. It's much more like what the logical conclusion of tools like Capistrano and Func/Fabric hope to be when they grow up.

It's also super fun to use, partly because Clojure is a great language.

Reply on news.ycombinator.com