Not trying to be too snarky, but why can't people just use shell scripts for automation?
They are easy to repair, in a language you (should) know, reproducible, and testable. They are native to your target, supremely portable, etc, etc.
For just one example, I've recently begun deploying CentOS 7 boxes whereas previously we mostly used 6 (and some 5). Largely the existing puppet modules kept on working despite major OS changes (hello systemd!). If I'd been maintaining a set of bash (or as used to be, csh) scripts I'd hate to imagine the amount of spaghetti I'd have to have mashed up.
We've also got almost every Linux distribution out there you can think of, versions going back a decade or more, Solaris boxes, BSD's and not long got rid of our last AIX box (and I'm going to ignore the last couple of OpenVMS machines). Handling it with hand-rolled scripts was really not fun compared to puppet.
For example, when running Puppet or Salt, I don't have to worry about which package manager is installed, or mucking about with checksums to see if a file needs to be replaced, or templating, or variables, or...
If I'm being honest, just the thought of writing dozens of sed and awk commands to mimic the templating offered by Jinja gives me hives.
That said, after fighting for a few years with Puppet, then Ansible, then Salt, I'm thinking that bash and make files are sorely underestimated for some of the simpler provisioning tasks. Bootstrapping a Salt Minion or a salt master is a lot more straightforward in bash than it is in Salt itself.
You can handle all of these things in a shell script, but that's not the question. The question is: will someone who's "just writing a quick script" think that far ahead?
My experience so far suggests the answer is usually no.
It's not the language. It's the mindset.
This is why I love Ansible.
Unlike many of the alternatives, Ansible doesn't try to invent a radically different DSL. Instead, ansible uses YAML playbooks that degrade very nicely to shell commands[0]. You can literally take almost any shell script and turn it into an Ansible playbook with a series of Vim regexes - or heck, write a simple sed script to translate it automatically.
Of course, if you use Ansible modules instead of the `command` directive, you get a lot of performance benefits and guarantees of idempotency. But it doesn't try to reinvent the wheel; it just builds on top of it.
It makes it really easy to hack together an Ansible script that is immediately understood by anyone who's used bash/sh and it makes it possible to improve upon it incrementally by replacing shell commands with the relevant Ansible modules.
I've used other automation tools, and the thing that I really disliked about them was that it was so difficult to migrate an existing workflow to a new paradigm. In 2015, systems work is still very imperative, so while declarative syntax is philosophically attractive, I still find it way more practical to use the tools that require smaller mental leaps.
[0] e.g., this thing was hacked together from a shell script, and I never bothered to clean it up, but it's still already more readable than the equivalent shell script: https://github.com/ChimeraCoder/znc-kibana-playbooks/blob/ma...
The maintainer is unusually hostile to user contributions, especially when compared to, say, Salt. A colleague of mine tried to contribute a (fully functional, tested, and documented) change to the MySQL user module to support passing in hashes for user creation - the pull request was closed with a comment like "this can go in a community module". Nevermind that the MySQL user module is a core module...
It's also horrendously slow when your playbooks grow. The constant copying back and forth over SSH can be downright crippling. Just running a "green" (the server is in the target state and no changes need to be made) playbook against a local vagrant instance can take over a minute for my current monitoring server playbook. As a point of comparison, Salt completes in about 5 seconds.
That said, it's still one of the best "no setup" tools, for the cases where you can't install a minion of one flavor or another. (Yes, theoretically the salt-ssh command line tool exists, but it requires NOPASSWORD sudo on the target server to run)
My use case is admittedly much simpler than what Ansible allows for: I use it almost exclusively to provision a host for containers (and then to initialize/start those containers). As a result, my playbooks tend to remain small.
I don't think this use case is realistic for everyone yet, but I think that's the direction we're headed (more containerization), and that it won't be too long before this becomes the norm in devops.
[0] Seriously, the "copy" command is the best illustration of this - thank God that "synchronize" also exists!
If your application is written in ruby or python, why should you learn another (worse scripting, but that's just my opinion) language to deploy it?
Only slightly being sarcy, but this perfectly describes my experience of Salt, Ansible and Chef, but instead of a handful of 'ps' commands to figure out what's going on, you have Python/Ruby interpreters on both sides of the SSH connection, giant gumps of weird SSH command lines, a big C++ library and multitudes of TCP ports and home-grown crypto (Salt+ZeroMQ), perfectly descriptionless error messages like "Timed out", "command failed", or even worse, something you never had to deal with in shell: "template syntax error".
I have a personal axe to grind in this department (been thinking about this style of system since the mid 2000s, but never dared start yet another project), so it's entirely possible my expectations are higher than normal here. Still.. processing YAML with Jinja2? Seriously, bash or the most arcane old Makefile syntax was better than this.
What Puppet et al brings to the table is a bit of structure and best practice. After a two day introduction, anyone can start making changes to our infrastructure, and I'm pretty confident that the new hire in two years' time will understand it as well. And if I need to restore a previous environment, it's all in version control, and will work since it defines state as opposed to transforming it.
But the big win with a configuration management tool is not even in the tool itself. It's having a central database of _all_ configuration in every environment.
Suddenly you can start doing things like tell an application to connect to its auth service, and it'll know which one it is or even mock it if required. Your monitoring software knows every app in every environment, and can configure itself, so it's literally impossible for someone to forget to set up monitoring for a new service.
It's a huge win. Once you used to it, you never really understand how you managed without. But you really need to go all in on it to see the benefits. Automated systems integration is possible only when you store configuration in one and only one place.
> But the big win with a configuration management tool is not even in the tool itself. It's having a central database of _all_ configuration in every environment.
I agree that is the big win. Though in theory you don't need a configuration management tool to get it. Just sufficient discipline and attention to best-practices.
What the config management tools provide, here, is a framework for combining "site-wide" configuration, "local/discovered" facts, and "human-assigned" roles and classes and exposing it all, consistently, to a specialized scripting language.
The library of modules/cookbooks/etc. also often take care of a lot tedious logic/details that would otherwise have to be in the code (although this part can still get in the way...).
Automation should be tooled to the production environment by people who are familiar with the production requirements and -- most importantly -- by the people who will be expected to maintain the environment and respond to critical issues.
I'm curious about this -- what's wrong with using the same deployment tool for running dev _and_ the other environments? Jenkins itself is a life-saver for, like you said, doing development builds/testing/reporting, but if we've got one script that deploys an image to production, why not trigger it with a Jenkins job, if only to keep everything centralized and maintain coherent deployment histories?
You end up with a process that pulls it from a database or something to fill out the configuration. At which point, you might as well just use a script in python or an existing system.
Because the syntax for if statements and while loops is godawful. I've taught it to myself at least 30 times.
Also, there is no debugger, so I can't step through my code like I can with pub.
Also, the tooling around automated testing is worse.