Shell Scripts vs. Ansible
devopsu.com
devopsu.com
When my shell scripts become untenable I happily hire someone who is an expert at installing packages on a cluster of machines.
It's a complex problem and CMs so far are complex enough that I'm put off by how dense the CM language is. Every time I come across chef my ears start leaking precious brain fluid. Recipes, cookbooks, knives, bags, dependencies, acronyms I haven't bothered googling for... there's enough there that someone can dedicate their time to mastering it and be paid a decent salary.
No thanks, I'm already spending enough time as it is being an expert in something else. I'll just stick with shell scripts because at least I can understand those without much effort. If they become a problem I'll hire someone to fix it. :)
update: That was a little rant-y... after reading this article I might try Ansible if I get the chance. The problem I have with CMs is that they tend to be very complex because they solve a complex problem. Shell scripts tend to be simple (but crude by virtue). I am curious whether there is a middle-ground.
Achieving a middle ground was one of the points of creating the project -- making configuration tools more accessible to everybody, that you can forget and come back to in six months and still know how to work with.
I definitely would be interested in your feedback after your trying things, but I think that's exactly the point Matt was trying to make in the article.
I appreciate attempts at reducing complexity. I've been meaning to look into Marelle a bit (http://larsyencken.github.io/marelle/) as I find logic programming and CBS to be where CMs should be heading. Is Ansible somewhere along that spectrum?
Multiple OSes are pretty straightforward though. Look at the 'group_by' module for dynamically carving up machines based on operating system, and things like the 'when' statement for introducing conditional behavior.
Variables can also be overriden based on groups, so you can define defaults in one group and then have different settings in other groups.
Fortunately Ansible has dramatically lowered the learning curve and now is a great time to make the switch. So, if you have friends that need an "intervention" from their manual installs, hopefully this will help you make your case to them :)
Of course there are many great config mgmt tools out there. I singled out Ansible because it's so simple to get started with and I've found that the learning curve is usually the biggest hurdle to adoption.
Two notes-
1) They all work well for the base cases, and make it much easier to handle hundreds of machines - Being able to say "Ensure libcurl is installed on appserver, and make sure that nagios has these plugins we wrote" is amazing.
But where every system starts to break down, is when we have setups that I have a good reason for, but don't fit into the preconceived notion of how to handle things.
For example, if I want to install a package from source, or if I want to manually wget a .deb from an internal server, then dpkg -i it.
What's nice is that most systems allow some flexibility - You can do 90% of things in the pre-written modules, and break out to do manual commands where necessary.
2) One thing I'd really like to see, though, is a system which operated with even LESS intelligence.
Rather than using YAML files, or any sort of "user-friendly" config, let me define variables and use pure Python to iterate and execute using them.
I'd love to have people give me pre-written modules as classes I can call, but having that exposed in a way I can get at it with native loops, if/then, etc, would make things a lot more straight forward.
for server in webservers:
server.ensure_perms('/var/lib/somefile',755)
if server.name in qa_servers:
server.ensure_line('/etc/profile','EXPORT RUBY_ENV=QA')
---
- servers: webservers
- tasks
- file: mode=0755 path=/var/lib/somefile
...
---
- servers: qa_servers
- tasks
- lineinfile: dest=/etc/profile line='EXPORT RUBY_ENV=QA' state=present
[EDIT]: Dropped names from tasks to make examples clearer.
In theory should produce the same output, but if something relied on doing it the first way that would be the difference.
If you really needed the second task to rely on the output of the prior task, you'd probably want to use conditionals - something you can do without dropping back to the API.
I think you should give Salt a go. While the usual way to run it is to describe stuff using yaml-via-jinja files, you can also create the same structure using standard python modules. Or if you really don't like the declarative way, just import the existing salt states and fire them the way you described over their api.
My vote for the one who puts here an elegant solution for something as simple as:
ssh user@host sed -i '/FOO/ s/FOO*/BAR/' /etc/bla/bla-*
The first parts is easy, you just have to learn a new DSL because ansible ignores your .ssh/config.
The sed is easy to replace to with a lineinfile:
But... how can I iterate over the remote files?
I don't like my solution, is not elegant:
- name: Get the list of remote files
raw: printf '['; printf "'%s'," /etc/sysconfig/network-scripts/ifcfg-*; printf ']'
register: p
- name: Apply the change to the list of files
lineinfile: dest={{ item }} regexp=^NETWORKING_IPV6 line='NETWORKING_IPV6=no'
with_items: p.stdout
But I see my old script to setup a daemon in a chroot, and I see "my" ansible version... and I don't know what to tell (the ansible version just multiplies the lines, in a much more verbose way, that finally looking at it, you have to think and concentrate much more, than looking at my old and direct .sh)
At the end... this is not new:
ssh WORD < SCRIPT
But I wish users of my software where so happy first day as I'm with my ansible experience.
The documentation is great, but there is no "a reference", you have to read 5 or 6 different (long) pages to have an overview of what the DSL is, but the only reference is for the modules, not for the core.
You can search for ansible playbooks in sites like github, and... even if you will find a few which do what they say, you will find many "test" repos, and you will find that the 95% of the available code (no from ansibleworks, but by third parties) implement very poor deployments, without even following basic best practices or any security in mind.
I (as final user) would like a "cpan" or "rubygems" or whatever of ansible playbooks, more extended than the ansibleworks examples. Where people could share, search, vote, download (ansible get?), etc ansible playbooks.
Just suggestions.
I would suggest looking at github.com/ansible/ansible-examples for a fair amount of examples.
I think it's generally true of most systems that a lot of the community implemented content is suited to the needs of someone who wrote it.
As for content sharing, we might be working on something :) Stay tuned!
use "-c ssh" for the native OpenSSH connection and it will read your SSH config. As far as jumphosts and other options, paramiko will not. "-c ssh" is the default on platforms that support ControlPersist (because it's faster), so this means CentOS 6 is still using paramiko as a default at least for now.
Will research more about the connection options, did go through the available tutorials, and as it did work with the defaults, I did skip the connections part.
Thanks for all the efforts put in this project.
P.S. A way to iterate over remote files could be nice. Something like with_remote_fileglob:
Ansible takes a different track because I want to get out of the practice of writing code. (Different strokes, etc -- but it's intended to be faster to write and review)
Chef - Infrastructure as a Code
Ansible - Infrastructure as a Documentation
?It runs via SSH and requires no particularly special setup:
http://docs.fabfile.org/en/1.8/
If you're a perl lover then my own tool, Slaughter, allows you to install packages, fetch files, and make use of the large CPAN library:
http://www.steve.org.uk/Software/slaughter/guide/
Both of these tools are more basic than Ansible, but also more flexible, each in their own way.
Many of my own web-applications use it for deployment, and for that purpose it works wonderfully:
* Login via SSH to upload a tarball, generated via git.
* Deploy on remote machine, with a versioned path.
* Reload services.
Just as simple as it needs to be.
Having a unified configuration that you can track and push is good, but if you are installing from RPM or DEB, you want to know if the package is pushing a new config itself, or providing a new config for review.
The utility I wrote looked for .rpmnew and .rpmsave save files and checked with a central repo for whether they were whitelisted (by MD5) or not. Anything not whitelisted resulted in mail being sent alerting admins. Along with an fairly comprehensive per-host auto-update exclude list (which I got yum-autoupdate patched upstream to allow), this allowed us to be fairly confident in allowing our 100+ boxes to auto update every day.
Without paying close attention to packages release notes, interesting things can come down from the maintainers, usually from them back-porting a fix from a later version (or a feature, if it's a RHEL point release).
I think configuration management is a good thing, but I'm not sure it's the best thing to be taught first, unless there's mentors to work with. Some things you just won't think of until it's bit you in the ass, and have that bite be at small scale until you've figured it all out is probably a good thing.
Then, even if a new package is pushed to the repo, you will not receive it (and will be notified by an error by subsequent runs if it's been manually updated) until you update the inventory file.
Many things in Ansible can be pinned in similar ways.
Makes things faster and ensures even if you don't pin versions you are not going to get upstream surprises.
So, we spent a friday and came up with http://fuckingshellscripts.org. At first it was just a parody, but then it started to get real after we found a lot of value in just using shell scripts for simple servers.
Granted, we now use ansible and it has been far easier to implement than chef!
I want to just use fucking shell scripts to configure servers. But it's a gem too which I don't want to install.
- name: Ensure https support for apt is installed
apt: pkg=apt-transport-https state=present
// add one to x
x += 1;
These are especially valuable when managing a large number of hosts.
Every few upgrades of Ubuntu I seem to get into a situation that I want or have to do a clean install, then waste time getting the new system back into a workable state. It would have to be pretty lightweight and intuitive for my purposes.
Github has a project called Boxen that does this for Mac OSX: http://boxen.github.com/
Google uses their own Puppet solution for their developer workstations: http://www.youtube.com/watch?v=YlKXRdSAZhY
Ansible will work great for this too - particularly if you're on Linux.
* Keep your dotfiles somewhere that can be synced back easily (e.g. Dropbox, VPS with rsync, git repo, whatever)
* Make all dotfiles symlinks to the above. This way, if anything were to blow your hard drive up, it will have a relatively painless recovery.
* Keep workspace stuff/code/things not too easily replaceable in a separate physical drive if possible. Mount this drive to whatever directory that would make sense to you through your fstab or whatever method you prefer. Adding the nofail option in your fstab is also a good idea if you won't have it connected all the time. Reason for all this: You can now wipe your Linux HDD more confidently if you ever need to do so. Important dotfiles and configs should all be safe, code is on a separate drive and other stuff is backed up off-site hopefully.
You have to be wary when apt-get is only one command away but in general it works quite well - I'm not editing so many global configs and got most of them in my dotfiles repo anyway.
But you don't actually use templating. You just copy the file over.
What do you do when you have to edit the configuration file? When it needs to be different on every machine for whatever reason (IPs to listen on, hostnames)? What is the smart way to do that automatically?
Templates use the jinja2 templating engine, so variables are represented by `{{ variable_name }}`, and you can put variables (as well as defaults) into Ansible configs to set them to different values for different servers.
Puppet is a bit difficult to setup (not too much, mind you, it does its own key generating and being a PKI on first setup just fine, and you can run it from its own internal webserver until you have to go all "webscale"), but after that it hums along nicely. Haven't had too much complaining to do.
Except that they changed the string escaping rules somewhere in the 2.7.x branch, which debian of course pushed because it's in the same point release. That was fun.
It only needs to be installed on the node you're working from, not every node (by default).
Dependency is determined by order in the file, not by a dependency graph.
So I looked through the Puppet docs for about 5 minutes and found I could just run "Puppet apply whatever.pp" on the server and got everything working in about 15 minutes. And I'd never used it before in my life. So for now, I'm quite impressed with the ease of Puppet.
Chef and Ansible both are really great for 80 to 90% of what I've needed to do in the past. But that last 10-20% is really painful. It requires huge contortions, or poking through the source code so that you understand the details of the system so thoroughly that you can write an new module, or abuse the existing modules and bend them to your will. I suspect that, in the end, Puppet will have the same sorts of limitations as the others. But it did have a very nice way of setting things up that didn't require a whole infrastructure.
It doesn't seem preferable to me, from this example.
Note, I don't need to be convinced of the benefits of configuration management in terms of reproducibility and reliability. I'm just saying the OP's examples don't really make the case it suggests they make, of showing how ansible is so much easier than shell scripts. It does not appear to be.
(A) there's error handling,
(B) the system will only run commands as needed
and of course using things like 'with_items' and 'notify' ends up making it a lot shorter.
But yeah, most of Hacker News already has a read on that :)
If the configuration is small enough it's easier to write a script in bash that approximates the re-entrancy well enough instead of having to bootstrap the configuration manager, distribute the description files and execute it. In fact a bash library could go a long way covering the scope of these tools.
[1]: https://en.wikipedia.org/wiki/Reentrancy_%28computing%29
I found Chef clumsy and hard to set up on its own (its central server requirement seems like it was designed just to support a business model), but with knife-solo it's extremely usable.
Huh? You run `knife solo cook example.com` on your local machine. It then sshs in and runs what it needs to. (Granted, it runs everything via chef-solo, but the user experience is running a command locally.)
In the article, you're just copying a new magic configuration from somewhere vs. "proper" sed use (in reality, it can be a bit more than that) in the shell script.
In most cases, we recommend using the 'template' module for this reason, in cases where you know what you'd like to put down for the file contents.
However, I wanted to show the actual process I went through. Editing the file in place seemed silly to do with Ansible, so I went with putting the config file in version control and copying it over. That also had the benefit of showing how Ansible encourages better practices than shell scripts.
Of course, that's totally subjective, so for some folks that use shell scripts, they may already know to put their config files in version control. However, in practice, I've not seen that be the case.
For instance (I know there's vagrant and stuff), I develop on a Mac but deploy on linux. My sed handles that pretty well.
DigitalOcean ships VM that don't have an admin group. So when I tried DO, I added:
grep -q 'admin' /etc/group || groupadd admin
Or, for changing the sshd port:
sed -i -r 's/^#?Port [0-9]*/Port '"$PORT"'/' /etc/ssh/sshd_config
Works great on all platforms... regardless of the default sshd_config.
And nginx is full of compile options and modules to enable and disable, so maybe "the same config in the same version" does not work on different platforms.
If you're reliant on modules being in there you compile it yourself with a source deb/rpm/etc from nginx. If you don't need a module then don't compile it in.
Does it beat sed? Probably not in the short term, but it makes it easier to not forget to sed a file when deploying (not to mention adding the ability to skip changing the remote file and sighup nginx when the file hasn't changed).