Servers are fun: Ansible
lauris.github.io
lauris.github.io
ansible all -i ./ansible_hosts -m command -a "cat /etc/issue"
As it might be useful for some I have next aliases set:
export ANSIBLE_PATH="$HOME/web/clusters-servers/mycluster"
alias a="ansible -i $ANSIBLE_PATH/hosts"
alias ap="ansible-playbook -i $ANSIBLE_PATH/hosts"
# for vagrant testing machines where I test playbooks before deploying
alias atest="ansible -i $ANSIBLE_PATH/vagrant/hosts --private-key=~/.vagrant.d/insecure_private_key -u vagrant -s"
alias aptest="ansible-playbook -i $ANSIBLE_PATH/vagrant/hosts --private-key=~/.vagrant.d/insecure_private_key -u vagrant -s"
This is why Ansible modules (and most other config tools) instead describe the desired end state, and perform the commands needed to get you to those states.
Basically all config tools arose out of a need for solutions to replace parallel scripting with something more robust that accounted for potential variance between systems, and also to make things more push-button.
The hosts/inventory in ansible as a source of truth for your infrastructure is also a nice feature. You could use a selection from your ansible inventory to open a cssh with 'top' running on all your webservers, for example.
This way, launching your test VM is just a `vagrant up` and running ansible against it (repeatedly, if needed) is `vagrant provision`. No need to pass inventory/key parameters at the shell this way - Vagrant calls Ansible with the right settings.
One thing that's easy to miss is to make Ansible use sudo when running against the Vagrant VM, since the vagrant user by convention has passwordless sudo rights (ansible.sudo = true in the Vagrantfile).
http://docs.vagrantup.com/v2/provisioning/ansible.html is a helpful reference doc.
hostfile = ~/.ansible/hosts
You can put any given set of playbooks and configs into git and share with your team so that everyone is using ansible the same way. And that repo won't even carry any auth info.
If using metal installs, you can also set this up in your preseed or kickstart.
However, you should never have to bootstrap a machine with another script to install things like python-apt, as you can just use the "raw" command to do this for anything that is not part of the box, see http://docs.ansible.com/raw_module.html - so it's still self-bootstrapping.
disclaimer: creator of project.
And thank you for the work you put into ansible sir!
Just compare http://www.ansible.com/ with the excellent http://www.vagrantup.com/ when it comes to downloading and running your first commands. After clicking around for a good while on the Ansbile site you finally find "To install on a Mac, make sure you have Homebrew, then run... ", yeah, no thanks. Oh, there's a "pip" command, if I happen to have that installed (hint: nobody on Mac has by default), but at least there's some more magic commands I can run to install that first. After running the "Looks Totally Safe To Me! (TM)" command "sudo CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments pip install ansible" it blows up on Mavericks with some error about something called paramiko and headers not being available. The attrition rate for newbies on Mac must be phenomenal, because you have to be really dedicated to give it another shot after this. What I'm getting at is, Python software distribution is an abysmal experience, always has been, and you should not subject your users to it. They're not Python developers, they're largely not even devs at all, they don't care about your pip's and brews.
Compare this with Vagrant: Double click this here DMG, run "vagrant init", "vagrant up" and done, in nice _big_ friendly letters on the web site.
Disclaimer: I use and love Ansible (on Linux), and want to see it succeed. Telling my Mac using ops team to check it out soured everyone's first impression right from the start. Rant over.
Finding the Ansible docs (http://docs.ansible.com/) was not as easy as it could be, however, the install instructions were easily accessible from there, exactly where I'd expect them: Introduction > Installation > Latest Releases via Homebrew.
Second, yeah, pip sucks, I try to avoid it at all costs. However, homebrew (http://brew.sh/) is that one thing that all OSX power users should install as soon as they touch their workstation. 90% of the software - CLI or GUI - on my MacBook is installed and managed by homebrew. I cannot recommend it highly enough. In fact, I didn't even install Ansible through their instructions. I read about it somewhere else, and thought, "Hmmm, I wonder if this is in homebrew? `brew search ansible` Oh! Sweet! `brew install ansible`" Worked perfectly the first time, and I haven't fiddled with it since.
I'm sorry you had install problems and had trouble reading our install chapter: http://docs.ansible.com/intro_installation.html ... I will say the docs are open source, so if you want to add a note about how to get something easier compiled on OS X or something, click the GitHub link in the upper right and you're off and running :)
We get a lot of strong positives about how awesome our docs are, so I'm really interested in what you think we could do to improve them -- but they were absolutely not modelled on Chef and Puppets. Chef has about 1000 pages of docs, Ansible covers the majority of everything in a dozen pages. Most folks get up and running (with working playbooks running real applications) in 30 minutes or an hour.
My philosophy on docs are mainly to address things in the order folks will want to learn things, but also make it easy to hop over sections people will not. Show examples, don't make a reference manual. Though there's a need for reference content, which is where stuff like galaxy.ansible.com content and github.com/ansible/ansible-examples steps in, so you can see it put together. More tutorials I think would still be useful in the main docs. But yeah, a discussion on what you like to see differently would be interesting.
While I do understand you have to do something to install the program, what you have to do to configure it once installed is pretty short, and the install itself should still only take a few minutes. I'll give you the fact that Macs are not Linux, but they are getting there :)
I'm interested in your Mavericks error and you should definitely stop by the list about that one, or just consider the homebrew install.
Another great thing to do since you are already are on a Mac is just start doing your development on Linux in a VM :) Not that we require it, we don't... but it's great when you've got a real package manager stock!
Another easy thing you can do is just checkout the git source and do "source ./hacking/env-setup" and then you are running straight out of source. This is what I do on OS X most of the time, and we have a large number of users that follow the development branch full time.
You should absolutely think of your software, even though it's free, as having a conversion funnel to happy users. You are competing both against established players like Chef and Puppet, as well as new up and comers like Salt. Once a potential user reaches ansible.com, at every single step of the on-boarding that follows you will have some % of attrition to your competitors and apathy. The more steps, the larger the attrition, the smaller your userbase is.
Think of it this way; if you add a new feature, you may gain some small percentage of users. But if you reduce the on-boarding attrition rate from 30% to, 1-2%, you will gain a simply massive amount of users over time. Do you know or track your conversion rate?
The best docs is no docs. When curious Mac user visits http://www.vagrantup.com/ they see the download button, they click the Mac installer, and they're done. At no point during the on-boarding does installing the software require even seeing the docs. Mac users and even devs expect DMGs these days. Everything from 'git', to 'vagrant', come in DMGs. On most sites (like git), you visit the front page and it autodetects you are on Mac and gives you the DMG, which every Mac user will immediately know how to use. Git comes installed by default in Macs these days. Most people on Mac never need to install git anymore. The DMG file for git on Mac has 11,000+ weekly downloads on git-scm.com (that's in addition to the brews and ports of the world, which I would imagine have less users). How would you like an extra 11K downloads a week? ;)
The tide has definitely turned in favor of Macs over the last 3 years in software shops, I see more and more developers using Macs exclusively for development. In my company, although all software gets deployed to Linux (on AWS), of the several hundred devs working on the software I would be surprised if more than a handful (like me) still use Linux on their development machines (deploying to Vagrant locally and the like). While many of the Mac converts are coming from Linux, most new devs are just given a MacBook Pro from day one. In this new reality, I fear that you are doing yourself a big disservice by trying to shoehorn the Linux software distribution models on the new Mac users.
People used to tell it is possible to bootstrap Ansible via Vagrant on Windows:
1. define a linux box as ansible control vm, provision it using vagrant's shell provider 2. mount the playbook and run it
Well, I have spent hours on this. There were errors with Python in Linux (I forgot which, but I could solve this via searches in news groups), mounting rights, but most importantly I couldn't get the ssh part working in a multi-machine setup. I still don't understand how this could happen without Ansible halting, but Ansible did apply all the remote playbooks on the control vm itself. Even whith explicitly different ip's specified.
In the end I did gave up, having wasted so much time on a productivity tool. I am ready to accept this is due to me being stupid, but other tools don't make me look as such which is better for my self esteem.
I'll have a new try soon as Vagrant and Ansible keeps maturing and maybe I can find a tutorial this time which works with the current state of Ansible.
I think that it's curious how you keep referring to Vagrant as an example of "All That is Good And Proper(tm)," but you do not mention how Ansible compares to Puppet, Chef, or Salt for on-boarding of Mac users.
I think it would be more aptly titled "First Steps", "Basics", or something along those lines.
Great project though! I like the methodology of it compared to chef.
The inventory docs is a single page. Specifying a directory or a host file as an inventory seem to work but it's quite unclear how the two differ in behaviour. One problem with specifying a directory is that everything in the directory is interpreted; I had a README.md in there which resulted in baffling errors.
Dynamic inventory is also way under documented; a brief intro then two giant examples (ec2 and cobbler). It needs a tiny 'hello world'-ish example. My first attempt produced output in 'ini' format'. It was almost by accident that I tried json and happened across something that worked.
But error reporting is the worst. So often ansible will produce error messages like 'please specify an inventory' when you did specify an inventory. The problem is that you misspelled the inventory file or it couldn't be read. No mention of the file's name or whether even any attempt to read it is made.
(tangentially - This latter problem seems to be endemic to Python and Java and perhaps most languages that have exceptions)
What I've ended up doing instead is keeping a simple tree of config files, which get run through a templating engine and then rsynced to the target host. For simple things there is zero overhead - etc/hosts is just the plain file that goes to all machines. But this system allows me to have eg one smartd.conf which contains all my disks, which is then split between machines.
A few things currently fall through the cracks (users,groups), but overall rebuilding a machine is a relatively painless process. I feel like there must be a better more standardized way though (which is why I keep looking at these tools). Any comments/experiences/etc ?
Ansible's out of the box experience allows you to start with some simple but useful constructs then allow you to graduate to more and more complex usage.
I started with some of the one-liners here, then setting up inventory files, then roles, some simple tasks and then (what really sold me) was templates.
Even just having those configuration files in an SCM is a huge win, but I honestly squealed with joy when I was able to write a nginx configuration file that dynamically included backend servers given a particular role. And that provisioning a new backend server took a couple minutes and was automatically added to the load-balancing proxy! (not to mention was instantly sending metrics to a graphite/statsd backend because collectd was installed automatically too as part of a common role and configured to point to that log collecting server!)
Magic!
I'm not a sysadmin or servers guy, but these days I happen to manage most of the devops and so getting servers and machines set up is my responsibility.
Ansible docs are not great for getting started, and I pretty much just learned by skimming through some Playbook, and figuring it out their — often intuitive — purpose.
After a few hours I got to a close to getting my machines set up with most of my Django stack: Nginx, Gunicorn, Redis, supervisor, celery, etc. The only thing that I couldn't properly set up was PostgreSQL. And here is something that most of these automation tools lack: debugging. When my postgres roles was failing, I had no clue how to even start debugging it, so after an hour or so, I just stepped back and went in to set it up manually.
I feel some of these tools need to be more friendly with devops that don't have quite the same knowledge and experience than the target audience of sysadmins and ops people. That said, I learned a ton starting to use Ansible and automated a big chunk of the process.