The setup described in this post looks fragile, regardless of whether it's literate. I mean, we used to do these things, but we've moved on.
There's one thing ansible/salt are good at: make it easy for cheap replaceable devops to write tons of repetitive boilerplate yaml. Harder for them to shoot themselves in the foot than with a real programming language. But once you descend into jinja hell that's not very true either.
Also: newer isn't always better and all that.
Automation is the right course, but this takes a very roundabout way toward producing an artifact that can be distributed and applied repeatably, compared to using something like an Ansible playbook that is applied using Virtualbox's built-in Ansible provisioner.
I would recommend anyone reading this thread against recreating this, except as a proof of concept, and instead concentrate on a workflow more similar to this one:
https://medium.com/faun/building-repeatable-infrastructure-w...
Simple clean architectures can be deployed with anything. I judge a tool by it's ability to "make the easy things easy, and the hard things possible".
But you do have a very good point here:
> the old ways are largely pointless for most clients and developers who have long left behind artisan infrastructure
I don't like that it is this way, maybe I'm just old fashioned. But yes. This is the way it is.
Puppet lays down those things that the platform "promises"* to provide - syslog, time, auth, DNS, etc, and Ansible does application-specific things.
* - Not a strict promise in the Mark Burgess "Promise Theory" sense, but similar in thought.
On the other hand, in imperative world you have other difficulties, because you have to check the current state first and decide what to do in different edge cases. Not sure which one in the would be more complex.
The best example I’ve come across is building software from source. When there’s no package, you have to do this. Some may argue that this is not Ansible’s job, but I don’t see how it’s different from `apt install`, nor where else it would fit into the pipeline. Anyway, the way I solved this for example was by pinning the idempotency on the existence of build artefacts. Those are often software-specific, so it’s quite fiddly/non-general to find the right artefact.
Package it as a deb as part of the CI in say jenkins and then apt install it with ansible.
https://github.com/wtsi-hgi/hgi-systems/blob/master/ansible/...
The replies to my original post say this is not something Ansible should be doing. Fair enough :P I was young!
Basically, it's better to stick with tasks that stand on their own. As soon as you stick complex task logic in it becomes kind of a Rube Goldberg machine. That being said, you can pull off quite a bit if you just test rigorously and write defensively.
http://reclass.pantsfullofunix.net/index.html helped a lit but still not perfect
You can break it by app, like, having many git repos one per app and then use ansible galaxy to reuse code
Or you can try having a dynamic inventory that pulls metadata from say consul.
We’re in the ballpark of 2k machines and still happy !
Sometimes it's a lot easier to stay closer to the metal for figuring out what you actually need, then break out the declarative automation to make it truly reproducible.
This looks to be a step-up from just manually tinkering with commands in some shell to figure out or explore what it is you need to declare.
Ansible was written before we realized immutable infrastructure is the best way to go. Configuration Management tools craft system state dynamically like a drunk sculpting a Roman bust with a Louisville Slugger. You can get them to do what you want, but it takes a lot of work, and even then the outcome is uncertain. CM tools are often complex because a system whose state is constantly shifting requires complexity to handle it.
If instead you make immutable artifacts, you don't need complexity. A simple series of straight-forward commands in a single version-controlled file does everything you need. Dockerfiles seem immature at first because of how simple they are, but in practice it's much more reliable than Ansible, not to mention easier to support. Thus, non-declarative automation, when used with immutable infrastructure as code, trumps declarative.
Ansible is also only optionally declarative, which people miss just because it has that bastardized form of YAML for a config file. The simplest tasks, roles and playbooks work when executed sequentially, but not necessarily when out of sequential order. When you do make it super-duper-declarative, it can involve tons of confusing logic that makes it nearly impossible to understand, and is much more verbose (and complex) than a simple script. As soon as automation is more work and cost than the alternative, it should be ditched.
If it's just a few lines can you give an example?
- name: install the latest version of Apache and MariaDB
package:
name:
- httpd
- mariadb-server
state: latestThings that you haven't done before are harder than things you do all the time, and I would've reckoned trying to do them in ansible is much harder than doing it interactively, but I'd be interested in learning.
If I find an article like this[1], how do you translate it into the artefact I would want to keep in source control? In org-babel it's this[2], so what would that look like in ansible?
[1]: https://www.digitalocean.com/community/tutorials/how-to-set-...
[2]: http://www.howardism.org/Technical/Emacs/linux-iptables.org....
Once you've done it and you've documented it, I can see how you might translate that into ansible (or chef or whatever), but that's a different thing.
Using an iptables module for Ansible it is straightforward to write the rules as yaml. The documentation has a clear example.
However, having used several of these languages for over a decade and realizing this is a contentious issue within the community, I would actually avoid using these tools for individual rules. What I would do is dump the rules to a file, distribute the file, and let Ansible make sure the file matches what is running (by way of a regular rule load when necessary).
The idea here is that I am already familiar with iptables rules and how to write them, and would expect any other ops-ish person to be the same. The source file matches the output, and any historical diffs will be much more straightforward to read, as there are no intermediary source formats that can change.
Also, there is one less Ansible dependency involved, and less syntax to learn (given that one can already read iptables rules).
Are you saying that everything (or almost everything) worth automating is already automated by someone else so learning how to do new things isn't important?
That's not a name, that an intention describing comment, why isn't it called that way (or for brevity's sake, just "comment")? That might be nit-picking, but IMHO such misnomer cause unnecessary confusion and at the very least make the tool harder to learn.
package %w(httpd mariadb-server)Modern declarative automation is great for a lot of things, especially at work and for CI/CD pipelines, but it's no substitute for learning what you need to do to reproduce the same setup by hand.
Inspired by https://github.blog/2015-06-30-scripts-to-rule-them-all/
Is the playbook always called "play.yml"?
And how do you handle multiple repositories (or is there only a sinlge one?)?
And no, it is not always called play.yml but according to its function, e.g. "install_foo.yml".
We typically have all our code for one customer in one repository.
And yes, one repository per configuration (per customer code).
Is it that you have one inventory per customer configuration? Or how do you manage multiple inventories in this setup?
We have different customers with different servers and applications. Every customer has its own repository with one or more inventories.
And how do you share the context / documentation which yaml file is intended to be called by which ansible utility with which utility parameters and switches?
I ask b/c this is normally the problem I run into when things grow and I found the suggestion in the OP interesting to this regard as it adds the context while implementing, not documenting afterwards (and such documentation often becomes stale). Also documentation is often declarative (like do this, then that etc.) and it does not show the original thoughts/ideas behind a certain utility invocation.
I'd be interested to learn a bit more here, so if you could share a bit of context from your end would be nice.
Like I said: Ideally there are no switches and parameters - the playbook should work as-is. If there are switches, we document them in our internal confluence. And yes, our docs get stale, too. That is a problem that we did not fix yet. :)
Many playbooks also get executed automatically, so there'S no need to remember parameters for them. You just start the CI-job that then runs the playbook.
Current services don't evwn allow for some of the bad practices of the past.