Do-nothing scripting: the key to gradual automation (2019)
blog.danslimmon.com
blog.danslimmon.com
Learn from my folly: I even called them “do nothing scripts,” referencing this article. However; I was judged by peers for not writing the full automation versions as they didn’t appreciate the idea of gradual automation (programmer hubris?). Saying “do nothing scripts” in meetings did catch the awkward attention of leadership.
As a description, “do nothing” communicates a lot. As a brand, “do nothing” can use some improvements.
My short prescription of turning “do nothings” into “do some things” into “do all the things” didn’t help. We had some new people join the team and they had fun turning the do nothing scripts into a document. * Sigh *
I still build these type of process description scripts still. I usually don’t advertise them to peers until they do some of the things nowadays.
Actually, other applicable terms could be: a) programmed documentation; b) script as single-source of procedural truth
Short for "print commands to run manually" script
Even if there's no code being run, having all the steps down and easy to follow is a way to make the process "automatic". The term is extendible to completely automated computer code, while including non-computer-code (human) automation.
Let's see, e i... e i... Oh! There it is!
"Immediate" is meant to accentuate that they can be deployed and start delivering value right now, something that (usually) can't be said for the implementation of actual automation.
And "placeholder" is meant to convey that you don't intend to stop there.
You know, it might be useful to do a little bit of extra work and turn it into its own exportable documentation so that people can't do that and have to rely on the script to get the documentation for it.
donothing.py exportI’ve added this logic afterwards.
. (Programming in this context does not refer to computer programming, but comes from the use of program by the United States military to refer to proposed training and logistics schedules, which were the problems Dantzig studied at that time.)
For example:
wish('copy these files to this host', files, host)
data = wish(Path, 'the path of the data files I need')
It works exactly like wish does above but creates a whole language out of it.
For a human it reads very natural but understanding which combination of human words that map to underlying handlers is a hopeless and unnecessary level of indirection, and is why anyone shouldn’t touch these tools with a ten foot pole.
procedure = [
CreateSSHKeypairStep(),
GitCommitStep(),
WaitForBuildStep(),
RetrieveUserEmailStep(),
SendPrivateKeyStep(),
]
for step in procedure:
step.run(context)
procedure = [
createSSHKeypairStep,
gitCommitStep,
waitForBuildStep,
retrieveUserEmailStep,
sendPrivateKeyStep,
]
for step in procedure:
step(context)
createSSHKeypairStep(username)
gitCommitStep(username)
waitForBuildStep()
email = retrieveUserEmailStep(username)
sendPrivateKeyStep(email)
Much easier to see how the data is actually flowing that way. I found the fact that the context was mutated in a function it was passed into kind of gross. It's just globals with extra steps. Besides, this is less LOC anyway.
If the idea were to leave the script as it is, then of course your way is simpler.
Not that it matters much either way.
In my opinion, in order to justify instantiating an object, it should actually contain data, and that data should actually be accessed by code other than methods internal to that class. This rules out using an object to represent a procedure - regular functions subsume that functionality.
Before first class functions became commonplace, it did make sense to use objects as procedures in some OO languages, to allow you to pass them around as values. Since most languages these days do have first class functions, this has become less necessary.
If you want a class in order to to split the procedure into multiple methods with access to shared mutable state, that's totally fine - you can just use nested functions to do the same thing.
This is a pattern easier to represent/embed within objects. But you might as well use a library of prefixed functions too.
But I agree with you, it's simpler to just call the functions only giving them what they need. This prevents them from implicitly depending on context and makes reasoning about the code much easier....
However if I'm going to start creating scripts like this then I think all the OO/Class boilerplate overhead would make me reluctant to do it. For me using functions with docstrings keeps more of the simplicity of the markdown approach while still allowing for gradual automation.
Here's a simple example:
https://gist.github.com/snth/c5c5a1236dd8ddf91973aed77d66cd9...
* https://pyvideo.org/pycon-us-2012/stop-writing-classes.html
> Send the user their private key via 1Password.
Why are you generating *private keys* for users, then sharing them? Not that this impacts the automation bits but IMHO users should known how to generate and maintain a key pair, and send you the public key.
Consider the usecase of generating a user their initial password for a service, this almost always results in the user needing to immediately reset their password after initial login, this doesn't happen if someone is generating both parts of the key pair for you.
My sysadmin doesn't need to know my password, and doesn't need to know my private key or passphrase, that would allow them to impersonate me.
*this may or may not be true depending on how Enterprise-y your workplace is.
Even as a security firm we have some exceptions in our company (disk encryption password for a physical device in the office is in a shared vault, for example), or support might do a remote control session with the user present, but it's definitely not the default setup procedure for a new employee to have shared credentials for example.
It's pretty much a necessity for any larger business that device data be protected with the user's own password and with a master password known to IT so that it can be accessed when the user's password is inevitably lost.
In principle, the sysadmin should explicitly avoid knowing any of the user's secrets, because if they do, the user can shift blame onto them: "It wasn't me, it was the admin!".
I will never generate a private key for a user and will initiate a reset process for any passwords and other personal secrets revealed to me; to do anything else would be irresponsible.
A workflow should be be
User creates an SSH key pair, the private key never leaving the computer
User sends public key to authoriser
Authoriser pushes public key to Git (presumably an email + key touple?)
Wait for the build job to finish (not sure what this does)
Build process sends the email saying "You can now use your key"
I think the larger point is that you just have step by step instructions, and thus dont need to catch edge cases, but it also makes it harder to avoid skipping a step, which seems reasonable (I do this myself in some areas)
We did this trivially with S3, our implementation guy gives them an access key ID and secret access key, tells them to install Cyberduck, and gives them a URL to paste. We’re off to the races.
Having the user generate the thing will turn going live from hours to days.
I’ve also done this analogously with SFTP. You keep the creds so you can help them because they’ll type it in wrong, their software will fuck it up, whatever.
Discussed at the time: https://news.ycombinator.com/item?id=20495739
But I wonder if it wouldn't be better if it were something like a ncurses list with checkboxable list items which one would progress trough.
Take a checklist for a pilot for example. Wouldn't it be not as good if the next item on the list only appeared after an item has been checked? It could be beneficial if you could peek a couple of items down the line in order to group your actions or get a better overview of the overall task.
"Do I put the water for the coffee in the microwave now or do I first finish this quick item and let the next long-running item start before I go and make my coffee?"
Falsy docs are worse than no documentation. We've tried looking at the scripts, but small divergents in the scripts destroyed our confidence in the scripts. Instead we figure how to do task by looking at the source code and actual deployments.
The lesson we learned from this is we need to treat scripts as a real software instead. We make scripts idempotent and run them periodically to see if anything breaks. For infrastructures we make them immutable and embraced the cattle mindset(vs pet mindset).
We make scripts idempotent
Not sure I follow. In my experience, we have _tons_ of process that's written up in random pages in our company wiki. I assure you, nothing diverges as quickly as a set (regrettably) of duplicated wiki pages outlining a process.
If you're making the argument that a fully automated script is superior to a checklist, then ... well, sure. But while you're spending the resources to fully automate one process, duplicated wiki pages are being written for the 100 other checklists (at least in my experience).
The key here is that I fully agree with you on the divergence problem. But if a scripted checklist diverges, people complain and it's trivial to update the script. If you don't have these, then people either add a comment to one of the wiki pages, or a side note, or worse, write up a "better" wiki page with the real actual this-time-for-sure steps.
I believe the key here is to implement a consistent tactical approach to incremental development, capturing divergence along the way. If you just let them lie fallow, then yes, that's bad. But surely a fully-automated script allowed to lie fallow and diverge from reality is no better off.
People like to brag that they never do something manually more than once, but in reality it takes time and practice to be able to fully automate any non-trivial process. You usually can't do it in one go. I really like that this article embraces that.
I especially like how it's broken down such that any particular function can be swapped out with full automation once the author gets enough confidence to be able to do that.
We were performing some migrations to our servers and we weren't sure we had nailed the process down. Automating the procedure has a problem in that I can't just "reset" the server and start again. I could try to make all my operations idempotent or using a babushka style "Check - Set - Check" methodology, but that would greatly expand the work for a one off procedure that once it's done it's done.
I wasn't even sure of the generality of the steps, and they were complicated enough that copy and pasting them with all their parameters was tedious. So I created a structure just like this!
I called it a living procedure. An excerpt:
#!/bin/bash
source ./procedure-lib.sh
question "Which client is this?" old_client
question "What worker is $old_client on?" old_worker
question "What is the targets name going to be? (it's fine if it's still $old_client)" new_client
question "What worker is $new_client going to be on?" new_worker
question "What branch is $new_client going to be on?" new_branch
question "What should we name the migration? (typically something like migraiton-$(date --iso))" migration_name
#================ Initialize the client on the new worker ================
step "Login to $new_worker and Initialize $new_client in legacy mode using run.sh init_legacy, set their branch to $new_branch"\
'eg.
ssh '"$new_worker"'.lightship.works "sudo -i bash -c \"
I've been thinking about creating a better structure to these scripts that allow you to gradually transform these living procedures into babushka style "Check Set Check" automations.
I mean, it even fetches the certificates prior to renewing them and then re-fetches them afterwards in order to make sure that the servers are actually using the new ones.
# colors it color A
## colors it color B
I say "almost", because it occurs to me that many steps are shared individually across many different workflows. If you see a single task as an automation target, then you end up writing simple scripts that perform very focused operations -- a strong advantage over a large complicated script that does many different things, some that are repeated for other complicated scripts.
In this setup, you end up with checklists that remain forever, with automation replacing some or all of the print commands.
Of course, my spidey sense tells me that I've just pushed the problem down a level, so ¯\_(ツ)_/¯.
A script is a recipe that your whole team can follow. It's easier to update a recipe than to propagate changes across the whole team.
It's also similar to a flight check, which is a fancy name for a reusable to-do list.
[1] https://nicolasbouliane.com/blog/no-script-is-too-simple
But, whatever works, works. Start somewhere, get it into a script, plan, whatever. Then, it is easier to identify steps that can be turned to entirely machine-operated.
Returning higher order functions and using format’s kwargs is an even more succinct version of this.
def explode(**k)
def run():
return TheBomb(
’{user} is!’.format(**k))
return run
run(user=‘gorgoiler’)Closures are also weird though, I get your point. I wouldn’t use them in code that’s widely shared.
The upside is truly private data.
Unless you need to keep state between two distinct actions on the same object, but in that case a single run() entry point will not be sufficient anyway.
Inheritance can be replaced by just calling a common function.
There are many “what if” here, which will likely never happen and until they do it’s premature abstraction. Most likely some other abstraction is needed by the time script grows.
https://youtu.be/o9pEzgHorH0?t=339
For glue code scripts like this you usually don't end up using encapsulation and inheritance or any other fancy OOP design patterns. It's usually grungy IO-bound procedural code, the sort of thing you'd write in bash if bash weren't terrible for long scripts. Don't burden it with premature abstractions. If you end up needing a class, just write the class later.
My continued thoughts: since this is a framework for trying to reduce cognitive load, I'm now thinking a pragmatic approach would be to try it out a few times, either as a set of function steps or a set of class steps and see whether one approach falls out as a preferable pattern to default too.
Thinking about it, I can see why classes might be a reasonable default, if you don't need the state, it doesn't matter, if you do, you might be tempted to refactor the script which I think detracts from the goal of this framework.
If anyone is using this, in python, would be interesting to hear their experience.
The test is you.
Many "slogs" require too much human decision to be automatable, but too little to be interesting. tedious. tiresome. irksome. In a way, also trite, bromidic/
You can run and edit them entirely in the browser, they can do anything you can do with JavaScript and they're easy to share with other people and create forks.
Just wondering, i know it isn't the point, but does this exist as a service? For example is there a web service where my sales dept could add all the steps and it just goes through them one by one, but can include a start/completed webhooks.
Another wise thing about it is that it's a relatively low-friction way to start documenting institutional knowledge. Your engineers might hate writing docs, but maybe they won't mind writing a bash script like this. Which is essentially documentation in disguise. And since it's (presumably) living with the rest of your version control system you increase the chances that your engineers will explain changes to the process over time (via commit logs).
This process has a number of steps, some of which are physical (handling paper), some of which are digital and scriptable, and some of which are digital but not scriptable (serious image modifications with photoshop.)
I use a process very similar to what's described in the article to manage the whole thing.
You need the interpreter to run it.
It’s less accessible to non technical people.
Actually running the script is an extra step that’s not needed.
This feels like an engineer overcomplicating something.
Then maybe you could request some help with the ones that are a stretch for you to automate yourself.
$ echo "Do the first step, then press any key."; read -s -k
[0] Modeled on https://stackoverflow.com/questions/5215343/how-can-i-pause-...
That being said, I have absolutely no clue what the advantage is to the python programming style exemplified in the article. Can someone explain why on earth I would ever want to convert:
def do_something(context):
pass
...
do_something(c)
class DoSomething(object):
def run(self, context):
pass
...
DoSomething().run(c)
1. You have a complex step that you want to refactor. Without classes you end up with something like:
def Actual_Step(context):
part1(context)
part2(context)
part3(context)
def part1...
def part2...
def part3...
class Actual_Step(object):
var1, var2, var3 # things needed for this
def run(self, context):
# set the vars somehow
self.part1()
self.part2()
self.part3()
def part1(self):
use var1
def part2(self):
use var2
def part3(self):
use var1, var2, var3
2. You want to inherit. Maybe you have a collection of steps that all have the same setup/teardown bits, you can do this:
class Common_Base(object):
def setup(self, context):
...
def teardown(self, context):
...
def custom(self, context): # should be implemented by each derived class
...
def run(self, context):
self.setup(context)
self.custom(context)
self.teardown(context)
class Specific_Instance(Common_Base):
def custom(self, context):
...
3. Customizing at construction time. Maybe you have a step that goes out to some database, but what database may vary depending on what you're doing (for instance, at work we have test and ops modes):
class Get_Data(object):
def __init__(self, db_name):
self.db_name = db_name
def run(self, context):
...
In theory you could grow this (though at that point you may be better off with another system) so that different reports can be constructed from the steps themselves, without the need to execute them. Add elements like documentation strings that can be returned and prettified for different user interfaces or reporting purposes. All without ever having to touch the actual methods doing the work, just the way you interact with the classes holding them.
def do_thing(context):
def step1():
pass
def step2():
pass
def step3():
pass
step1()
step2()
step3()
def make_thing_doer(db_name):
def f(context):
pass
return f
But I specifically asked why that initial example posted to the blog was written the way it was. I will stress that "because in the future you may want to do X or Y with it" is not a really good reason because there is absolutely no reason why you couldn't just replace the function when it came to that point.
Moreover, the database example you gave would require changing the code just as much as the function example I gave.
specifically, in your case:
tasks = [
ThingDoer1(),
ThingDoer2(db_name),
ThingDoer3(),
]
tasks = [
do_something_1,
make_thing_doer_2(db_name),
do_something_3,
]
Edit: I missed the inheritance example at first read but this can trivially be implemented like so:
def make_thing_doer(custom_step):
def step1():
pass
def step3():
pass
def f(context):
step1()
custom_step(context)
step2()
return f
def do_thing(custom_step, context):
...
step1()
custom_step(context)
step3()
def special_step(context):
pass
do_special_thing = lambda c: do_thing(special_step, c)
You can still make the code do this specific thing without ever needing to make it so ugly to begin with.
Nothing stops you from mixing the style and implementing __call__ in the class (or subclassing from a class which implements __call__ and calls run() if you don't like double underscores.
Precisely because it's an unnecessary amount of code overhead.
There's no indication at all that IF he needed to expand the code that classes would be necessary UNTIL there were very specific requirements and there's also no reason why he should put that code in place in case of that requirement popping up.
It's always cheaper to write the simplest code you need at the time and then delete it (or rework some of it) and write the more complicated code you need in the future than it is to write more complicated code up front and expand less on some of it in the future.
If the point of the article is effectively "minimising the activation energy required to start automating a task" then surely the code should also be minimal?
One extra line per step is not a lot of overhead.
The fact that I looked at that code and spent 5 minutes trying to figure out why you would want classes there and what exactly it achieves is precisely what you want to avoid in codebases.
I audit code for a living, a lot of it, it takes far more work to audit code where the authors put a lot of unnecessary boilerplate or unnecessarily flexible code in place because it means I have to start figuring out what the authors intentions were. The expanded realm of possibilities means I have to check for more behaviors. It slows down the reading process.
Though, it's also possible that the writer simple did'nt know better and has not enough understanding of python, or thinks other people who might extent them lack those knowledge.
This is a workflow in a python script but could also be a tool just by adding argument parsing. But also tools can be called from software pipelines and have tools to execute this for us (automation).
Look at https://aosabook.org/en/500L/a-continuous-integration-system...
I'm thinking of a pilot's checklist, or an operating theatre checklist.
But here, you have code that plays the same role, but which can eventually be changed out with code that actually does the work for you. It's a seemingly small difference but it's actually huge if you have some decent developers who can spend a bit of time every once in a while pushing the needle forward in automating these manual steps.
Another note: I've found that it's hard to even get teams to consider the minor change of using the code version instead of the wiki. Inertia is a real problem.
Using something like https://camunda.com [0], you can execute a business process model. Such a model can include sending emails, waiting for responses, human action, custom API requests, etc.
Camunda is not no-code, but it comes close to what you imagine. I suspect that when the existing API landscape of a company needs to be used for workflows, coding/integration is inevitable.
There's also (truly) no-code platforms like https://bryter.com
[0]: I only know of this because there's a department at my current job that specializes in implementing workflows using camunda. I got explanations for it during moderate drinking, so my recounting of it may be incorrect in parts.
Confluence is wiki-style (edit first, ask questions later), but I could definitely see git-style (pull request first) as an improvement for some situations.
So - got my external validation . Will expand that repo when i get my own time back
Consider something like "incrementally inclusive" or "eventually complete" or similar.
* Documentation of the entire procedure is contained in one place. No need to go sifting through 20 different sources of documentation. This lowers the human emotional barrier to "just get it done", as people will always avoid things they aren't comfortable/familiar with, or don't have all the steps to. This central point of documentation also enables rapidly improving the process by letting people see all the steps in one place, which makes it easier to fix/collapse/remove steps.
* Automation in small pieces over time avoids the trap of "a project" where one or more engineers have to be dedicated to this one task for a long period of time. Most things shouldn't be automated unless there is demonstrably greater value in the cost of automating them than the cost of not doing so. Automating only the most valuable/costly pieces first gives immediate gains without sinking too much into the entire thing.
* One unified "method" to encapsulate any kind of process means your organization can ramp up on processes easier, reducing overall organizational cost.
* In the absence of any other similar process, you are guaranteed to save time and money.
I would say that the only potential downside is if someone decides to "engineer" this method, making it more and more and more complicated, until it loses its value. KISS is a requirement for it to be sustainable.
It can be engineered, if you follow a gradual process.
On servers, I keep a log of what was deployed in a root directory following the sequential number _ goal format (ex: 00_partitions ... 90_web_server)
It is not fancy, most of the logs are not even scripts: many are just ASCII text files, that will only be used as a checklist if the same "goal" has to be achieved again. For example, 00_partition may be "gdisk /dev/nvme0n1" followed by a copy-pasted list of the partitions and some quick description about why it was done that way.
But that's on the first iteration only: the next iteration turns that into "do-nothing" script, the next iteration into a better script with basic checks (supporting both /dev/nvme0n1 and /dev/sda), then exception handling (if partitions already exist, etc), and so on: this gradual complexification process avoids the "premature optimization" of creating infrastructure-as-code for what you rarely need, while optimizing and fine tuning the parts you most often need.
Someone will certainly mention Terraform, or Ansible, or something else - yes, they exist and they are nice, but if you are doing everything there, you are over-engineering and wasting time: not everything needs your equal attention!
If you only install a webserver once in a blue moon, make a .txt checklist of the steps you followed.
But if you leave and breathe nginx options and certificate deployment, fully automatize all that, including the obscure details of what may fail if you use let's encrypt with some specific DNS configuration!
And if you don't know yet which is which, start small (a .txt checklist will cost you a few minutes) and the next time you find yourself doing the same thing, do it better using the previous artifact (the .txt file) to create a better one (a script, then a better script etc)
This brings up a very important point about checklists that I don't think gets enough attention.
The problem happens when somebody "updates" that web server in-place. If they try to record what changes they made in the middle of the checklist, eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected. This happens to me when I try to record changes in my VirtualBox configuration after I add a new system package or something; later I try to re-deploy my vbox, and it breaks.
So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end. This way you catch the unexpected problems and confirm the checklist still works for the next person.
Imagine this is 28-nginx : I would create another script 29-nginx-update only recording the update, even if it: "echo apt-get update; apt-get upgrade nginx ; echo "make sure to fix variable $foo"
Next time I have to do that, I will integrate that into 28-nginx and remove 29-nginx-update
> eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected.
Maybe I don't understand the issue, but my scripts or text files are simple and meant to be used in sequence. If I hack the scripts, I make sure it still works as expected - and given my natural laziness, I only ever update scripts when deploying to a new server or VM, so I get an immediate feedback if they stop working
Still, sometimes something may work as expected (ex: above, maybe $foo depends on a context?), but it only means I need to generalize the previous solution - and since the script update only happen in the context of a new deployment, everything is still fresh in my head, so I can do it easily
To help me with that, I also use zfs snapshots at important steps, to be able to "observe" what the files looked like on the other server at a specific time. The snapshots conveniently share the same name (ex etc@28-nginx) so comparing the files to create one ot more scripts can be easily done with diff -Nur using .zfs/snapshot/ cf https://docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.ht...
Between that + a sqlite database containing the full history of commands types (including in which directory, and their return code), I rarely have such issues
Shameless plug for that bash history in sqlite: https://github.com/csdvrx/bash-timestamping-sqlite
> So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end.
I agree: if I don't have time to fix 28-nginx, I write 29-nginx-update instead, with the goal next time to integrate it. But I don't try to tweak 28-nginx if I know I won't have the time to test it.
If you don't want to repeat the steps from the beginning, you could make a completely separate checklist to be followed on a given system that includes things like "make sure X package is installed", "make sure Y configuration is applied", so that the new checklist accounts for any inconsistencies. This is pretty common anyway as checklists are broken up into discrete purposes and mixed and matched.
he asked if there was any way to access a server, and the answer was "no". the only way to "access" our production server is to modify the provisioner script. there is no way to "update it in place". it's taken a while to get here, but it's really freeing to realize that yes, i have the credentials and could probably get in, but i know my changes would be automatically reversed in the near-term and there's no point in even attempting to access a server directly. the server belongs to the deploy script, not to me.
I prefer it when both the server and the deploy scripts belong to me :)
"infrastructure as code" with no way or extremely limited possibilities to ssh for emergencies strikes me as foolish overengineering / painting yourself in a corner, but if you like that, why not?
And then the server gets deployed via the script and the quick change isn't there any more.
Whoops.
My EC2 instances are all configured so that they can't be accessed from the outside. They boot up, fetch their install script from a set location and run it.
If they need changes, I either update the base image or the install script.
You lose some time and flexibility, just because you are afraid you may forget to integrate the quick change in your scripts.
My bash histories go into a global database to avoid this
Using SSH is what you do if it's a pet.
You can do a lot with four servers.
In the simplest case, a deployment consists of: 1. Install packages 2. Install configuration files, possibly from templates. 3. Configure services to start on boot.
This kind of automation is trivial to do with almost any tool, but there's no reason not to use something like Ansible that's designed for infrastructure automation, because you get encrypted secrets, templating and idempotency with zero effort and the result can be stored in a git repository somewhere.
The Ansible playbook is as fast to write as installing and configuring things manually, and after some practice, it's faster and the resulting system is of higher quality.
Even if you end up using the automation you write only once, it still has value because it doubles as a formalized description of what you did, and can be stored together with additional documentation. Over time you will also accumulate a library of bits and pieces that you can copy over to new setups, further improving your speed and quality.
For simple configuration management, Ansible is a straight upgrade to most shells because of idempotency alone, never mind the fancier features like the more advanced modules, multi-node orchestration, or encrypted vaults. The YAML syntax is dumb and it has its issues, for sure, but it still does even the simple things much better than plain old shell.
Anyone who has any familiarity at all with UNIXy systems can learn Ansible from zero well enough in a day or two for it to start becoming truly useful, and if you don't have the foundation for that... why on Earth are you setting up a web server? I mean, it's of course fine to tinker with things for learning, but I was assuming a real deployment scenario.
Don't I still need to know shell programming for Ansible? Or at least know all the systems I want to manage with it inside out?
Yes, I need to learn tools at some point. But as I see it, I am not a system administrator of anything but my own network of 8 infrastructure hosts. The effort required to recreate this with ansible (and I don't think ansible can actually idempotently handle ALL of these devices, not without serious limitations) seems far greater than maintaining a few scripts and keeping backups. Also, I already know bash (unlike ansible).
> Ansible is a straight upgrade to most shells because of idempotency alone
So, as I said, I know nothing about Ansible. But idempotency implies that Ansible always starts from nothing and builds from there. Does this mean that every time I want to change my server I have to wait 15 minutes for it to re-install the distro and re-configure everything? Do I have to keep my state on a different server? I don't see how this can't be achieved with just as much hassle with a script?
Surely I misunderstand this. But if I did, then surely it's not THAT idempotent.
> Anyone who has any familiarity at all with UNIXy systems can learn Ansible from zero well enough in a day or two for it to start becoming truly useful
My problem with this is that every time I've looked into Ansible, it didn't look like a day of work. It looked like a week of work converting my entire infrastructure to it, for very little benefit, in addition to having to change the way I do a lot of things to fit the Ansible blessed method of doing them. It may take a day to learn Ansible but it probably takes even more time than that to learn it to a standard where I would consider the knowledge reliable. It would require making mistakes and lots of practice before I felt like I could quickly recover from any mistake I could make using it as well as avoid those mistakes. Not just that, but because of my nonstandard setup I would likely have to spend extra time learning Ansible well enough that I can actually replicate my nontrivial setup.
No, it doesn't. In Ansible you say something like "make sure apache is installed" and if apache is installed, nothing happens. If it isn't, it gets installed. Then you say "make sure apache is running" and if apache is running, nothing happens. If it isn't, it is started.
For example, it has idempotent modules for all sorts of things - contents in files, files and directories in the file system, etc - things that you COULD script in an ad-hoc and verbose way, but things which come built-in as one-liners in ansible.
It's quite convenient.
> idempotency implies that Ansible always starts from nothing and builds from there
...is wrong. It might be true that Ansible is unusable in your environment for some reason, but that's quite different fromage this specific false claim.
Here are a few more quotes that imply you should learn about Ansible before critiquing it for your use case (or, if you don't have time, then refrain from critiquing it in general):
> Don't I still need to know shell programming for Ansible?
No, Ansible uses a custom non-shell syntax and python modules. You can dip into shell scripts but you don't have to. Examples are everywhere in the Ansible documentation.
> Does this mean that every time I want to change my server I have to wait 15 minutes for it to re-install the distro and re-configure everything?
No. Ansible will examine your existing system and apply the changes you configure. Idempotency does not imply or require a functional-like OS or rebuilding from scratch; Ansible is more imperative.
No. Ansible has its own built in functions for creating files, managing systemd, docker and so on. These are built with idempotency in mind.
You can however call out to shell for situations where there is no built in. There are a lot of people who only ever use this role, and just see ansible as the fleet orchestration layer. Which imo defeats most of the benefits of using it, you might as well ssh a full script in that case.
As a side note I wouldn’t actually recommend Ansible for server management. Like you say learning all these blessed roles feels like relearning basics you already know and the syntax and directory structure is messy. It has no place if you use containers.
Do I still get idempotency if I do not use systemd or docker?
> You can however call out to shell for situations where there is no built in. There are a lot of people who only ever use this role, and just see ansible as the fleet orchestration layer. Which imo defeats most of the benefits of using it, you might as well ssh a full script in that case.
So it sounds like I wasn't entirely wrong in my first impressions that it would be useless for my situation where I don't think any of the "built ins" would really be suitable. Of the 8 machines on my network, only one has systemd (and I'm in the process of phasing it out because systemd seriously struggles to deal with services with dependencies on specific network interfaces being "UP", these issues are documented by freedesktop[0]).
> As a side note I wouldn’t actually recommend Ansible for server management.
Given the background of my infrastructure being a mixture of FreeBSD, OpenBSD, non-systemd Linux and systemd Linux machines. What would you recommend?
[0]: https://www.freedesktop.org/wiki/Software/systemd/NetworkTar...
As a result, all your arguments against Ansible seem to be based upon assumptions, some of them completely false.
Why don't you tell me which "arguments" (correction, they're my opinions) are based on false "assumptions" (correction, they're my impressions) rather than just giving me this blanket statement to work from?
If you want to learn Ansible, you don't even have to throw away your scripts; Ansible is a perfectly good way to run ad-hoc scripts if that solves your problem better than writing a full-blown playbook or even a custom module.
Ansible is weird and annoying in the beginning, but it's still a good tool to learn on top of your existing knowledge, because it provides extremely useful features beyond what's possible with plain old shell, and more importantly, it's a common language for system administration tasks that anyone can learn and understand without having to figure out how your specific scripts accomplish the things that Ansible gives you for free. The same applies to any management tool like Terraform, Puppet or even Kubernetes manifests. I put my expertise in my Ansible scripts and provide an easy interface to them such that a more junior person can, say, upgrade an Elasticsearch cluster by issuing a documented "make upgrade" (I like to use Makefiles to provide a neat interface for "standard" operations. "make help" and anyone can get going.) command that does everything correctly even though they have no idea how to actually upgrade it manually. If they wanted to learn, they have all the resources available required to read and understand my playbooks and figure it out without me being there to teach them the particulars of whatever unholy custom script setup I might have used instead.
Ansible is also mostly useful once you already have a server up and running but with 0 configuration; it's pretty bad at actually installing new servers, and I'd recommend using better tools for that part (Terraform, kickstart, or maybe just a script that clones an image). Just a manual next-next-next install is also perfectly acceptable way to get the base OS installed if the defaults are fine, though beyond a few servers it's a good idea to have a better process.
My perspective is that of someone who works with very varied systems daily, ranging in size from one to hundreds of nodes. I can manage that kind of scale alone because I use automation, and Ansible in particular is a tool that fits extremely well in the 1-20 "size range" for an environment; It is extremely lightweight and low-investment and can be used for even single nodes to great effect; once you get beyond a couple dozen, something more "heavyweight" like Puppet will start showing its usefulness.
As for idempotency, it's a very useful feature for automation: basically "Only do something if it is required". With a shell script, you have to implement manual checks for everything you run such that if you re-run a script on a system where it's already been run once, you won't accidentally break things by applying some things twice. A side benefit of this is that you can run your playbooks in "check mode", ie. "Tell me what you would do, but don't actually do it". Extremely useful and very error-prone to implement manually (Ansible doesn't always get it right either).
Using tools like grep + basic logic like || and && goes a long way...
I'm not saying there is no place for ansible, but in my personal experience, it's a very small one.
> Ansible is also mostly useful once you already have a server up and running but with 0 configuration; it's pretty bad at actually installing new servers, and I'd recommend using better tools for that part (Terraform, kickstart, or maybe just a script that clones an image).
Agreed!
More recently, I've found zfs clones of base installs surprisingly flexible.
Now I only with there was a way to do some kind of merge or reconciliation of zfs snapshots from a common ancestor that haven't diverged much in practice, spawning the differences into separate datasets per subdirectory (ex: if /a/ hasn't changed but only /a/b/c/d1 and /a/b/c/d2 differs, move d1 and d2 off to create a separate d dataset mounted in /a/b/c/ so you can keep the common parts identical )
I would argue that in most cases, they don't need anything but some documentation explaining what was installed, and why.
I will take a word file with screenshots over a broken script in an obscure language every single time.
> but there's no reason not to use something like Ansible that's designed for infrastructure automation,
There's a big one: my time isn't free.
If someone is willing to waste money on that, sure, I'll be happy to bill them for their extravagant tastes (but only after having done my best to explain them it's a waste of money)
And still, I will think about the next person that may have to maintain or tweak whay I wrote, so I will also leave a document full of screenshots in case they don't know ansible or whatever new fashionable tool that the client may have specifically requested.
> it's faster and the resulting system is of higher quality.
Not everything needs to be of high quality.
Forgive me if I'm assuming your gender, but I see a lot of black-and-white thinking among male sysadmins/devops: it's good or it's bad, it's high quality or it's not.
I prefer to have a "sufficient" degree of quality: if a checklist is enough, I will not waste time writing a script. If a shell script is enough, I will not waste time writing proper code - and so on.
> Over time you will also accumulate a library of bits and pieces that you can copy over to new setups, further improving your speed and quality
Except you assume a continuous progress, without any change of scope or tools, and with the tools themselves never evolving. It doesn't work like that: over time, you will accumulate a bunch of useless code for old versions.
Even small inconsequential changes (like unbound in debian 11 requesting spaces before some options, which wasn't the case before) will take some time and effort. Why waste your energy one one shots?
The do-nothing approach argues that you should avoid premature optimization, which strikes me a good approach in software in general.
And for me, having configuration management is the minimum level of acceptable quality. It's simply not possible to have acceptable quality of a system without some form of configuration management. I can't recall a single instance where I (or anyone else involved) ever said "wow, this unmanaged mess sure works well" :P
In some cases, the management can be as simple as a comment in some script explaining some part of the process was done manually, or simply a periodic snapshot backup of the server that can be restored when the configuration is broken, but the point is that a process must exist and it must be consciously chosen.
Free-form documentation is not an alternative to configuration management either; if you can document your configuration in a wiki, you might as well put it in a git repository in the form of a script or a template.
When done properly, It's the exact same amount of effort, except when you use automation tools, the documentation is in a format that's not ad-hoc and can actually help do the things it documents instead of requiring a human to interpret them (possibly introducing mistakes). "Setting up" Ansible requires literally nothing but a text file containing your server's hostname, and SSH access, which you already will have.
Also, I don't know where you got the idea that I would somehow assume unchanging scope? I am the first person to throw away useless code and tools; I consider code my worst enemy and it's practically my favourite activity to delete as much of it as I can. If some piece of automation is no longer fit for purpose, it gets rewritten if necessary. Throwing away code is no big deal, because the tools I use allow me to get things done efficiently enough that I can spend time refactoring and making sure the computer properly handles the tedious parts of whatever I'm working on.
Your unbound example is something that is trivially solved with configuration management. After an upgrade, you notice your configuration does not work, navigate into your git repository, update the template, and then deploy your changes to any server that happened to be running unbound using that same template (because you might have redundancy, if you're running DNS). If you make a mistake, you revert and try again. There is no manual ad-hoc process that comes even close to allowing you to manage change like this, but it is trivially enabled by existing, well-understood automation tools.
It does not take the same amount of effort, if only cause you mention how for unbound, you have to update the template.
For one shots, this is overkill.
All systems have a lifecycle, and even on a "trivial" system you have backups, access, monitoring, logging and security maintenance to worry about even before you consider how installing any useful software affects those things.
There are exceptions to any rule, of course, and I did in fact create a system where the configuration management is a snapshot backup just two weeks ago; but that system has no data on it, its lifecycle is expected to last for less than a year, and if/when it breaks, a backup restore can be performed without any additional considerations. It was also an emergency installation into a network that's not easily accessible with SSH, which is why I did not just use Ansible from the start.
I thought it would be a oneshot, but I did end up having to create a second instance of the system a few days later, fortunately with less emergency :P
Still even ignoring that, I fail to see what could possibly be overkill about literally 3 small files in a git repository. You call "overengineering" what is to me "5 minutes of effort with extremely relevant upsides". That's literally how much time it would take me to create a playbook for unbound if I already know what the configuration needs to look like; probably less, but most of the time will be lost to context-switching overhead.
My point being, most of the time will be spent actually configuring the software and the automation overhead is nothing in comparison compared to the value you get from it, and that's why I generally automate things by default: It provides more value than I put in effort.
When you start of learning configuration management and infra automation tools, there's a learning curve; in the beginning, you will be "wasting" time learning (what a silly statement) how to use your tools effectively, but with practice, you will learn how to effectively use the tools and where to apply them and how to approach managing specific kinds of systems such that over time, using the automation tools is simply easier and faster than doing it manually. That's what I meant when I mentioned "higher quality" earlier; you get it for free, with no effort, once you've put in a bit of practice first. It just sounds to me like you're arguing against doing things well in favour of doing things with strictly inferior tools.
Because even if you will only ever have a single production machine, it will have something go horribly wrong with it and need recreating from fresh hardware (or from a fresh VM or whatever).
I guess, if you're cloud-based, you could turn your finely tuned test box into a template, then you have something that is (effectively) scripted.
It's a no-brainer nowadays that software should be developed with version control. I don't see why infrastructure should be treated differently.
[0] Ansible playbooks are code, no matter what some people may think. It's a declarative'ish programming language with a silly syntax.
Male lifetime linux nerd here, who started as a sysadmin, checking in just to say that I agree with everything "policy related" in your comments on this article. Knowing where to tune the knob between "high quality"/"good architecture" vs "can i just get this done now and move on?" is difficult, at least I don't know how it could be taught other than experientially.
IME, the predilection to see things as black-or-white is more correlated with age, than gender.
Anyway, "not all men". :P
Your nick seemed familiar - now I remember, I read your great comment in "I just want to serve 5TB" earlier today!
I also agree with everything you wrote about simplicity in software development: I'll take almost every time some dirty php scripts running baremetal over Docker + Golang + Kubernetes + Terraform + Gitlab + Saltstack + Prometheus + the new fashionable tool because with so many parts now begging for attention, nothing will get done quickly - if we're lucky and something gets done.
Knowing where to tune the knob is indeed very difficult, and I'm afraid most people now are just doing a cargo cult of whatever google does, except they are not google, and they don't understand the tradeoffs or the possible alternatives.
But at the scale of most companies, it's a folly to sacrifice flexibility and simplicity to some unachievable desire for software perfection!
It's also a very costly hubris: I have been asked way too often to improve the performance after having thown very expansive hardware at the problem, that still performs miserably due to missing the big picture.
The solution was almost always removing the useless parts, or when trying to disentangle the architecture astronaut fancy mess would have been too costly, start from scratch with a saner design: most recently, I replaced a few hundreds java files (and test and stuff) by about 10 lines of bash, and 20 lines of awk.
My work is not fancy, but it works, unlike the previous solution that was going to be ready the next month, every month, for almost a year...
To all those who want to do things like google, maybe apply there instead of over engineering/polishing your CV with fancy keywords at your employer or client expense?
> IME, the predilection to see things as black-or-white is more correlated with age, than gender.
I had noticed this weird pattern, and it was my best explanation even if I didn't like it much, because it's sexist.
But your version seems more plausible (Occam's Razor!), so thanks a lot for taking the time to post!
But stuff connected to the internet needs to be or it will be compromised before you even finished installing it.
The word software configuration management can mean 2 related but different thing:
1. Configuration management in system engineering sense, which is is a process to systematically manage, organize, and control the changes in the documents, codes, and other entities during the Software Development Life Cycle (guru99).
2. Something to manage your config files, from something simple as python/bash scripts to full infrastructure-as-code solutions such as terraform and ansible
When I think about configuration management, I (and the parent) thinks about the second meaning, but if I googled that, all of the search results points to the first meaning
Let me tell you, just a wiki with a search is not enough beyond a certain size, and you hit that faster than you'd think.
No.
It's a powerful tool, Ansible, but let's not get carried way. There's a ton of complexity behind the scenes. If you over-do it you end up with a ream of ugly yaml and you're fighting with the tool as much as you are any real problems.
The solution ultimately is for PMs to get it into their heads that software and infrastructure require maintenance like anything else, and consistently refusing to schedule time for software/dev-tool maintenance (such as updating documentation) has the same effect as refusing to schedule time for physical equipment maintenance. Then and only then do engineers have the freedom to set up mandatory procedures and checklists for their work, the way all engineers should be allowed and encouraged to do.
why would your procedure be to do anything _other_ than "run script foo and do what it says"? If your procedure is not that, then your procedure doesn't reflect reality, and thus is outdated documentation that needs to be updated.
if the steps of the procedure only exist within the script then there's only one place to update it. And yes, this suggests the script should be very readable.
Configurations change all the time. There is no technological safeguard against someone forgetting to write down the change in the playbook script; it has to be organizational.
I mean, if /etc/hosts is more recent than /log-directory/03-static-hosts-in-etc or the md5 you have recorded for this file, a daemon can easily create a ticket / send an email to whoever was logged at the time of the change.
You can help solve the problem with technology, you just have to make the solution easier than working around it.
Not always, there are frequently ways to do an "end-run" around tools like Puppet and Ansible; take for example the following list of /etc/*.d directories on a Redhat distribution:
/etc/bash_completion.d
/etc/binfmt.d
/etc/chkconfig.d
/etc/cron.d
/etc/depmod.d
/etc/dracut.conf.d
/etc/gdbinit.d
/etc/grub.d
/etc/init.d
/etc/krb5.conf.d
/etc/ld.so.conf.d
/etc/logrotate.d
/etc/lsb-release.d
/etc/modprobe.d
/etc/modules-load.d
/etc/my.cnf.d
/etc/pam.d
/etc/popt.d
/etc/prelink.conf.d
/etc/profile.d
/etc/rc0.d
... <snip> ...
/etc/rc6.d
/etc/rc.d
/etc/rsyslog.d
/etc/rwtab.d
/etc/statetab.d
/etc/sudoers.d
/etc/sysctl.d
/etc/tmpfiles.d
/etc/xinetd.d
/etc/yum.repos.d
Someone can manually log onto the environment and drop additional configuration files into those directories that vastly effect what is run on the system (and when it's run in the case of cron.d for example).
"Idempotency" tools like Puppet and Ansible are very good at saying, "this file should exist in this directory with this MD5 hash", but not as good at saying "this directory shouldn't contain anything except these files".
Of course you can list all the files out that you consider to be valid and their signatures in the above directories, but that's going to break next time Redhat pushes an update that installs/removes files from those directories.
I guess you could setup an audit script that checks that all the files in those directories match the expected RPM signatures, and then account for any local customisations (additions, removals, changes etc). But you are starting to get into a lot of extra work there.
Point I am making, is that these tools are not as forcibly idempotent as a lot of people assume.
For example, I manage nodes with Puppet, and Puppet can and will "clean" things like sudoers.d, yum.repos.d, nginx.conf.d etc. of files that it does not manage.
I don't do this for every possible directory because so far configuration drift in those has not been a problem and generally whatever comes from the packages by default functions fine and crucially, the system can be rebuilt from scratch using the configuration that is managed, so the important bits are there.
I will simply start managing more directories as needed.
- I am still developing the procedure in the script, so it is premature to automate. A do-nothing script still benefits you by telling you exactly what to do - in my case, spitting out exact commands to run - but you can assess its steps before performing them.
- The script still gathers a lot of information and associates it together in order to figure out the correct commands. That work is valuable all by itself.
- Even though you have to run commands yourself, it's copy-and-paste vs. hand-typing, so it's already less error-prone.
- The script documents the procedure even without automation, so the benefit is immediate.
Now, I think that it's better for a do-nothing script to _evolve_ into a do-something script. But, if that effort is delayed or never happens, at least you've got something.
> At first glance, it might not be obvious that this script provides value. Maybe it looks like all we’ve done is make the instructions harder to read. But the value of a do-nothing script is immense:
> * It’s now much less likely that you’ll lose your place and skip a step. This makes it easier to maintain focus and power through the slog.
> * Each step of the procedure is now encapsulated in a function, which makes it possible to replace the text in any given step with code that performs the action automatically.
> * Over time, you’ll develop a library of useful steps, which will make future automation tasks more efficient.
> A do-nothing script doesn’t save your team any manual effort. It lowers the activation energy for automating tasks, which allows the team to eliminate toil over time.
Do nothing script:
echo “Go to your Google Account settings and set a vacation auto responder”
import oauth2 # I have already lost, there is no salvation in life, without truth or peace there is only the void that beckonsYou will, eventually, ideally.
But that's extra up-front cost (especially when do something involves a complex integration), the do-nothing script crystallizes the definition of the existing manual process allowing incremental automation of steps (also, making a to-do list for automation.) It is an example of “do the smallest useful unit of work”.
I prefer checklists. With a checklist, I can mark my progress as I go through the motions and, more importantly, interrupt the work even for several days, before I pick it up again. Checklists are much, much easier to adapt. They can also hold more information than progress indication, like important outputs of the procedure that need to be used somewhere else later, etc. They can be archived in case I need to understand how I did it on that particular occasion one year ago.
Google Docs added checklist a while ago and I think they are very handy. And very KISS.
echo "## Step 2/10 : preparing the SSH key for xxx"
(...)
KEY=$( cat .ssh/id_rsa.pub )
echo "## Do not forget to use this important output somewhere else later: $KEY"
> They can be archived in case I need to understand how I did it on that particular occasion one year ago.
(...)
# 20191102
# TODO: 2 years ago, we decide to use 4096 bits key, make sure to check the length
# in case I forget, how to do that, the number of bits can be forced with -b 4096
# 20201102
# WONTFIX: use ecdsa for the specific host yyyy
So I stand with the author: this is the absolute best solution there is: it can be as simple or as thorough as you need, while being very low tech and simple to use.
And when you find yourself needing to make say an Ansible configuration, you already have most of what you need.
An aborted script has a good chance of having left things in a broken, half-constructed state, such that simply re-executing the script results in more errors. Attempting to manually resume the script midway will have likely discarded important env variables that I must now figure out how to reconstruct.
...which isn't to say the scripting approach doesn't have value - poor checklist discipline by my coworkers has led me to painstakingly automate more than one perfectly fine checklist, and some stuff is quite scripting friendly - but I've also written my share of scripts that ate up more time in debugging and troubleshooting than I ever saved by writing them in the first place.
If you really want to argue about details, install git bash or msys2 or busybox and be done with it...
> Or on *nix if I absentmindedly used `bash` specific syntax and I'm now in a real `sh` shell.
Write your scripts in whatever you like as long as you are consistent!
Protip: starting your script with #!/bin/bash will limit the chances they are executed by sh
> An aborted script has a good chance of having left things in a broken, half-constructed state, such that simply re-executing the script results in more errors
Yes, please write good scripts that clean up and are idempotent (!!)
> but I've also written my share of scripts that ate up more time in debugging and troubleshooting than I ever saved by writing them in the first place.
With experience, you will avoid doing things like `echo something >> file` and will replace that with `grep something file || (echo something >> file)` (you get the idea)
"And now you have two problems!"
(My main goal here isn't to argue the details, but to exemplify the many ways scripts end up brittle and go boom in opaque ways, in a way that a checklist and common sense might not. To argue the details anyways: git bash / msys2 scripting always causes more problems than it solves IME - at that point, fire up a real *nix, or switch to a non-shell language like Python or Rust, instead of further entertaining the sunk cost fallacy.)
> Protip: starting your script with #!/bin/bash will limit the chances they are executed by sh
I do so when I can, but bash is missing frequently on oddball targets, so I have good uses for the russian roulette that is `#!/bin/sh`.
> Yes, please write good scripts that clean up and are idempotent (!!)
Easy enough if the underlying tools the scripts invoke are idempotent. Of course, the underlying tools are never idempotent - that'd be too easy.
> With experience, you will avoid doing things like `echo something >> file` and will replace that with `grep something file || (echo something >> file)` (you get the idea)
I've done plenty of that style, and it still tends to be brittle as heck.
`file` might be in use or protected (IDE? Anti-Virus? Previous unkilled build? Uninstall gone awry? Directory corruption? Owner/permissions issues? I've seen it all...)
`file` might fail mid-write (packaging/archiving stuff is a great way to run out of disk IME)
`file`'s format can change wildly between tooling versions, and the resulting errors from using the wrong format might be absolute garbage to the point of not even mentioning `file`.
What you grep for can depend on all kinds of implicit state (I once had a Android unit-testing script break because `adb logcat`'s default format depends on settings stored on the phone, and the default changed between Android versions at some point! Grepping errors to differentiate retryable errors vs fatal vs expected non-errors tends to break as tool authors don't think of error messages as part of their stable ABI, nor should they...)
Experience helps, but sometimes the problem is fundamentally intractable/dynamic enough, and the task executed infrequently enough, that even an automation expert would be better served by manually executing a checklist and hand-editing files, instead of writing a script and trying to decode the error messages when it inevitably explodes each time it runs due to unexpected dependencies on undocumented underlying state.
What I've ended up doing is to just write Word documents containing what I do. I keep my Word documents in a single directory on my OneDrive, so I can access them on all my machines.
Raspberry Pi iSCSI server? One document for that, containing the links to guides used, commands I've run and scripts made. I use headers to organize the sections, like one for compiling the custom kernel, one for configuring iSCSI on the Pi, one for using it on the NAS etc.
Need to create a new Docker SMB mount? Once the right incantation has been found, a small script is made and also added to the right Word document, ready to be pasted into the next machine I might need it on.
It's not terribly pretty and not at all fancy, but I found it's low enough barrier that it's easy to do and maintain, and it's very helpful to have it all in one place.
Had I been a Linux guy first and foremost I would probably have used something else than Word, but it works and I find the separation between the regular font for comments and monospace for script makes it easy to quickly distinguish. It's also easy to add screenshots for clarifications etc.
The images could be for things like performance graphs, interactive menu choices, photos of GPIO wiring etc. Not a huge loss to not have them, but given it's dead easy to add to a Word document, why not?
As a windows girl, I recommend you give a try to the notebooks like RStudio: you can add screenshots (or script them with ahk, nircmd etc to automatically screenshot at some points of the execution) and execute blocks of commands in about anything.
The notebook approach has the additional nice feature of stopping execution as soon as a block fails, giving you the opportunity to fix it before you continue from that point, something often more tedious with Linux scripts (where you need to commend the beginning if your script isn't idempotent)
Ideally, you would always write idempotent scripts, but who've got time for that :)
In practice, if you try to avoid wasting effort, it's often the icing on the cake, once everything else has been done. So I like notebook environments for this simple feature: piecewise execution with verbose output (similar to bash -xe)
That said, a directory per machine (and a subdirectory with all the drivers and specific software) on Onedrive with a RTF file full of screenshots (because wordpad is everywhere!) is how I work most of the time :)
Stop scripting random stuff to create multiple untracked resources in different places by hand.
Slap this[1] on top of this[2] and add whatever other services you want, and boom. Done.
1. https://stackoverflow.com/questions/49743220/how-do-i-create...
2. https://registry.terraform.io/providers/1Password/onepasswor...
Replace the steps with "Releasing a build to my app store" or something else.
There are a few solutions to this. Documentation with manual actions, a bunch of imperative code to create but not manage these things, or something that describes what you want without the “how I get it”.
If you’re at the stage where you think writing a bunch of random Python code to prompt a user with steps is an acceptable solution then I can’t see how the situation can get much worse.
Your proposed solution, as correctly pointed out in stack overflow, is a security nightmare.
> I should point out here that passing private keys around is generally a bad idea and you'd be much better having developers create their own key pairs and provide you with the public key that you (or them) can use to generate an AWS key pair
Using 1Password as a middleman here is quite convenient - it's a secure encrypted way of transfering them to the user that is accessible to the company when/if they need it.