We were performing some migrations to our servers and we weren't sure we had nailed the process down. Automating the procedure has a problem in that I can't just "reset" the server and start again. I could try to make all my operations idempotent or using a babushka style "Check - Set - Check" methodology, but that would greatly expand the work for a one off procedure that once it's done it's done.
I wasn't even sure of the generality of the steps, and they were complicated enough that copy and pasting them with all their parameters was tedious. So I created a structure just like this!
I called it a living procedure. An excerpt:
#!/bin/bash
source ./procedure-lib.sh
question "Which client is this?" old_client
question "What worker is $old_client on?" old_worker
question "What is the targets name going to be? (it's fine if it's still $old_client)" new_client
question "What worker is $new_client going to be on?" new_worker
question "What branch is $new_client going to be on?" new_branch
question "What should we name the migration? (typically something like migraiton-$(date --iso))" migration_name
#================ Initialize the client on the new worker ================
step "Login to $new_worker and Initialize $new_client in legacy mode using run.sh init_legacy, set their branch to $new_branch"\
'eg.
ssh '"$new_worker"'.lightship.works "sudo -i bash -c \"
These question functions also remembered the answer you put in last time (it was based off what the text of the question was, so questions that depend on the answer of previous questions did not presume anything) so that if you just hit "enter" then it would load the last one. This was vital because when a procedure ended up incorrect or needed to be amended, you could ctrl+c the script, fix it, then spam enter until you reached the point you were last time.I've been thinking about creating a better structure to these scripts that allow you to gradually transform these living procedures into babushka style "Check Set Check" automations.