In praise of --dry-run

What I don't like is passing those as params as args. Currying this stuff around is very error prone and cumbersome. I prefer to set environment variables for DRY_RUN and DEBUG (with debug accepting higher integer levels for more debugging).

This works wonderfully for me since I never have to worry I didn't pass something along correctly, I'm always asking the global set at runtime.

--dry-run is one of the best motivations for using granular, extensible effects

when you constrain literally every bit of IO your program is allowed to do, you can now 100% know you've stubbed them all out when doing a dry run

Another important property is idempotency. Especially if the script involves network requests or moving files around, you'll want to reach the goal by re-running in case something breaks half-way.

- cancel

- undo/redo

- progress bars

Progress bars with accurate timing are notoriously difficult, or even impossible to get right. But even regardless of timing, having a progress bars that really shows progress and doesn't freeze is hard. Every slow operation has to have some sort of callback mechanism to update progress, and you have to know the number of steps in advance.

Undo requires, for each operation, to know how to roll back. You also need to have every operation go through the undo stack. Another option is to have an efficient snapshot system, which may be just as hard or even harder.

Cancel is actually the hardest to do right because it combines the difficulties of both the progress bar and undo. You have to have a way to interrupt a long operation at any time and get back to before it started. And because "cancel" is most often used when things go wrong (ex: disk full, bad connection, ...) you have to be very careful with error handling.

It's an architectural step a lot of people skip, and by doing so you often end up in a situation where the only way to make things faster is via caching, and chasing caching bugs for the rest of your tenure.

One of the less appreciated aspects of model-view-controller is that usually it demands that you plan out your action before you do it - especially when that action is read-only. Like a cooking recipe, you gather up all of the ingredients at the beginning and then use them afterward.

By fetching all of the data early, you reduce the length of the read transactions to the database, allowing MVCC to work more efficiently. You also paint a picture of the dependency tree in your application, up high where people can spot performance regressions as they show up in the architecture - where there's a better opportunity to intercede, and to create teachable moments.

To make dry-run work you are best served by book-ending your reads and your writes, because if the writes are spread across your codebase how do you disable all of the writes? And if any reads are after writes, how do you simulate that in your dry-run code? It won't be easy, that's for sure.

The problem is that we tend to write code stream-of-consciousness, grabbing things just as we need them instead of planning things out. This results in a web of data dependencies that is scattered throughout the code and difficult or impossible to reason about. This is where your caching insanity really kicks into high gear.

To my eye, Dependency Injection was sort of a compromise. You still get a good deal of the ability to statically analyze the code but you can work more stream-of-consciousness on a new feature. But it does rob you of some more powerful caching mechanisms (memoization, hand tuning of concurrency control, etc)

Certainly it's not something that can be applied for every use case, but when it can, it has so many benefits - an audit trail of what actually happened, ability to undo, and a common structure and approach for what dry runs should do (just output the log)

Seems to be followed most of the time.

Same here. In addition if there's never a use case for running in production (e.g. at one point we had a command that reset a dev environment to a clean slate) then the command will actually specifically check for the production environment and refuse to run as an extra failsafe.

    1. do-step-one --dry-run
    2. (do validation)
    3. do-step-one --real-run
    4. (do validation)

Internally, I'd like an enum flag that's clearly dry_run or real_run, so the guards are using positive logic:

    switch(run) {
      case dry_run:
        print("Would do this...");
      case real_run:
        do_real_thing();
    }

—i-acknowledge-this-is-dangerous-and-i-gave-this-more-than-cursory-thought

It's not material to the point, but I think there's a small typo on line 12 of the second "Finishing the example" snippet: it looks like

    let instructions = gather args

    let instructions = gather inputGlobs

Consider a conceptually very similar case: instead of finding and deleting files in a filesystem, think about finding and deleting lines matching a pattern within a file. Then the query is something like grep...but what do you put for the command on the other side of the pipe?

Of course, you can tell grep to output line numbers and use a command that operates on line numbers, or similar. The point is, in order to achieve this kind of segregation, you need some common way of naming operands on both sides of the divide. And naming things is hard, so segregating things this way is hard, and thus there's a lot of tools with mixed query/command responsibilities.

  DRYRUN=echo
  
  cmd() {
    $DRYRUN "$@"
  }

?

So if you need to install a package which sets up a systemd service, the subsequent dry-run of managing that systemd service would fail because the service doesn't exist yet. Or you need to make assumptions in the service dry run that it should have been set up before.

You can just assume that the service would have been setup and report that you were, say, enabling and starting the service. Or you could require some kind of hint to be setup from the package installation to the specific service configuration (for well known things this could be done by default, but for arbitrary user-supplied packages this cannot be done). Or you could list the entire package contents and try to determine if the manifest looks like it is configuring a service.

And it still may fail because you don't know the contents of that file without extracting it and the service may not parse at all.

And that's just a simple package-service interaction example. You could spend a week noodling on how to do that fairly precisely, then there's a hundred or a thousand other interactions to get correct.

You're being told not to do the thing, so there's fundamentally a black-box there, and you need to figure out how much you're going to cheat, how much you're going to try to crack open the black box into some kind of sandbox so you can figure out its internals, and how much you're going to offload onto the user. Not actually an easy problem at all.

Maybe in your case the level you are trying to report dry-run information at is too granular.

Or maybe you have too tight coupling between your code that determines what needs to be changed and that which actually does the change and you might want to refactor the code that determines changes to the code that makes the changes.

This is problematic in multiple ways:

1) The script uses a single repository of project metadata to derive the rest of the build. This repo needs to be present and updated to figure out what to do.

2) There are sequential dependencies on a great deal of steps that can fail without much predictability (including network traffic, git commands, and the module build itself). But in a mode where you're compiling every day, even a transient module build failure does not necessarily mean you shouldn't press on to also try and build dependent modules.

I've ended up using an amalgamation of logical steps:

* If a pass/fail check can be performed non-destructively, do it (e.g. a dry run build will fail if a needed host system install is missing, or the script will decide on git-pull vs. git-clone correctly based on whether the source dir is present) * If pass/fail can't be done, just assume success during dry run (e.g. for git update or a build step) * For the metadata in particular, download it to a temp dir if we're in dry run mode and there's no older metadata version available. * Have the user pass in whether they want a build failure to stop the whole build or not.

For separate reasons, I've also needed to maintain logic to help modules with build systems that didn't support a separate build directory to act as if the build was being executed in the source directory (this was done by copying symlinks to the source directory contents into the proper build directory, file by file).

Having this separate, disposable directory made it safer to not have to be perfect on tracking files changes or alterations perfectly. And of course it helped greatly being able to offload the version control heavy lifting to CVS (back in the day, then Subversion and now Git).

I have never actually encountered such a race condition while using any tool I've written this way. But I do remember one place where I specifically didn't add a feature to the tool because it was so vulnerable to a TOC/TOU race condition. Instead, I turned that feature into a "did I just manage to do the thing successfully, and if not, why not" check at the end of the "do-it" stage.

So yes, that concern is extremely valid, and if you're doing an operation whose validity is liable to change after you've done the check, then you do just need to follow a different approach for that operation. (But then there's probably no possibility even in principle of a `--dry-run` for such an operation anyway!)

"Would have gotten all servers tagged foo (currently 1,341) and deleted them"

See this past submission: https://news.ycombinator.com/item?id=18499712