In praise of –dry-run
henrikwarne.com
henrikwarne.com
The tool tells you what it would do in the current situation, you take a look and confirm that that's alright. Then you run it again without --dry-run, in a potentially different situation.
That's why I prefer Terraform's approach of having a "plan" mode. It doesn't just tell you what it would do but does so in the form of a plan it can later execute programmatically. Then, if any of the assumptions made during planning have changed, it can abort and roll back.
As a nice bonus, this pattern gives a good answer to the problem of having "if dry_run:" sprinkled everywhere: You have to separate the planning and execution in code anyway, so you can make the "just apply immediately" mode simply execute(plan()).
For something like in the article, I’m pretty sure a plan mode is overkill though.
Planning mode must involve making a domain specific language or data structure of some sort, which the execution mode will interpret and execute. I’m sure it would add a lot of complexity to a reporting tool where data is only collected once per day.
First, it is hard, especially in at least somewhat portable manner.
Second, serialization only matters if you cannot (storage, IPC) pass data around in-memory anyway. That's not the problem raised, though. Whatever the backing implementation, the plan, ultimately, consists of some instructions (verbs in parent) over objects (arguments in parent). Serializing instructions any other way than dropping non-portable named references requires one to define execution language, which is not an easy feat.
> The only thing you need to do is have a representation for the plan in your program.
That "only" is doing lifting heavier than you probably realize. Such representation, which is by the way specified to be executable bidirectionally (roll back capabilities), is a full blown program, so you end up implementing language spec, godegen and execution engines. In cases of relatively simple business models that is going to be the majority of the engineering effort.
I'm curious what portability concerns you've run into with JSON serialization. Unless you need to deal with binary data for some reason, I don't immediately see an issue.
> Such representation, which is by the way specified to be executable bidirectionally (roll back capabilities), is a full blown program
Of course this depends on the complexity of your problem, but I'd imagine this could be as simple as a few configuration flags for some problems. You have a function to execute the process that takes the configuration and a function to roll back that takes the same configuration. This does tie the representation very closely to the program itself so it doesn't work if you want to be able to change the program and have previously generated "plans" continue to work.
The hard part concerns instructions and it is not technical implementation of serializing an in-memory data structures into serialization format (be it JSON or something bespoke) that is the root of complexity.
> You have a function to execute the process that takes the configuration and a function to roll back that takes the same configuration.
Don't forget granularity and state tracking. The opposite of a seemingly simple operation like "set config option foo to bar" is not a straightforward inverse: you need to track the previous value. Does the dry run stop at computing the final value for foo and leaves possible access control issues to surface during real run or does it perform "write nothing" operation to catch those?
> This does tie the representation very closely to the program itself so it doesn't work if you want to be able to change the program and have previously generated "plans" continue to work.
Why serialize then? Dump everything into one process space and call the native functions. Serialization implies either strictly, out of band controlled interfaces, which is a fragile implementation of codegen+interpreter machinery.
Ansible implementation is okay, but not perfect (plus, this is difficult to implement properly). For cases like file changes, it works, but if you install a package and rely on it later, the --check command will fail. So I am finding myself adding conditions like "is this a --check run?"
Ansible is treated as an idempotent tool, which it's not. If I delete a package from the list, then it will pollute the system until I create a set of "tearing-down" jobs.
Probably, Nix is a better alternative.
I've split the process into three parts:
1. Walk the filesystem, capture the current state of the files, and write out a plan to disk.
2. Make sure the state of the files from step 1 has not changed, then execute the plan. Capture the new state of the files. Additionally, log all operations to disk in a journal.
3. Validate that no data was lost or unexpectedly changed using the captured file state from steps 1 and 2. Manually look at the operations log (or dump it into an LLM) to make sure nothing looks off.
These three steps can be three separate scripts, or three flags to the same script.
Not just any compiler, but a non-typesafe, ad-hoc, informally specified grammar with a bunch of unspecified or under-specified behaviour.
Not sure if we can call this a win :-)
And how do you imagine doing that for the "rm" command?
I think the current semantic of "rm" works fine. But I understand the new world where we'll perhaps gonna be deleting single files using Terraform or cluster of machines, or possibly LLMs/AI agents.
Folks usually want reversibility in the context of a logical set of operations, like a Terraform apply. For shell commands like "rm", the logical set of operations might be a session (having a ZFS snapshot taken on terminal session start, with a sane auto-delete/age-out rotation, would be super useful! I might script that up in my shell profile in fact) or a script, or a task/prompt/re-prompt of an AI agent. But yeah, it definitely shouldn't happen at the level of a singular "rm" call.
Since filesystem snapshots (in most snapshot-capable filesystems, not just ZFS) are very simple to create, and are constant-time or otherwise extremely fast to perform, the overhead of taking this approach wouldn't be too hard.
Not to take anything away from your comment but just to add a related story... the previous big AWS outage had an unforeseen race condition between their DNS planner vs DNS executor:
>[...] Right before this event started, one DNS Enactor experienced unusually high delays needing to retry its update on several of the DNS endpoints. As it was slowly working through the endpoints, several other things were also happening. First, the DNS Planner continued to run and produced many newer generations of plans. Second, one of the other DNS Enactors then began applying one of the newer plans and rapidly progressed through all of the endpoints. The timing of these events triggered the latent race condition. When the second Enactor (applying the newest plan) completed its endpoint updates, it then invoked the plan clean-up process, which identifies plans that are significantly older than the one it just applied and deletes them. At the same time that this clean-up process was invoked, the first Enactor (which had been unusually delayed) applied its much older plan to the regional DDB endpoint, overwriting the newer plan. The check that was made at the start of the plan application process, which ensures that the plan is newer than the previously applied plan, was stale by this time due to the unusually high delays in Enactor processing. [...]
previous HN thread: https://news.ycombinator.com/item?id=45677139
Just last week I was writing a demo-focused Python file called `safetykit.py`, which has its first demo as this:
def praise_dryrun(dryrun: bool = True) -> None:
...
def gather(paths):
files = []
for pattern in paths:
files.extend(glob.glob(pattern))
return files
def execute(files):
for f in files:
os.remove(f)
files = gather([os.path.join(tmp_dir, "*.txt")])
if dryrun:
print(f"Would remove: {files}")
else:
execute(files)
Assuming our system is complex enough. I guess it sits between if dry_run and execute(plan()) in its complexity.
I've yet to have anyone mistakenly modify anything when they need to pass --commit, when I've repeatedly had people repeatedly accidentally modify stuff because they forgot --dry-run.
$ rm file.bin
$ rm —-commit file.bin
$ cat foo.txt > bar.txt
$ cat foo.txt | tee —-write-for-real bar.txt
$ cp balm.mp3 pow.mp3
$ cp —-i-mean-it balm.mp3 pow.mp3
if [ -n "$DRY_RUN" ] ; then
alias rm='echo rm'
alias cp='echo cp'
fi
`rm` a single file? Fine. `rm /`? Maybe block that one.
What it should do is generate a user-friendly overview of what's to be deleted, by grouping files together by some criteria, e.g. by directory, so you'd only need to confirm a few times regardless of how many files you want to delete.
$ rm -rf /
rm: it is dangerous to operate recursively on '/'
rm: use --no-preserve-root to override this failsafehttps://learn.microsoft.com/en-us/powershell/module/microsof...
I use "live run" now, which I think gets the point across without being sort of uncomfortable.
--make-it-so
--do-the-thing
--go-nuts
--safety-off
So many fun options.
For example, our database restore script has a parameter `--yes-delete-all-data-in` and it needs to be parametrized with the PostgreSQL cluster name. So a command with `--yes-delete-all-data-in=pg-accounting` works on exactly one system and not on other systems.
--moistenPros and cons to each but I did like that because it was much more difficult to fat finger or absentmindedly use the wrong parameter.
By default it'll only tell you which files are identical between the two parallel directory structures.
If you want it to actually replace the files with hard links, you have to use the --execute flag.
My latest script which deletes the entire content of a downloaded Sharepoint (locally only) and the relevant MS365 account from the computer runs by default in a read-only mode. You have to run it with an explicit flag to allow for changes.
Also, before it actually deletes the account, you need to explicitly type DELETE-ACCOUNT in order to confirm that this is indeed your intent.
So far, nobody managed to screw up, even in heated situations at client's place.
And it's pretty nice. The downside is if you get used to that behavior in things that don't have it the consequences can be bad. (Like the common `alias rm='rm -i'`. No, that's just a trap; don't do it.)
Put another way: your dry code should do everything up until the point that database writes / API calls / etc actually happen. Don't bail too early
For little scripts, I'm not writing unit tests- running it is the test. But I want to be able to iterate without side effects, so it's important that the dry mode be as representative as possible for what'll happen when something is run for real.
For me personally, I tend to draw the line at write operations. So in your example, I'd want a dry run to verify the permissions that it can (if I expect those to be a problem). But if that can't easily be done without a write, then maybe it's not worth it. There are also situations where you want a dry run to be really fast, so you forego some checks (allowing for more surprises later). Really just depends.
Not really. Testing is a way to increase confidence that code does what it is specified to do, because it is cheaper than full-blown formal analysis :)
The problem raised by OP here is granularity. Operation like `update(record, field, value)` is itself a tree of smaller sub-operations that may do some permissions checking, locking, network calls, even checking for presence of record if it has upsert semantics, all of which could fail. A dry run with a plan that is too coarse can succeed while the actual operation fails over things left unchecked.
Right - so the dry-run has to actually do as much of 'what will happen' as possible, except the actual things.
You want to put the check as far down, close to the 'action' as possible. You don't want any additional business logic gated by the dry run check.
Also, if I'm being honest, it's much better to use `--wet-run` for the production run than to ask people to run `--dry-run` for the test run. Less likely to accidentally fire off the real stuff.
Then I would say you have not arrived at the optimal solution. Keep looking.
I'm using MVC design patterns for some codebases, I'm using DDD plus Event sourcing and Event Driven for others.
I suspect that you are thinking of a small subset of design patterns (eg. Gang of Four derived patterns like Visitor, Strategy, or Iterator )
All examples of OO nonsense. There is only one pattern you need (functions) (tongue-in-cheek): https://fsharpforfunandprofit.com/fppatterns/
Edit: Also consider using the ELM architecture instead of MVC: https://guide.elm-lang.org/architecture/
Did not like, Won't use again.
MVC is well understood and works perfectly in my experience.
edit:
Even though you were tongue in cheek about F# - this stood out on the page
> The functional programming community has design patterns and principles as well.
Why hasn't it cut through, like OO did?
and
Beam has been around since the 90s, had a good marquee project (RabbitMQ) but still has low adoption
no patterns. -> Everything must follow the gang of four's patterns!!!! -> omg I can't read code anymore I'm just looking at factories. No more patterns!!! -> Patterns are useful as a response to very specific contexts.
I remember being religious about strategy patterns on an app I developed once where I kept the db layer separated from the code so that I could do data management as a strategy. Theoretically this would mean that if I ever switched DBs it would be effortless to create a new strategy and swap it out using a config. I could even do tests using in memory structures instead of DBs which made TDD ultra fast.
DB switchover never happened and the effort I put into maintaining the pattern was more than the effort it would have taken me to swap a db out later :,) .
For example, I have an app that uses Postgres as the database. I have a lot of functions, schemas, triggers, constraints in Postgres for modifying database state, because the database is 100x faster at this than my application will ever be. If I have an in-memory version of Postgres, it would need to replicate those Postgres features, and at that point I really should just be standing up a database and testing against it.
I have worked with people claiming that unit tests need to hermetically run in-memory, because reasons. Ok, I don't disagree, but if my bug or feature requires testing that the database is modified correctly, I need to test against a real database! Your in-memory mock will not replicate the behavior of a database _ask me how I know_ ...
These days, Docker makes this so easy that it's just lazy to not standup a database container and write tests againts it.
For services, I prefer having them detect where they are running. Ie if it's running in a dev environment, it's going to use a dev db by default.
The program should default to actually doing whatever thing you're asking it to do.
On the other hand it would be great if every tool had an --undo argument that would undo the last thing that program did.
> NILFS2 is a log-structured file system (LFS) supporting continuous snapshotting. In addition to versioning capability of the entire file system, users can even restore files mistakenly overwritten or destroyed just a few seconds ago.
https://docs.kernel.org/filesystems/nilfs2.html
It doesn't always work but sometimes I use `diff` to help with that. For example, if you have a complicated `sed` replacement that you plan to run on a file, you can use diff like this `diff -u <(echo "hello") <(echo "hello" | sed "s/hello/hi/g")` to help show what has changed.
I've written about the value of dry run too at: https://nickjanetakis.com/blog/cli-tools-that-support-previe...
I guess the goal for both is to give the user a chance to get out of autopilot, and avoid up-arrowing and re-executing.
Nowadays I still use the technique for a lot of the tools I make. I typically do a Terraform-like approach: create a plan, validate the plan, render the plan to the user, exit early if we're in dry-run mode, and apply the plan as the final step. Whether dry-run is enabled by default depends on the risk of the operations and who will be using the tool.
This makes it exceedingly clear what actions the tool will do. Plus it has the added benefit of being able to split the plan and apply steps into two separate steps. For example, you can create a plan, serialize it to JSON, store it (e.g. in VCS, Jira ticket, whatever) then apply it during a change window.
plan = createPlan()
print(plan.string())
if (dryRun){
return
}
plan.apply()And this works well if your CLI command is simply performing a single operation, e.g. call this REST API
But the moment it starts to do anything more complex: e.g. call API1, and then send the results of API1 to API2 – it becomes a lot more difficult
Of course, you can simulate what API1 is likely to have returned; but suddenly you have something a lot more complex and error-prone than just `if not dry_run:`
Rest bloat is insane. Kidss nowadays wants EVERYTHING to run over IP/TCP/https. Why?!
Learn to write local tools first.
Quite handy.
This should always be included in any application that has a clear plan-then-execute flow, and it's definitely nice to have in other cases as well.
The code is not littered with dry-run flag checks; the internal code doesn’t even know that a dry run is possible. Everything is rolled back at the end if needed.
All database referential integrity checks run correctly.
Some drawbacks: any audit logging should run in a separate transaction if you want to log dry runs.
> These reports show that a dry run in the jargon of the fire service at this period [1880s–1890s] was one that didn’t involve the use of water, as opposed to a wet run that did.
I've resorted to copying the entire directory (including the .git part) and then trying on the copy. The issue is that I'm working on a C++ program that has a few gigabytes of data.
"Do you really want to 'rm -rf /'? Type 'fiberglass' to proceed."
--nuke Delete all of the archives stored. To protect against accidental
data loss, tarsnap will ask you to type the text "No Tomorrow"
when using the --nuke command.Anybody who rebooted a wrong server can say that this tool is brilliant.
Like "--dry-run" but for "reboot."
Dialogs that pop up and ask "Are you sure you want to delete ...?" -> users just automatically click yes, because they already did that the last 10 times and just want to get on with their work.
Logged in to server "alpha" instead of "delta" because you thought that's the right one. Tool asks you to write the server name. You type "alpha" because you know you're on alpha. Reboots the wrong server.
Github ask you to confirm the repo name before deleting by typing it into a text field. User looks at what the repo name is and types it without thinking. Or, like lazy me, mark and drag the displayed name into the field, so you don't even have to type.
The point is, users already decided to do the action when they started. It's nearly impossible to consistently make them stop and re-evaluate their decision, because that's extremely high friction and annoying. They quickly learn to circumvent the friction as efficiently as possible (i.e. without thinking about it).
A better solution is to just do it, but let the user undo it if it was a mistake (not always possible of course).
Yes, the same, unfortunately. This rail guard works for only a couple of days or weeks when the perception is fresh. After that, it's always "select, copy, paste, confirm" without thinking.
I believe the better approach is to practice "Chaos Monkeys" and improve the infrastructure to survive even if someone makes a mistake.
Pretty easy to solve this sort of stuff with timers: "Sure, the repo will be deleted in <timer>. Press here to abort: <button>."
Or have the timers on the backend and just show the user an undo button.
Otherwise it's not very wise to trust the application on what should be a deputy responsibility.
Nowadays I'd probably use OverlayFS (or just Docker) to see what the changes would be, without ever risking the original FS.
There's two main ways overlays are used, first at build time, each line/command generates a new overlay based on the previous base, so when you do something like
FROM debian RUN apt-get update
it creates a base from the debian image , and then creates an overlay that only contains the changes introduced by apt-get update.
If you use docker inspect or docker show on the image you get a json showing exactly where the overlay directories are, you just need to navigate the overlay directory.
Second: on runtime. [Assuming you are not using volumes, (and if you use volumes, just make sure the volume starts out as empty, instead of sharing your host files)] OverlayFS is used for the runtime file changes as well, the last image is used as a base, and every files changed during runtime are added to the runtime overlay. That filesystem won't be deleted, if you only stop the docker container, the runtime files will still be present, and you can reach them by docker inspecting the running docker processes/instances, and then navigating the overlay fs as you would any directory.
You can also just use overlayfs, as far as I recall, you just use mount and unmount while specifying the OverlayFS driver and special parameters like base and overlay. Conjugating a chain of overlays is a bit more complex, but it's the same interface.
I prefer the inverse, better, though. Default off, and then add `--commit` or `--just-do-it` to make it actually run.
Sounds like a case for the state machine pattern
It's even more relevant now that you can get the LLMs/CLI agents to use your deterministic CLI tools.
(Don't try it in a terminal window, though)
--really
--really-really
--yolo
Not to be overly critical (I think it's great OP found value in adding and using --dry-run), but I am willing to bet that this was a suggestion/addition from a coding agent (and most likely Claude code/Opus). Having used it myself to build various CLI tools in different languages, it almost always creates that option when iterating on CLIs. To the point where it's almost a tell. I wonder if we're entering a moment of convergence where all the tools will have similar patterns/options because they are similarly written by agents.
> I remembered how helpful that was, so I decided to add it to my command as well.
He mentions the reason he added it, and it's a compelling enough story to be true.
Any time you have a script that needs to run for a long time or might involve destructive actions, having a way to run the script in a "tell me what you would do without actually doing it" mode is a fairly obvious user story to throw in.
This isn't something with surprising nuance like how a McDonald's milkshake serves a non-food "job to be done" during a shopper's morning commute. As evidenced by all the others in this thread pointing out other tools that do similar things, it's a fairly obvious idea to come up with after actually using a new tool.
You'd be more likely to learn about it doing product comparisons of other tools, although since there is a lot of common art for AI training to draw from, yes it is also possible to hear about it from your AI first.