More Control with Idempotency
buttondown.email
buttondown.email
This article glances off a really interesting related problem - the choice between “at most once”, “at least once” and “exactly once” events.
If the process is due to send an alert mail, but the process (or the server or whatever) dies just after the mail is sent but before the status is updated in the system’s state.
Yeah, I'm not a distributed systems expert, but I think this is one of the foundational problems of distributed systems. And that exactly-once delivery is considered to be impossible over a lossy channel. (That excludes abstractions used to create lossless channels over a lossy channel, such as TCP.) Hence the choice between at-most-once and at-least-once delivery, of which I think most of us choose at-least-once.
From an architecture point of view, maybe you’d like to change the space vs. time tradeoff for a component (maybe apply memoization) in response to a change of business need but that can be nasty when the component itself knows how to go lookup business date by itself or how to send a notification etc.
Determinism/purity could be defined the same way; the difference hinges on your interpretation of "run the program multiple times on the same input". That is, if you run `f(x)` and then `f(x)` again (on the same input, see!), you ought to get the same result. Given the spotlight pure functions have received for the last N years, maybe this distinction is worth making explicitly?
The "on the same input" part is, I think, what puts it uncomfortably close to determinism. Hmm...
> How I like to think about idempotence is that we have some desired state (the input), and then we have an actual state and the purpose of the program is to bring the actual state to the desired state.
This is a lovely way to look at it!
Deterministic (but not idempotent): send_message(id=1, body) without verifying the state of id, could result in the message being received multiple times.
Idempotent: send_message(id=1, body) would verify that the id token hadn't been consumed, before acting, thus only allowing the message to be sent one time.
If you imagine the hidden state that `send_message` reads and writes to be part of the input, then when you run it the second time, its inputs have changed. So it wouldn't be correct to say you're running it multiple times "on the same input".
These details are not obvious from the given definition, and that's why I say it's uncomfortably close to purity. Idempotence is very much not about purity (though you can have both), and that's why I like the author's goal-oriented explanation better: it makes the role of state very explicit.
You could easily split the deterministic action, and comparison/verification against existing state into two different Pure functions. Being able to perform the verification separately is valuable for addressing the "at most once", "at least once", and "exactly once" assurance described in another comment in this the thread.
I was casually opening my scope up to the outer state, in the example, to express the "assurance" portion of the definition. But you're correct, it's an erroneous example that doesn't pass the more rigorous definition of Pure Determinism, when applied the local scope of the function.
If we consider the state to be part of the input, then it's easier to say that when f(x) = y, then f(y) = y as well. In other words, f(f(x)) = f(x). Which, as it just so happens, is literally the mathematical definition of idempotence which the programming concept is based on.
In any case, this still gets to the same overall conclusion: the point is to arrive at a specific destination, not to move towards it. That shift of perspective is helpful in framing problems towards an idempotent solution.
Agreed! The idempotent operation acts on some state; how that state is modeled is an independent concern.
> In other words, f(f(x)) = f(x). Which, as it just so happens, is literally the mathematical definition of idempotence which the programming concept is based on.
That's the one I like the most, because I try to concentrate impurity to a small region of the overall program. In a language like Haskell, in some ways your hand is forced! But, as you say, it's biased toward a particular way of modeling state and effects.
My point is, if a program is said to be idempotent if running it multiple times on the same input produces the same output, then you exclude the Haskell approach to idempotence. Half the point of `f(f(x)) = f(x)` is that the input f(x) is typically not the same input as x itself!
I wish I had thought of that example too when I wrote about idempotency!
I wrote a post about idempotence as well a couple months back (https://chrisprijic.com/idempotency-is-key/) and use the definition: "its the output + side-effects that should remain the same, regardless of how many times the function with the same input is run".
I had struggled with "on the same input" when I was putting the post together until I thought about how it usually is due to side-effects that idempotency becomes desirable.