Heard of Idempotency but unsure what it is? | Hacker News Reader

Heard of Idempotency but unsure what it is?

timmoth.github.io

29 pointstimmoth_j46 comments

I’ve heard of idempotency before, so this won’t change my opinion of it.

haha good one

> Idempotency refers to the ability of an operation to be performed multiple times whilst still yielding the same outcome.

I'm not a fan of this definition because it might give someone the wrong impression.

Suppose the operation is "deposit $10 to account 1234". If that operation is processed twice, a total of $10 should be deposited to account 1234 (for it to be idempotent). By the definition from the article, it sounds like perhaps it would be idempotent if the operation deposited $10 each time it was processed.

Yea, I see Idempotency as being you can submit same transaction multiple times without duplicate actions. So you to use your example, if I submitted "Deposit $10 to account 1234, transaction ID: ABC" four times, the system wouldn't execute it four times because it has enough data to determine if transaction is duplicate or not.

"Same state" instead of "same outcome" would make it clearer.

The example is not really great, but Idempotency is a very important concept for everything 'data'.

You very much want to be able to run the same pipeline several times and get the same result, or overwrite the results if you changed the inputs. You don't want to wonder if somewhere down the line, there will be duplicated data that will corrupt results for processes that consumes this data - you just want to be able to re-run your pipeline.

Sadly, that's not how many systems seem to be designed.

This fails to address that fundamental confusion, which is that there are several different (but related) things all called "idempotent":

1. a value `v` such that, for some pure binary function `f` (specified by context), `v === f(v, v)`

2. a pure binary function `g` such that, for all values `x`, `x === g(x, x)`

3. a pure unary function `h` such that, for all values `x`, `h(x) === h(h(x))`. Often this is formed as a partial evaluation of a binary function, as a relaxation of the previous kind.

4. a (non-pure) procedure that, when called with "the same" arguments (FSVO same), has no side-effects after the first time.

5. a binary relation `R` such that, for all values `x, z`, `R(x, z) <-> there exists some y such that R(x, y) AND R(y, z)`

(Edit: note before I clarified my variable names `v` used to be another `x`, and `g` and `h` both used to be another `f`)

I've yet to encounter the second definition, and the first only when f is associative (otherwise f(x,x) = x doesn't seem strong enough).

All the others are just the many ways x^n can equal x (for all n>0) for various associative operators. Reusing 'f' for both the binary functions and the unary ones is needlessly misleading by the way, the unary f behaves more like x in the other examples.

And yes if you want R^2 = R then the condition only needs to hold for some y. It's related to transitivity.

Good point about reusing variables, I've renamed all except the ones introduced locally.

I would only say that these are "isomorphic to" others; in the real world people do think of them differently, so it is important to enumerate them. And in particular people do think "the function", even though different the functions act differently.

(the real question is, did I ever see "left-idempotent" and "right-idempotent" somewhere?)

You don't need associativity:

(x ⋅ x) ⋅ x

(x) ⋅ x

x ⋅ (x)

x ⋅ (x ⋅ x)

You do need (some subset of) associativity:

  (y0 * x) * x ?= y0 * (x * x)
        y1 * x ?= y0 * x
            y2 ?= y1

y2 is not in general constrained to be the same as y1 otherwise.

I'm pretty lost, I thought idemopotent was basically a synonym for a projection (which sounds like your #3)... do you have any links to literature using it differently?

https://en.wikipedia.org/wiki/Idempotence#Definition

That seems consistent with #3, I don't see how it's different. Operator or function, the name doesn't change anything?

Operators and functions are effectively the same thing, yeah.

In CS we normally mean #4 though ... which can technically be thought of as a variation of #3 passing/returning "the whole universe" as an argument (and including the actual arguments in the implicit lambda capture), but thinking like that only makes sense for functional programmers.

Passing the entire universe as an argument... dang... either that's deep conceptually, or I'm shallow, or both!

The wikipedia definition is directly equivalent to #1. Try rewriting it with f(a, b) = a ⋅ b.

That's not to say #3 and #1 are unrelated, though.

Where I've been lost is where the comment said "which is that there are several different (but related) things"... if they're equivalent then isn't this just making my point?

Definition 3 is the same as definition 1. Defn. 3 is idempotence (in the sense of defn. 1) of the function f under function composition. In other words, f ∘ f = f.

A valuable insight, but doesn't that come with a caveat. Def 3 is only def 1 when the 'x' in def 1 is a function or an operator or some kind. If it's "truly a value" then x is instead an identity value and not idempotent isn't it?

And for #1, what if "f" is a function that selects the first of the two arguments? Then the relation in #1 holds true for all values of 'x', but that doesn't feel like idempotency to me. And we definitely can't say that 'x' is idempotent. As 'x' defined to be "add1" wouldn't be idempotent according to the definition, but would pass the relation if "f" is fst().

I think #1 only holds under compose or 'f's that are similar to compose when 'x' is a function. And at that point you've added so many restrictions you've made definition 3. So I think I disagree with definition #1 as being a def of idempotency.

I've renamed the variables for clarity. So now your comment can be stated as:

  v = h (no longer f)
  f = ∘

But when we're trying to bring clarity to the masses, remember that most people don't think in higher-order functions, so it is certainly necessary to state them separately.

Idempotency: the ability to be able to hit Ctrl+C several times in a row, just to be sure.

Oddly, C-g in Emacs isn't idempotent when it's hung or doing something slow.

https://www.gnu.org/software/emacs/manual/html_node/emacs/Em...

A more subtle case is that C-d (EOF for anyone except DOS) is widely not idempotent. Depending on the exact programs involved, it changes after pressing it about 10 times.

If I've told you once, I've told you a thousand times

That's more nullipotency

George Bush understood idempotency - "Fool me once, shame on... you. Fool me twice... can't get fooled again!"

If George Bush were idempotent he would get fooled every time tho. That he doesn’t get fooled a second time means he’s stateful, like strtok().

He would appear to get fooled every time, but that's just a cached response of him getting fooled. He wouldn't actually be getting fooled again.

So he's memoized then.

My mental model is to replace “do” words with “ensure”.

Instead of “send notification message”, it is “ensure recipient is notified.”

The example is flawed -- if there is an exception incurred before or during the sending of the email, the operation will be erroneously marked as "cancelled" without having completed all of the cancellation flow. Namely, with the example as given, it is possible to have a cancelled order for which an email has not been sent out.

Moving the second line (the one that changes the status to "cancelled") to the end fixes this issue, though it does not implement "exactly once" semantics for the sending of the email. If "exactly once" is desirable, then additional logic is required. But in every case, the example as given is incorrect.

IMO these statements contradict themselves:

> Idempotency refers to the ability of an operation to be performed multiple times whilst still yielding the same outcome

and

> The issue with the above code is that if the event is processed twice the EmailService will send a duplicate email.

In other words, the operation (three lines of code) yields the same result (sending an email) after repeated execution, and thus is idempotent isnt it? Or is the point that idempotency is harmful in this case?

The world is a system that has state. If two invocations of your function with the same parameters both mutate any part of that state, and nothing else mutated that part in between, then it's not idempotent. In this case, there are now two emails in the recipient's inbox.

I.e. "result" refers to state, not actions. "Send one email saying 'hi'" is an action. "There is one email saying 'hi' in their inbox" is a result.

There is also usually an implicit limiting scope (except in very pure functional contexts). E.g. just because two invocations both produce entries in an activity log somewhere doesn't usually count as "not idempotent".

You can also think of it in terms of declarative vs imperative. 'x = 1' is declarative (and idempotent). 'x += 1' is imperative. YMMV with this use of those terms, though.

To oversimplify, an idempotent function is "safe to spam", e.g. if you're not sure it worked the first time, you can do it again without worrying about duplication. A very relevant concept in networked communication.

Edit: I should also clarify that "same parameters" includes any state the function reads that is mutable over the scope in which you would like to define idempotency. I.e. if the function reads some external state that might change between calls, that counts as "parameters" in my simplified definition.

> If two invocations of your function with the same parameters both mutate any part of that state, and nothing else mutated that part in between, then it's not idempotent.

Still not accurate. What matters is whether N calls to the function changes the state differently than a single call. If not, then the function is idempotent.

Another issue with idempotency is that there's an implicit temporal component that is rarely explicitly discussed. Most people don't expect a function to necessarily be idempotent over long periods of time.

>What matters is whether N calls to the function changes the state differently than a single call.

I thought "any 2" would be the relevant subset of "N" that you can use to define idempotency in simpler terms. Also, I'm using the phrase "mutate state" as shorthand for your "change state differently", I think. I.e. if two things both change state, then by definition they changed it "differently". If two things both set x to 1, the second one didn't "change state".

>there's an implicit temporal component ... idempotent over long periods

This sounds a bit like what I mentioned about other things touching state between calls to your idempotent function.

Maybe I meant the same thing as you, but used different words?

> In other words, the operation (three lines of code) yields the same result (sending an email) after repeated execution, and thus is idempotent isnt it?

The meaning here is: after the operation is performed at least one time, exactly ONE mail is sent. Note the part "at least one time": it can be one, two, or more.

thanks, that clicks.

"Simply put, we can perform an idempotent operation multiple times without changing the result.

Furthermore, the operation must not cause any side effects after the first successful execution."

Baeldung got it right.

"outcome" here refers to "resulting state of the world", not "actions which are performed as a result of operation".

Which makes it a bad choice of terminology, because while both of those are things for which "outcome" is a natural word choice, the latter is at least as natural of a reading, and is what idempotency is defined in opposition to, so at best the statement is ambiguous on exactly the point it is trying to clarify.

Yes, the first statement can be easily misunderstood. It could also mean that the operation `f` does not have side effects. What it actually wanted to express is that `f(x) == f(f(x))`.

I heard of it once and now every time I remember.

I'd make a joke about nullipotency if I could remember what it meant

There's a draft standard for how to make POST and PATCH requests idempotent which is quite an interesting read https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempote...

There seem to be at least three meanings of "idempotence":

1. Some x is idempotent relative to f if:

  f(x, x)=x

2. Some f is idempotent if for all x:

  f(x, x)=x

3. Some f is idempotent if for all x:

  f(f(x))=f(x)

Im not sure why people post these kinds of things. First of all it does not define really anything, it just gropes at the subject trying to find something to hold on to.

And of course theres the crazy web articles that made funny memes about cows multiplying by calling POST over and over.

So Im not going to actually get into any of that text book stuff because alot of it is actually described in academic language, or simply being metaphorical, or describing a “result” as this 5 min blog post did.

So heres what idempotent actually is:

ID - empotent

Means literally “empowered by ID”, or ID aware if that feels better to you. It means that things that have been created will not be recreated because an ID prevents that from happening. And thats all it means. To make that mean something in your app, you have to convert that to a semantic and hopefully contractual interaction.

For example lets use REST.

POST /addresses

{ "first-name": “bob", “last_name": “johnson”, “street”: “foo… }

201 CREATED: { “id”: "8c3d03c4-7fe7-4355-afd4-25e838c4f884" "first-name": “bob", “last_name": “johnson”, “street”: “foo… }

The server returns the ID of the created address, which should now forever identify this address.

So what happens if we do the POST to addresses again and omit the ID? We will be creating a new address we already created. This is exactly what should happen.

But what should happen if we call POST again with the ID?

POST /addresses { “id”: "8c3d03c4-7fe7-4355-afd4-25e838c4f884" "first-name": “bob", “last_name": “johnson”, “street”: “foo… }

You have some options for the response here. Here are some obvious choices:

409 CONFLICT 400 BAD REQUEST

The point being that POST should not process requests to create items that already have an ID assigned as provided by the client.

You can return a response like “Call PUT dummy” but thats not necessary.

And thats really it some shops sort of muddy the waters by saying that IDEMPOTENCY means “upsert”.

Again thats describing a behavior with an intended result not IDEMPOTENCY. While its technically true that APIs should when provided with the same input have the same output it doesnt mean you should code a POST as an upsert.

We have semantics for creates and updates already in REST. So be empowered by ids and make your apps semantics match.

Isn't it "idempotence"?

Reply on news.ycombinator.com