Hacker News Reader

Comment by phamilton | Hacker News Reader

Embrace idempotence. Being able to perform am operation multiple times with the same result adds a lot of flexibility.

For example, using the rails TODO list, if you add a task to "buy milk" and then on another client view the list and don't see "buy milk" there (because consistency hasn't been reached), you might be inclined to think you forgot to save it or something. Then you might enter it again. If this is not idempotent (as is the case on the classic tutorial) you will end up with 2 "buy milk" tasks instead of just one.

View on HN

This is actually a great advice - but not all operations can be made idempotent. Adding and substracting money from bank accounts comes to mind (although you can make the operation "write transaction to some log" idempotent, you can't perform the actual computations in an idempotent fashion).

You handle this by modeling the account as a ledger with immutable entries containing unique identifiers rather than a single mutable balance. The ledger entries are facts. The balance is a derived computation. Only store facts, not derived computations.

(At some point for efficiency reasons you might need to prune the transactions, but this often involves trade-offs like not allowing any new "lost" transactions older than date X.)

and how would you make that idempotent? ;)

The key is to consistently derive a primary key. With the same key, last write wins. The second create turns into an update.

Making the primary key "buy-milk" would do the job. Getting a consistent key requires a bit of UX but isn't actually that difficult.

These days you can often get away with just using a cryptographic hash function. Blake2b is really fast. Makes life unbelievably simple.

I don't think it would be very useful in a TODO list situation though.

What if I want to enter buy-milk twice ;)

Different applications could use different schemas, you just keep in mind the eventual consistency. i.e. you can use event sourcing or just treating everything as a log that gets read and merged (think of it as a CRDT) should be able to handle those kind of scenarios. That use case matches Cassandra quite well with its wide rows.

Idempotence can handle that possibility by embedding the ordinal of the purchase you wish to make:

  buy-first-milk
  buy-second-milk

But now the user is burdened with having to be explicit.

This is exactly the type of case where I'd much rather have the app be explicit about it's actions: allow both to be added, and at most show a notice after synchronisation saying "I've noticed you've added two buy-milk; merge them or leave them separate?".

I've had nothing but bad experiences with apps that thinks they know best and try to merge records behind my back.

The idea behind idempotence is that you can apply multiple times, in any order, and still come up with the same result. http://en.wikipedia.org/wiki/Idempotence

If you say, "Buy_first_Milk" and "Buy_Second_Milk" - and the order is reversed, and the "Buy_Second_Milk" is applied first, and the "Buy_first_milk" is applied twice then everything happens as desired.

Being explicit is what allows everything to work without confusion.

You don't know what the users intent is. Maybe they meant to enter "buy milk" twice, maybe they didn't.

That's the point I'm trying to get across - you know the user's intent if they are explicit. If the user enters "Buy Milk" - you don't know their intent. But, if they enter "Buy First Milk" and then "Buy Second Milk" - you know precisely what they want.

From a user interface perspective, if the user ever enters "Buy Milk" - and there is no existing milk, that is always, "Buy First Milk". If they go to a different client, and the milk order is there, and they enter "Buy Milk" - that then becomes "Buy Second Milk". But, if "Buy Milk" hasn't synced yet, they can explicitly flag it as "Second Milk Order".

Right, you are shifting the burden of merging conflicts to the user. That may be the best approach, but you are giving up on achieving consistency purely through tech.

No, the app can manage this in most cases - if you enter buy milk twice on the same browser session, it will know how to construct the keys properly. If you buy milk once in two different sessions that are not yet consistent, in both cases you would see only one in the list and so adding another in either or both sessions will yield correct results.

But you are guessing what the users intent is, but you could be wrong.

Reply on news.ycombinator.com