Hacker News Reader

Comment by k__ | Hacker News Reader

I had the impression idempotency was the way to go in distributed systems and not differences.

Those two concepts are not mutually exclusive. Each change can be idempotent by marking some identifying information about the state it was trying to update.

    increase balance by 10 (it is currently 100)
    balance is now 110
    increase balance by 10 (it is currently 100) <- duplicate event!
    event has already been applied
    balance is now 110

It can definitely break down at some point (maybe the user really did want to increase by 10 two times, they just clicked twice really fast?). But idempotency and change-based flow can live together.

This is where CQRS comes in. Every unique action has its own command id, so if a user clicks submit twice, the same command id is being sent twice and after the first success all future ones don't affect anymore change, they just return automatic success since that command id succeeded in the past.

Combine this with versioning, where you assert an event stream version on write, and now you're able to affect change on a known state that cannot have changed (and if it did change, your command would fail and cause no persisted changes server-side).

I'm well aware that there's no absolute standards for this, but I've built a number of CQRS systems, and I didn't ever have idempotency IDs as a core tenant of the system.

What literature, or framework or ecosystem holds this to be a hard-and-fast requirement of CQRS?

Personally I've solved this problem similar to Git, modelling CQRS on top of Merkle trees so your system at least knows if your incoming edit (command|commit) is being applied on HEAD (no changes to the model since the command started) or to HEAD+n (needing to re-run some command validations, and "rebase" the incoming command).

I don't think there is any "standard" that strictly governs the implementation of a CQRS system. We just need to be aware of the nuances of the fault-tolerance aspect in distributed systems:

- idempotency is ONLY a hard-requirement if the events are transmitted via lossy channels such as network. ID is one out of several ways to ensure idempotency. If you're implementing CQRS for your distributed system then it's needed - otherwise, if the events aren't lossy and can be processed atomically, you don't need to ensure idempotency. think implementing CQRS for your user interface where events are just transmitted between different objects in memory rather than via network

That's generating an idempotency ID on the fly, isn't it?

That means commands have to be stored as well (at least for some time) in a non-eventual-consistent way?

If a command is successfully converted/persisted into an event, that event should contain the command's ID. When the command is replayed, the domain aggregate/event stream will contain that command ID, and will therefore reject/ignore the replayed command.

So just to get it right: 1. it means that there is only one instance that processes commands and 2. this instance checks all command ids if they appeared before (so keeps track of all ids in an efficient way)?

> it means that there is only one instance that processes commands

Not necessarily. Depends on how you shard the aggregate/streams. Since aggregates _by definition_ maintain their invariants, you can have literally a shard for every aggregate. Overkill, but it illustrates the point. Since there's no state shared between aggregates, you can process each aggregate/stream on its own instance.

> this instance checks all command ids if they appeared before (so keeps track of all ids in an efficient way)

Yep. If you trust the client and don't mind losing old commands, this isn't too hard - use a monotonically increasing number (e.g. ms since the epoch) as part of the commandId. Any command with an id less than the commandId in the aggregate will be rejected as "out of date". Note that this serves as an optimistic concurrency check.

If you don't trust the client (or you don't want to lose old commands issued on a possibly out of date view) and just use GUIDs as commandIds, then yes you'll need to keep track of some ids - but not necessarily all. Do you really need commandIds past 1000? (Depends on your domain.) You'll need to "roll up" the events into some summary view anyway to check/maintain business invariants, so keeping, say, the last 100 commandIds as part of that summary view is simple.

Thank you, that makes sense and a lot more clear to me.

Now I only wish there would be a good alternatives for when you want to have multiple instances per aggregate (even if you may have to give up certain guarantees).

Interesting... just to be clear, an _individual stream of user events_ is an aggregate. E.g. a user Bob Smith's events are independent of another user like Jane Doe. It isn't the case that the entire "User Table" is a single aggregate - each row in it is an aggregate.

If you're trying to do multimaster db storage, like a phone app syncing with some cloud server, where an aggregate resides in two separate locations, this is where event sourcing shines. It's just like git, right, and so you can do all the stuff that git can do - merge event streams, rebase events... it's actually the reason why my pet project switched from an RDBMS to event sourcing. Related video: https://skillsmatter.com/skillscasts/1980-cqrs-not-just-for-...

You can run multiple instances, and use something like distributed locks to avoid contention causing failures.

Exactly. When you are affecting a change on a stream, it has to read the events anyways, so it will be able to tell if the command id is already associated with the stream since each event has the command id in its metadata.

> It can definitely break down at some point (maybe the user really did want to increase by 10 two times, they just clicked twice really fast?)

Well, that is exactly what OP meant - in the context of a distributed system it is not "at some point" it is just a given.

What if this is concurrent with "decrease balance by 10"? If the decrease occurs between the two increases, then the increases won't be detected as duplicates.

It’s my understanding that these systems don’t use the value, they’ll instead reference some kind of “state id” that refers to the state of the system as the result is some previous, accepted, change. This way, even if a decrease races the duplicate increase, the duplicate is still correctly detected.

The two are not necessarily mutually exclusive.

    Balance is 100
    Operation #1 says add 10
    Balance is 110
    Operation #1 says add 10
    Balance is still 110

If you can create unique IDs for your operations, idempotence is really easy to implement.

Reply on news.ycombinator.com