Storing the checkpoints along with the mutation was for idempotency. If the checkpoint was in the document, that meant the mutation had succeeded and a retry should be no-op
Doesn't the KV provide idempotency on it's own -- so long as you're checking that no changes have happened between read and write, why wouldn't doing the same write twice produce an idempotent result? A change happenening between read and write seems the only reason that would be a problem.
But clearly it's complicated and we don't have the whole picture as to business needs. Definitely sounds awful.
you can imagine this:
```
var thing = KVStore.Get<MyThing>(...);
if (things.Checkpoints.Contains(myUuid) == false) {
thing.Counter += 1;
KVStore.Update(thing); ```
having an etag doesn't help you with retries, where we expect that `thing` could be mutated by another flow between your retries.
i may be missing something though?
when you retry and load the object, you get {count: 12, etag: 3}. how do you know if your previous attempt had successfully persisted or not, or if the updates to the object came from other processes/requests?
you're mixing up conflict handling vs. idempotency
Anyway, okay, I get that I don't get it, and I get that it does sound terrible, agreed!
Imagine this: you issued a write, a few things can happen: 1. The callsite crashed, maybe due to an out of memory issue or whatever. You don't know if it succeeded
2. The database returned an acknowledgement, but then the callsite crashed before storing the acknowlwedgement for the next step.
> Unless the entire end to end operation is wrapped inside a giant transaction, no system in the world can give you the confirmation.
But how do people use, say, redis, without an external "checkpoint" system, how do you do, say, an INCR operation, and know if it succeeded or not?
Or are most uses of redis actually dangerous and subject to these error conditions, perhaps it's tolerable to risk that error for most redis use cases, but wasn't for the system under discussion? Most developers using redis definitely aren't using external "checkpointing" systems, or considering if they should be or not -- should they be?
But of course you've convinced me that the system under discussion would have been better off using an rdbms, something I never doubted.
Where our conversation is at, it's not about RDBMS vs. NoSQL.
On the Redis INCR operation example (or an equivalent operation in any database, even in an RDBMS). In short, you don't always have the guarantee to know the result of the operation. This is part of what the 99.9% means when services advertise their reliability. 99.9% of the time you'll get an acknowledgement back, but not 100%.
The question comes to, what happens if I miss an INCR, or if I accidentally double INCR when I retry? For some use cases, it's acceptable: like counting the number of views on a Youtube video. For others, say a financial system counting money or making a payment, that's not OK. That's where idempotency really matters (btw this is why many redis operations have the NX option).
Let's go completely off track for a little bit, say you're Strava. You let devices and users upload workouts to your service. What happens when a phone sent you the workout, then disconnected due to bad receptions? In this case, the phone doesn't know if the operation succeeded so it's forced to retry. How do we make sure the workout isn't created and counted twice on Strava? Well, each workout has a unique UUID generated by the device. Strava uses this UUID to dedup when it receives an upload. The UUID is the idempotency "checkpoint".
In this case, it doesn't matter what kind of database Strava is using. The idempotency problem is the same.