Hacker News Reader

Comment by NicoJuicy | Hacker News Reader

I think the author is missing an overall picture, eg. Event driven scenario's.

Where you don't have to check collisions with a db. He mentioned generating the pk's on remote client, but that doesn't capture the interesting bits.

You generate the newly created object with the guid.

You send it to the API/Microservices and it's generated, fire-and-forget style. And the remote client has an Id of the newly created object to do something with.

View on HN

But now the remote client has an ID of something that may or may not exist the next time they try to use it depending on whether or not it actually made its way into the database.

I've seen this kind of architecture before. It sounds nice but is loaded with consistency problems.

That's why the pattern is eventual consistency.

You receive a message "{entity}Created" and it contains the Id of the full object.

It's like, _maybe_ eventual consistency. Hopefully the client doesn't try to do anything important with the ID/new object

If you get a "Created" back, it is possible for it to be guaranteed that it'll be created eventually.

I don't think you would ever get a 201 Created in an eventual consistency scenario. You would get a 202 Accepted. Unless the API is lying to the clients.

Fair enough, though what I said remains true for an "Accepted" response.

But Accepted doesn't mean created.

The feedback flow is async from the creation flow.

If you get created through the feedback flow it's created.

Accepted can mean that it will eventually be created though. In the sense that it's validated and guaranteed to be created at a later point ("eventually").

Created can mean different things to different people. Maybe an ecommerce order is created in the sense it's been updated in the inventory database and a record added to some sort of fulfilment system but there's still outstanding data warehouse sync jobs, some order analysis/product recommender job, and some anti fraud jobs (that just need to complete before the order picking in the warehouse begins)

What if the object is deleted again? You could still have an inconsistent picture, no? And you could create and delete over and over again, so that a client has an inconsistent picture all the time.

That's a different design issue then - it means lack of synchronization. It is orthogonal to using uuids at a client. In fact, (properly) using uuids are a reasonable way to resolve this.

Not arguing against using UUID. Just exploring that async concept further and the challenges in that. OK 2 separate issues : )

That's what eventual consistency is, isn't it? It's definitely not the easiest thing to design systems around, but at a big enough scale you sometimes don't have a choice.

There's also many ways to "work around it", so that it doesn't seem inconsistent to the user.

I'm definitely not arguing that it's how most systems should be designed though.

If the client needs to use the object right there and then, it can start polling the GET for that ID until it returns 200.

A slightly more sophisticated pattern is to have the initial creation command return an ID of the creation event. Then the client can poll that ID and just check that the event has been processed successfully. This requires the client to trust the server a little, but it means it can just poll a small, temporary, probably in-memory "recent events" table instead of hammering the main API.

A more efficient design is to YOLO, submit the second command while the first is still being processed, and trust the server to handle the commands in the right order. This is fine for backend services, unfortunately human users get annoyed at their frontend when it says "sorry you have to do the purchase finalization again because an earlier 'add to cart' command failed and I only noticed it now".

Well, you would have some kind of synchronisation protocol where the database confirms it has received that record and it now exists.

There's patterns for implementing it. It's not really any different than using more than 1 database which is unavoidable in many scenarios (interacting with 3rd parties)

https://chrisrichardson.net/post/sagas/2019/08/04/developing...

Reply on news.ycombinator.com