Example: https://delta.io/blog/2022-05-18-multi-cluster-writes-to-del...

Assuming you want to write to dest_path with put-if-absent semantics, here's a sketch:

- Write to a unique temporary path (call this temp_path)

- Commit a record to DynamoDB with put-if-absent semantics on the key (dest_path) and the value (temp_path, "incomplete")

- Three possible outcomes:

1. Your write succeeded; proceed to issue a CopyObject call from temp_path to dest_path and if successful mark as complete in DynamoDB.

2. The row already existed in DynamoDB as "complete" -> do nothing

3. The row already existed in DynamoDB as "incomplete" -> another write has been committed, but is not complete; attempt to repair it by issuing a CopyObject for the path in DynamoDB. On success, mark as complete in DynamoDB. ("repair" step)

Reads could also hit DynamoDB before S3 in order to perform the "repair" step if applicable.