Hacker News Reader

Comment by mrkeen | Hacker News Reader

Using one-log-only for an entire system does have its upsides, but it will kill performance. It would be like building a CRUD system with a single mutex for everyone to share.

View on HN

I believe you want "one log" in the logical sense. In theory, you could have "one log" per user or group of users, or whatever sharding technique makes sense for multi tenancy model. It can also be "one log" per bounded context - e.g. the entire payment pipeline in one log.

Yes, exactly right. One log per logical entity, here "payment ID".

The way our open source project implements that is with a partitioned log and indexes at key-granularity, so it is like virtually a log per key.

How do you deal with one side of transaction getting lost? Traditional double entry accounting puts entry for each side of the transaction. It will cause problems if one side is lost or delayed.

How do you handle transactions in general? It is pretty common to have multiple changes that need to be applied together. In a single log, can write the changes and then write a record that says the changes are committed. This is how databases work.

You can simplify things significantly if you can maintain a 1:1 mapping of log entry to transaction (in the business sense). This way you advance the state of your system in discrete steps by advancing in the log, and can make rewind easier too.

That's what the rest of us are doing in event-sourcing land, but TFA is arguing for something much stronger:

  If everything’s in one log, there’s nothing to coordinate

The rest of us have to coordinate the logical log of user creation/deletion and the logical log of user payments, etc.

Separate logical logs with no need for coordination can only work if they are truly independent systems - no causality between them.

Reply on news.ycombinator.com