Why it's time to retire CRUD

xtdb.com

9 points·TheFreim··4 comments

Hmm. Accounting solved this issue several hundred years ago. I wish vendors would catch up to what monks did in the Middle Ages.

First, entries are made in an append only journal. When a new month begins, you archived the old one and start a new one.

Second, you can have several journals. For example one for each subsidiary.

Third, you collect each day’s worth of entries into a consolidated journal.

Fourth, from the consolidated journal, you make adjustments to your running totals/running balance. However, this ledger is not the source of truth. If there is any dispute, you go back to the journals.

What this means is the source of truth is immutable append only (transaction journal). For speedy retrievals of data, you cache the running totals (data ledger) by upserts.

To make any corrections, you create a new journal entry that negates the bad one.

The problem is that if you only have a raw SQL database, adding these layers is a lot of work. And there’s almost zero open-source concerns about that.

Apart from that, yep, appending a change/fix/action-describing document with a set of triggers/modules which post its essence into pre-designed caches is how you do it.

mariadb, mysql, pgsql, they all works like that: it's called "bin logs", "wal" or whatever. A record of everythings. Takes a lot of spaces.

When you have a TB-ish non-archival database, keeping those is a nogo. This is why they are thrashed, and custom logic is implemented for the data that actually needs auditing capabilities.

Databases are tools, you still need to work to tune them to your needs.

CRUD doesn't mean CRUD all the things. If your use case requires it, you can build a create-only environment where you never delete anything.

Reply on news.ycombinator.com