Is there any performance gain over writing append-only data to a file?
I mean, using a merkle tree or something like that to make sense of the underlying data.
I mean, using a merkle tree or something like that to make sense of the underlying data.
(yes it's fashionable, but it's still terrible for random read performance)
I mean, what's the trick NVMe can do to be meaningfully faster?
But seriously, I've been thinking about an append-only files + memory indexed DB for the past couple of weeks - any prior art or links or papers or anything, lay it on me.
Write speed wasn't my primary motivation though. I wanted a data storage solution that is hard to fuck up. Hard to beat append only in this regard. Plus everything is stored in merkle trees like in Git, so there is the added benefit of data integrity checks. Yes, bit rot is real, and I love to have a mechanism in place to detect and fix those.