Hacker News Reader

Comment by yosamino | Hacker News Reader

I'm having trouble understanding the term "log structured". Wikipedia has an entry for it, but that seems to describe a very specific mathematical theory that flies right above my head. Is it related to that or does it mean

    "This database shares characteristics with a log, in the sense of an append-mostly data structure"

And if I squint at, it's something like a "transaction log" or an "event sourcing" just at a different abstraction level ?

View on HN

Yes, "log structured" basically means "append only (mostly)"

Traditionally on-disk data-structures are modified in-place. But random writes keep disks busy (especially spinning disks, though sequentially writes are still a bit faster in solid state drives), and are also very hard to do transactionally.

So "log structured" data-structures try to make do with only appends (and usually background "compaction"). This also has the benefits of automatically providing history (global versioned snapshots!) and an audit trail "for free", if you want that.

"Log structured merge trees" are a now popular data structure that implements a key-value map using several sorted, append only sequences, instead of the traditional hashtable or b-trees which require in-place modifications

https://www.cs.umb.edu/~poneil/lsmtree.pdf

I found these: https://github.com/barrucadu/logdb http://vldb.org/pvldb/vol5/p1004_hoangtamvo_vldb2012.pdf

Basically, what you said. Append-only, and in this case apparently just S-expressions.

Reply on news.ycombinator.com