Hacker News Reader

Comment by gchamonlive | Hacker News Reader

Are logs all you need for durable workflows? I'm confused here. How'd persist and query nested or related data over logs? By logs I assume you mean something like elasticsearch or meilisearch?

View on HN

Pretty much every durable system has an intent log of some sort. The log provides durability, the database system just integrates that log into a more queryable format.

I swear it didn't occur to me that that mean WAL, makes much more sense now LOL

I assume they meant a log like a WAL. A WAL should be (quite literally?) all you need for durable workflows.

A distributed WAL (to survive a machine death) would also probably be something I'd want, and … something I'm not sure you're getting directly from SQLite.

Is it common to use logs as a proxy for write-ahead logs?

Folks this is meant to be an honest question, not a snarky comment. I'm not a DBA, I'm DevOps/SRE and logs for me always meant execution logs. I'm just curious if between those involved in database domain logs is used to refer to WAL.

I think the original poster in this thread was joking. A fair number of databases use "logs" as a core mechanism for storing and sorting data. "Logs" in this context is not to be confused with stdout/stderr output that you may collect from a running program and forward somewhere like Cloudwatch/Elasticsearch etc. "Logs" in the context of databases here refers to the data structure; which can generally be defined as simply an append-only "file" (I put file in quotes because just because something is a log does not mean it is necessarily written to disk yet - that's why write-ahead logs exist). It's not just write-ahead logs.

Google "Log-structured merge trees" if you want an interesting read.

I read the parents comment as sarcasm and not a serious suggestion.

Log as in the structure.

Reply on news.ycombinator.com