Litestream Eliminated My Database Server for $0.03/Month
mtlynch.io
mtlynch.io
It’s cool, don’t get me wrong, but given the above constraint isn’t it just:
while True:
sleep(1)
upload_file(“sqlite.db”)
Litestream is doing more magic than it seems, so it's much better than a naive while loop.
Litestream is only streaming the changes in the database, whereas the naive implementation would have to upload the entire database every second.
Also, SQLite has a write-ahead log, so you're not guaranteed that all the data is always in the main .db file. Litestream watches the write-ahead log files as well and immediately streams those changes to cloud storage. A system that only watches the main .db file would have long time windows where data loss is possible.
I discovered Litestream here on HN after benbjohnson shared it here a few months back.[0] I wasn't interested at first because I don't ever use SQLite, but it stuck with me. I eventually realized it's a great way to achieve vendor flexibility in hosting apps, so I used Litestream to build a pastebin-style service for my app and wanted to share what I learned.
Happy to answer any questions or hear any feedback about this post.
In terms of latency alone it sounds like it risks adding tens or even hundreds of milliseconds to even a fairly trivial request: out of my datacenter, across the wire, into theirs, grab the data, and then reverse the path. And perhaps a charge for data going out each way.
Am I wrong about that? Am I overestimating the travel time, or perhaps they end up being located in the same datacenter so it's all high speed?
The OP's approach of SQLite & Litestream removes that latency entirely since the database is colocated with the application. Litestream will stream backups to an external service (e.g. S3) and that can have tens or hundreds of milliseconds of latency but that won't affect the application's performance since it is asynchronous replication.
what is the performance like for a database server that's separated from your host like that?
Yep, you do have to download the DB on every deploy. It's negligible for my deploys because my database is a few MB, but if it got to be in the GB range, I'd have to find some way around that.
isn't this wrong? The S3 may not be the remote DB, but it is sorta is, in this case.
The major thing is the app doesn't have to be coded with s3's latency in mind when it accesses the db.
Would a multi tenant SaaS with databases on the much smaller size compared to what you all talk about do well to use this, with proper backups?
That interval determines your window for data loss in the event of a catastrophic failure (e.g. disk failure). I’ve run multiple DigitalOcean machines for years and have never had a catastrophic failure so they are rare events but they can happen.
Dqlite and rqlite run every write through a distributed consensus algorithm which guarantees that your writes are persisted across at least a majority of nodes in your cluster before returning a success. In that situation, you’d need a catastrophic failure on a majority of nodes to lose data, however, there is generally a large performance trade off when running distributed consensus.
p.s. Thanks for litestream!
Yes, there is no reason why that wouldn't work. rqlite supports on-disk mode so you could run litestream alongside an rqlite node and backup the underlying SQLite database to your favourite cloud provider using litestream.
The only downside is that rqlite performance is very sensitive to the number of writes to disk, and that's why rqlite uses an in-memory SQLite database by default, and lets the Raft log persist to disk. In exchange you can be sure that your writes are persisted to disk when the API acks your request. Perhaps if rqlite used a RAM-based filesystem for everything, and you combined it with litestream you could get much higher-performance from rqlite, with backup (with only a tiny window for data loss) if you use litestream. It's not entirely trivial, however, due the difference in data consistency models. For example, which SQLite database should be updated if the Leader in the cluster changes? It gets complicated.
Ben has written a great program here. I wish I had his ideas! It's got me thinking. :-)