5,065 karma · joined April 3, 2014
https://www.philipotoole.com
[1] https://www.rqlite.io
The biggest performance improvement since is due to the introduction of Queued Writes. See https://rqlite.io/docs/api/queued-writes/
That's correct, there is a write-performance hit for the reasons you say. All Raft systems will take the same hit, and SQLite is intrinsically single-writer -- nothing about rqlite changes that[2]. That said, there are approaches to increasing write-performance substantially. See [1] for much more information.
Write-performance is not the only thing to consider though (assuming one has sufficient performance in that dimension). Ease of deployment and operation are also important, and that's an area in which rqlite excels[3] (at least I think so, but I'm biased).
[1] https://rqlite.io/docs/guides/performance/
[2] https://rqlite.io/docs/faq/#rqlite-is-distributed-does-that-...
[3] https://rqlite.io/docs/faq/#why-would-i-use-this-versus-some...
One notable production user is Replicated: https://www.philipotoole.com/replicated-postgres-to-rqlite/
https://www.philipotoole.com/testing-rqlite-read-consistency...
The report itself: https://github.com/wildarch/jepsen.rqlite/blob/main/doc/blog...
It's written in Go, so it'll help if you are familiar with Go. But the code is not difficult to understand even if you don't.
Before we jump into what is encrypted, and what is not, what are the threat vectors we are actually concerned about?
With rqlite all traffic between the nodes can be encrypted, as can be the HTTP API. Yes, the data at rest is not, but there are many ways to protect that[1]. One way would be to use an encrypted file system, perhaps.
Encrypted data-at-rest is a valid request, but what use cases would it address?
Over time rqlite has became a much more substantial database than it originally was, with a focus on ease-of-deployment and ease-of-operation. I have written a fair amount about how the design and implementation has evolved over the years[2], which may be of interest.
[1] https://www.philipotoole.com/replicating-sqlite-using-raft-c...
However, running it on Kubernetes should work quite well. Check out the documentation at https://rqlite.io/docs/guides/kubernetes/
>If a node dies due to, say, hardware failure, when it rejoins is it seen as a slightly out-of-date existing node since it’s part of statefulset? If so, assuming quorum is maintained, this is probably a lot easier than I was making it out to be.
Assuming I follow you, yes, this should work fine -- and exactly as you expect. I suggest you try it out, and if you don't understand what you're seeing file a ticket on GitHub, or join the rqlite Slack[1].
rqlite, in contrast, adds reliability and high-availability via clustering. This means that any application talking to rqlite shouldn’t notice if a node fails because other nodes in the cluster automatically take over. But rqlite is not a drop-in replacement for SQLite.
https://rqlite.io/docs/faq/#how-is-it-different-than-litestr...
rqlite[1] creator, happy to answer questions too.
https://rqlite.io/docs/faq/#is-it-a-drop-in-replacement-for-...
"rqlite supports adding read-only nodes. You can use this feature to add read scalability to the cluster if you need a high volume of reads, or want to distribute copies of the data nearer to clients – but don’t want those nodes counted towards the quorum. These types of nodes are also known as non-voting nodes."
[1] https://rqlite.io/docs/faq/#can-i-use-rqlite-to-replicate-my...
[2] https://www.philipotoole.com/9-years-of-open-source-database...
Simple as that. Of course, sometimes I got the "message too big" error. So I pasted bits of the sources files, choosing pieces I thought were reasonably self-contained. I also fed much of my unit tests through it, asking "see any missing test cases?" While some of the answers were not that helpful, digesting the feedback from GPT-4 made me think more about my code, and make some changes for the better.
It's saving me time (sometimes 2x speed up on certain, well-specified, tasks), and I enjoy using it. I wrote a blog post with some details on how it has helped me code the database: https://www.philipotoole.com/what-did-gpt-4-find-wrong-with-...
That said, the most recent release of GPT-4 seems a little more buggy[2].
https://www.philipotoole.com/9-years-of-open-source-database...
https://github.com/rqlite/rqlite/blob/master/auto/backup/sum*
It took a few minutes, might have taken me 1+ hour of some research, and mechanically typing it all in.