Sanakirja: Transactional, branchable key-value store in Rust | Hacker News Reader

Sanakirja: Transactional, branchable key-value store in Rust

pijul.org

127 pointsKlasiaster15 comments

Don't know much about this project but interesting to see the name they chosen. "Sanakirja" means dictionary in Finnish, literally "a word book". Finns often name tech something English sounding, since they are worried if others will find it too strange sounding.

Author here! I did start Sanakirja in Finland, but I'm not a Finn, so I thought I was allowed to break that rule.

Also, getting traction for Sanakirja has never really been a goal, at least not as much as getting something that we could use in Pijul.

Sana kirja is word book but sanakirja is dictionary.

This seems to be a component of a version control system with a very interesting data model. Check it out at the main page https://pijul.org if you hadn't heard about it like me.

To make these transactional too, they are actually stored in a different B tree. Cycles are avoided by not storing reference counts less than or equal to 1.

How does that work? If you don't store a refcount when it equals 1, how do you know not to delete the thing? And if you never delete anything, why keep refcounts?

With a strong ownership model like Rust’s, the type system can essentially be used to handle the case of a ref count of zero or one.

(I haven’t actually looked at the Sanakirja code.)

Author here. So I thought when I started the project. But when you think more about it, Rust cannot allocate in an mmapped file, so you're basically on your own, and have to do plain old C-like memory management in the file, except that unlike in C, there is the extra constraint that everything is transactional, pulling the machine's plug needs to "unfree" all freed pages, and "unalloc" all allocated pages.

Good question. The structure is a B tree. When you traverse it, and arrive to a block, but don't see it in the reference counting base, you know that it is referenced in the tree, so its reference count is at least 1.

Then, if you want to free it, you have to delete all references to it in the tree. Maybe that blog post could have been more verbose about another feature of Sanakirja, the list of free pages.

The more I see from this project, the more I like it. The Pijul team seems just completely serious about building system software right. Also while I could wish for more doc, what's there is a really good overview.

Author here! Thanks! Documentation is quite high on our priority list, actually just after fixing a few security issues on nest.pijul.com.

I'm curious if this software could benefit from what was discussed here: [1].

[1] https://news.ycombinator.com/item?id=13890011

What's the comparison to TiKV(https://github.com/pingcap/tikv) ?

1. Sanakirja is fully transactional.

2. Sanakirja supports fork.

3. TiKV is distributed.

4. Sanakirja was started before (the Pijul project needed something like that, and TiKV was not around).

Pijul is also described as distributed, although that's not the same as Sanakirja itself.

It would be useful to be able to browse the sourcecode online, without actually cloning it.

It seems somewhat weird to use the VCS itself for the VCS being developed. Well, maybe that's just me. Until you offer the ability to browse sourcecode online, maybe mirror the repository to some online source hosting site and disallow issues tracking there?

Reply on news.ycombinator.com