Tikv: Distributed key value store powered by Rust | Hacker News Reader

Tikv: Distributed key value store powered by Rust

github.com

137 pointsadamnemecek44 comments

I know the authors of this project also read Hacker News, so it probably is better if we wait till them to state their motives. But I suppose I could give a brief first, since most of the confusing comments are due to language barriers(sorry if you don't know enough Chinese, you might not have read enough about this project).

This is actually affiliated to https://github.com/pingcap/tidb, which is another implementation of Google Spanner/F1, similar to CockroachDB. So think TiKV as spanner, and TiDB as F1.

They actually started out using Go. And the reason they are changing to Rust is not trying to be a Rust cool kid, but due to real constraints: Go's GC introduces too much of latency time (and no, Go 1.6 is not enough despite being better), they are also not satisfied with cgo's costs. That's why they are changing to Rust. Note that according to their words, they still love Go better, and TiDB is also happily using Go. But for TiKV's case, Rust is perfect.

Disclaimer: I'm not working on TiKV or TiDB, I'm just a fan of their work(they are also the founders behind Codis: https://github.com/CodisLabs/codis, tho not working on that anymore). I'm happy to list references on where I got all the information in case anyone are interested but they are all in Chinese, which I think might not be that polite since not everyone here reads Chinese.

This looks like it could be a drop in replacement for FoundationDB but I don't see enough documentation on the guarantees they give or limitations. Any more information? Maybe even translated from Chinese?

Unfortunately I don't see much material written in English at the moment.

To your technical question, I'm not that familiar with FoundationDB to comment, but I can tell the author's intent is building an open source solution for Spanner/F1. So I'd say TiKV/TiDB is really a drop-in replacement for Spanner/F1, not FoundationDB.

So in that case, I think the guarantees they want to meet include at least distributed transactions on a KV store(Spanner), and zero downtime SQL schema changes(F1).

FoundationDB was a drop-in replacement for spanner as well. Can't wait to see more docs.

I'm super excited to see a Mandarin speaking team building something in Spanner class of databases.

Curious why the language that the team speaks matters to you at all?

I think everyone could benefit from extracting that raft implementation to a crate. I'll see if I can get started on that.

I think hoverbear has a raft crate: https://github.com/Hoverbear/raft-rs

Yes, I was just wondering why they didn't use that? I reviewed that raft implementation. Looks decent and uses mio for non-blocking io, which was quite clean (I'm jealous).

Most raft implementation don't implement dynamic reconfiguration or snapshots/compaction. AFAICT hoverbear implements neither, and Tikv implements at least snapshots.

You're right. And thanks for the pointer on this implementing snapshots.

Hi! Thanks for linking this! We're hoping to add more robust features like dynamic membership soon! I'm graduating and will have more time.

If you feel like hacking on our code please just let us know, we'd love to help.

tikv's raft is a port of etcd/raft, which is used by a few projects in production. So we can expect rust raft to be mature soon.

Distributed KV store based on rocksdb. Isn't that just reinventing the wheel? Is the fact that it is written in Rust that important?

It looks like a parallel implementation of TiDB, which they had built in Go. I would assume that avoiding garbage collection and reducing heap usage were valid motivations for the project. But it doesn't really have enough information on the repo to tell what their motivations are.

Not quite. TiDB can use it as a data layer, as an alternative to Rocks, Bolt, etc. Different part of the system

So when the Readme says "Distributed KV Database", I'm imagining that it refers maybe to being able to be used by TiDB which provides the distributed logic, while TiKV concentrates on a compatible interface to TiDB and local non-distributed storage?

Right. It should be roughly equivalent to InnoDB/MyISAM alternatives in Mysql (I believe)?

It's transactional and mvcc, you can't say the same for most KVS around.

CurioDB too: http://curiodb.jupo.org

To be fair, I was considering learning Rust and implementing a distributed hash table just as an exercise. With a like-minded team, reimplementing a toy Spanner sounds like a fun project.

Language is one of the more important features for attracting other contributors to open source projects.

As to reinventing the wheel, given the number of competing tire companies that exist apparently even the wheel still has room to improve.

That's a silly and un-applicable example. How many tire startups are there?

At least one: http://www.timesofisrael.com/with-innovative-tire-israeli-st...

This is offtopic but as I do not have the time to investigate what this website tries to do, I must alert that it completely froze my system, twice. Had to actually do a physical reset.

I could not find a description of the algorithm used for distributed transaction. I can only assume it's similar to CockroachDB but looking at the code I see a lot of reference to "LOCK" in the protocol buffer definition which mean they might be using something else.

Anybody know where I can find more documentation ?

Says they are using Raft: https://raft.github.io/

Raft is used to Replicate write in a shard but distributed transaction is used to make transaction that span several shard atomic (ACID). I could find the implementation at : https://github.com/pingcap/tikv/blob/master/src/storage/mvcc... But a lot of the detail are not clear.

Raft is a replication protocol which is responsible for the consistency of multiple copies of the same data but it isn't applicable when the data is sharded.

You can use raft as to synchronizing other data structures. If this a KV store then that's as easy as key level locks / ownership / placement.

The transaction model is similar to Google's Percolator, and with some performance improvements.

Thanks after seeing its based on themis code that is itself based on Percolator the distributed transaction logic make sense. Any idea how it compare to CockroachDB ?

In CockroachDB using SSI isolation if a Writer encounters a more recently read key the writer transaction restart. This guarantee that Write Skew cannot happen.

In percolator design only allow for SI isolation unless lock are taken while reading.

From the README:

> Thanks rust-clippy. We do love the great project.

This just made my day. :-)

I appreciate the sentiment, but this project is very difficult to understand what's connected to what. That's not a good sign for a database

The first PR was merged ~3 months ago, I'm not surprised that they don't have more info in the README.

Can someone ELI5 what a distributed key value store is useful for? Is it like a DHT for torrents?

With "inspired by" meaning the very same thing as in the opening credits of a Hollywood movie...

I'm pretty much sure this software wouldn't get a lot of attention if it was not built in rust. Darn I need to build something in rust and get some attention!

Sure, trends come in trends—in programming as in other things. But please don't post comments that put down other people's work.

If you think other stories should get attention on HN, please submit some.

It's a joke man. Why would I put down someone I don't even know!

> Why would I put down someone I don't even know

I share your question, yet it seems to be what most internet forums are for.

You're always welcome to build an opensource transactional bigtable clone!

And do it in INTERCAL or Befunge to boot!

Reply on news.ycombinator.com