Hacker News Reader

Comment by adamretter | Hacker News Reader

Have you considered Meta's RocksDB as an option?

I wrote a time-traveling database (where you can query a table/row as of a specific point in time and join it to data at another point in time; we used this for AI training to predict future behavior in users) completely from scratch (that was the coolest work project ever, btw) that was built on Hadoop/Hbase. I understand RocksDB is fairly similar ... however, I want to stay as far away from any of those kinds of APIs. I have scars from dealing with hbase and writing query planners and figuring out how to do performant joins in a white-room type environment. No. Thank. You.

It was fun at the time, but I don't want to go near it ever again.

[RocksDB](https://rocksdb.org/) isn’t a distributed storage system, fwiw. It’s an embedded KV engine similar to LevelDB, LMDB, or really sqlite (though that’s full SQL, not just KV)

Yes, it's based on the same paper as hbase, IIRC.

To be perhaps overly detailed: Hbase is an open source approximation of bigtable. Bigtable _uses_ leveldb as its per-shard local storage mechanism; Rocks is a clone+extension of leveldb.

Bigtable and hbase are higher level and provide functionality across shards and machines. Level and rocks are building blocks that provide a log-structured merge tree storage and retrieval mechanism.

> Bigtable _uses_ leveldb as its per-shard local storage mechanism

Ah, that's probably what I'm conflating with it then.

Thanks for the information.

Reply on news.ycombinator.com