Hacker News Reader

Comment by LAC-Tech | Hacker News Reader

Which one have you used, the kv store or log?

Reason I submitted this because I'm curious about peoples real world experience.

I was looking to use both. Actually. I discovered FASTER when looking to port durable functions to php (it’s called durable-php if you want to google it, though the implementation is nothing like it) and the netherite engine uses faster.

It’s perfect for my use-case, more so now than when I originally researched it as a possibility. Back then, I didn’t even have threading solved for php. Now that’s all a solved problem (threads ftw) and I’m refactoring log storage now to better support things like faster.

Have you considered Meta's RocksDB as an option?

I wrote a time-traveling database (where you can query a table/row as of a specific point in time and join it to data at another point in time; we used this for AI training to predict future behavior in users) completely from scratch (that was the coolest work project ever, btw) that was built on Hadoop/Hbase. I understand RocksDB is fairly similar ... however, I want to stay as far away from any of those kinds of APIs. I have scars from dealing with hbase and writing query planners and figuring out how to do performant joins in a white-room type environment. No. Thank. You.

It was fun at the time, but I don't want to go near it ever again.

[RocksDB](https://rocksdb.org/) isn’t a distributed storage system, fwiw. It’s an embedded KV engine similar to LevelDB, LMDB, or really sqlite (though that’s full SQL, not just KV)

Yes, it's based on the same paper as hbase, IIRC.

To be perhaps overly detailed: Hbase is an open source approximation of bigtable. Bigtable _uses_ leveldb as its per-shard local storage mechanism; Rocks is a clone+extension of leveldb.

Bigtable and hbase are higher level and provide functionality across shards and machines. Level and rocks are building blocks that provide a log-structured merge tree storage and retrieval mechanism.

> Bigtable _uses_ leveldb as its per-shard local storage mechanism

Ah, that's probably what I'm conflating with it then.

Thanks for the information.

Reply on news.ycombinator.com