TidesDB: A fast key/value store for modern hardware

Hey hackernews! I'd like to share a project I've been working on for almost a year now. It's an open-source storage engine similar to that of LevelDB/RocksDB but written entirely in C.

The storage engine is called TidesDB.

TidesDB is an open-source storage engine similar to LevelDB/RocksDB but written entirely in C. It's designed as a fast, transactional key-value storage engine built on a log-structured merge-tree (LSM-tree) architecture.

My journey with TidesDB began nearly 2 years ago while I was experimenting with various data structures and databases in Go. When I encountered the log-structured merge tree concept, I was initially overwhelmed by its complexity after reviewing other implementations.

However, after studying the original paper, I realized I could potentially simplify the design by focusing on just a 2-level approach(memory level and a disk level). This was challenging at first, and I scrapped many ideas along the way, but over time I discovered how powerful this design could potentially be.

The beauty of this architecture is its extensibility. Since the disk level contains many SSTables (Sorted String Tables), we can efficiently pair and merge them in various ways - whether in parallel for speed or incrementally to minimize resource impact.

What began as a challenging learning process has I believed evolved into a unique engine design and library.

You can check out TidesDB code here: https://github.com/tidesdb/tidesdb

Website here: https://tidesdb.com

Currently TidesDB is nearing its first major release, we are still in beta development :)

I'd love to hear your thoughts on the library. I am very open to discussions, we have weekly code dives and reviews and yeah. Looking forward :)

tried tidesdb with python and it's really fast. a transactional key-value database written in c with a modern lsm-tree architecture. worth checking out