https://github.com/danielealbano/cachegrand :)
I am not going to get into what's better and what's not, especially because I haven't released the v0.1 yet and therefore it's not usable, but I am working on cachegrand which is aims to be (also) a redis compatible platform.
I have done A LOT of research and development before picking up the current architecture (you can see it from the amounts of commits) and I am trying to test as much as possible (Almost 1000 unit tests so far, but there is still plenty to do).
if you look at the repository please bare in mind that:
- there is no v0.1, the code available in the repo only supports the basic GET, SET and DELETE (apart from a few additional commands like HELLO, QUIT, PING)
- the code in main currently supports only storing the data on the disk, which is also why the tests are failing, I am doing some general refactoring and need to bring back the in-memory storage (issue n. 88)
- there are some general performance metrics available on the repo
- don't enable verbose logging, it's currently synchronous :)
- cachegrand is able to fully run in single thread mode so I can actually compare it to redis (well when it will make sense)
- only linux, requires a kernel 5.8 at least (e.g. it's provided by ubuntu 20.04.2 lts, but I didn't really care too much as it will take quite a bit more before I get the first stable version and by that point the kernel requirement will not be an issue anymore)
What I can say is that the project really focus ONLY on performances, therefore is not as memory saavy as redis or similar platforms, and it actually aims more to compete with Redis Enterprise long term than just Redis, on the other end it implements a number of things from the ground to boost massively the performances:
- cachegrand architecture follows almost the share nothing principle with the only exception of the hashtable because it has been built around that need
- I implemented from ground an hashtable capable to deliver lock-free and wait-free GET operations and which uses localized spinlocks for the SET and GET operations, basically the contention is spread across the hashtable instead of being bound to X queues
- the hashtable also support SIMD operations (AVX, AVX2 and AVX512F), it's heavily optmized to reduce the memory accesses is able to embed short strings in the bucket to further reduce memory accesses
- cachegrand will support both memory and ad-hoc backend for the storage that is going to be basically a time-series database (cachegrand is not bound to redis functionalities, the redis command set is just a way to expose these for now)
- I implemented from scratch a fiber library able to do a context switch in just 7ns
- the network and storage backend are modular, currently it really only support io_uring but the goal is to also add XDP+the FreeBSD network stack support for the network (e.g. similar to what has been done with F-Stack and DPDK) and then io_uring with the NVME passthrough for the storage (not sure if I will also add support for SPDK)
- I have also implement an ad hoc memory allocator which waste some memory but it's able to do memory allocations and free in O(1) (here a nice chart https://www.linkedin.com/posts/danielesalvatorealbano_dublin...)
- most of the code is built aiming to be zero-copy (there are a few places where it happens right now as I need to fix a couple of things)
Just to underline it, currently it's not possible to play with it, until I merge the branch I am working on, because performances would be terrible (only on-disk storage and currently without caching), the tests are broken for the same reason.