romange

I think its ok that we have innovation at legal front as well. And innovation requires exploration, choosing road not taken...

Yes this problem is complex and i guess this is why it have not been solved before.

We do provide atomicity guarantees for all operations like Redis! We use an algorithm from a 2014 paper - see our readme, we provide the link to the paper.

We may do it in the future. We will see if the community will accept Dragonfly with BSL. MariaDb were pretty famous and i do not remember any loud objections to their license.

By preventing them from using dragonfly. Many companies prohibit from dev teams(aka our users) from downloading and using agpl licensed software.

Redis has a BSD license because it was written by antirez who did not intend to make it a commercial enterprise. Today its too late to change and wont help Redis Inc to protect their ip. We looked at elastic, mongodb, cocroach labs, mariadb, redpanda, atlassian, grafana labs , scylladb. The last two are agpl but the motivation is the same: to protect themselves.

Exactly. Finding the right balance is the right thing to do. I am not angry at cloud providers , but i am angry at OSI and their blind idealism that leaves us out there alone. I think that not solving the problem by them, hurt first of all the oss community.

I do not have any practical, first hand experience with either license. Re -agpl, i know for a fact that it is banned in corporations. We choose BSL to avoid this. Say if a team in amazon would want to use DF internally they could. I know for a fact that AGPL is a big no there. If we are mistaken and BSL is also banned then we will need to see what to do.

I hope people will understand the reason behind bsl. I hope people will see apache there and will help us to grow. Based on HN comments some already do.

It does not. If you refer to Redis Modules we are not planning to support those - we can just implement same features within DF.

Well in that case, I hope to have an good answer for you. Right below the benchmark graph I deliberately put a table that shows 99th percentile lantencies at *peak* throughput as reported by memtier_benchmark. 99th latency percentile of Dragonfly at its peak throughput. I put it here as well

  op r6g c6gn c7g
  set 0.8ms 1ms 1ms
  get 0.9ms 0.9ms 0.8ms
  setex 0.9ms 1.1ms 1.3ms

Yes, I found many different bottlenecks.

I was mostly running on AWS. In terms of hardware, for small-packets loadtests, most systems are constrained on throughput, i.e. number of packets per second. Some instances saturate on interrupts reaching 100% CPU on all cores and some can not even saturate the CPU and you will see that CPU is at 60% but you can not go beyond in throughput. The best systems network-wise are c6gn family types. They are also better than instances that other cloud provide. btw, you mentioned hypervisors... About 8 months ago I opened a bug on AWS Graviton team https://github.com/amzn/amzn-drivers/issues/195 - about performance issue they had on their instances at high throughput. Recently they issued the fix. I suspect it was in their hypervisor.

In terms of my software I found many performance bugs at those speeds. For example, using a default allocator is a big no. I use mimalloc for uncontended allocations. In general, you can not use mutexes and spinlocks at those speeds. Those will just cripple the system. Sometimes it can be very annoying since you can not rely on a 3rd party library without carefully analyzing its design. For example, I could not use openmetrics c++ library because it was not performant enough. Even to implement a simple counter, say to gather statistics for INFO command becomes an interesting engineering problem: With share nothing architecture, I use a lot of thread-local counters that I aggregate only when stats are pulled.

As a general note, I expect that Dragonfly will stay very performant with the tailwinds from recent hardware advancements. For example, c7g (Graviton 3) is much better than c6g and DF shows it.

it could be done technically. not sure how widespread that usecase would be.

dragonfly is a linking of a library and a main file dfly_main.cc so without this file you will have the lib.

Redis is "Remote Dictionary Server". You gonna loose the remote part :)

We will be able. It's just matter of time and priorities.

I do not know much about RDMA. Our goal is to provide a memory store that is fully compatible with Redis/Memcached protocols so that all the existing frameworks could work as before. I am not sure how RDMA fits this goal.

not yet. We are focused on building the community at this point.

You are welcomed to send a PR with the fix :)

Everyone has his dirty laundry :(

Thanks for providing your feedback. As Redis Manifesto states - our goal is to fight against complexity. antirez - you are our inspiration and I seriously take your manifesto close to heart.

Please allow the possibility that Redis can be improved and should be improved. Otherwise other systems will eventually take its market apart.

I appreciate your comments very much. I've wrote about you in my blog. I am an engineer and I disagree with some of the design decisions that were made in Redis and I decided to do something about it :) to your points:

1. DF provides full compatibility with single node Redis while running on all cores, compared to Redis cluster that can not provide multi-key operations across slots.

2. Much stronger point - we provide much simpler system since you do not need to manage k processes, you do not need to *provision* k capacities that managed independently within each process and you do not need to monitor those processes, load/save k snapshots etc. Our snapshotting is point in time on all cores.

3. Due to pooling of resources DF is more cost efficient. It's more versatile. We have a design partner that could reduce its costs by factor of 3 just because he could use x2gd machine with extra high memory configuration.

Regarding your note about memcached - while we provide similar performance like memcached our product proposition is anything unlike memcached and it's more similar to Redis. Having said that - I will add comparison to memcached. I do believe that memcached as performant as DF because essentially it's just an epoll loop over multiple threads.

Re you comment about snapshotting. We also push the data into serialization sink upon write, hence we do not need to aggregate changes until the snapshot completes. The complex part is to ensure that no key is written twice and that we ensure with versioning. I do agree that there can be extreme cases where we need to duplicate memory usage for some entries but it's only for the entries at flight - those that are being processed for serialization.

Update: re versioning and memory efficiency. We use DashTable that is more memory efficient that Redis-Dict. In addition, DashTable has a concept of bucket that is comprised of multiple slots (14 in our implementation). We maintain a single 64bit version per bucket and we serialize all the entries in the bucket at once. Naturally, it reduces the overhead of keeping versions. Overall, for small value workloads we are 30-40% more efficient in memory than Redis.

We do not choose AGPL so we would not harm our users. Lets not confuse the means and the goal here. AGPL is copyleft and restricting. Fair users refuse using it. OSI does not help technological companies to find a fair solution and force us to go outside of "OSS".

I remember that I read about Anna. Very interesting paper. From what I remember that require that the operations will be conflict free.

I think came to conclusion that it could be interesting as an independent (novel) store but not something that can implement Redis with its complicated multi-model API, transactions and blocking commands. I do not remember all the details though...

Not really but I read pelikan posts by twitter team.

One thing in common - we both thought that cache-based heuristics can be largely improved compared to memcached/redis implementations. We did it differently though. I think our cache design has academic novelty - I will write a separate post about it.

no it's not us. we are two guys right now. bootstrapped with hopes that community will embrace the project and we will be able to grow the team.

btw, it's not just pretty - it has brains too. take it for a spin.

Guys, please open a bug in the repo and plus one it there :)

We plan to implement everything but your votes can affect the priority of the tasks.

We compared DF with Redis OSS and KeyDB OSS, we have not used any of managed services for the comparison. AWS was used as a platform for using EC2 and cloud environment.

How you would suggest to demonstrate visually that there are no trade-offs?

The tradeoff the way I see it - one needs to implement 200 Redis commands from scratch. Besides, I think DF has a marginally higher 50th percentile latency. Say, if Redis has 0.3ms for 50th percentile, DF can have 0.4ms because it uses message passing for inter-thread communication. 99th percentiles are better in DF for the same throughput because DF uses more cpu power which reduces variance under load.

Re-durability - what durability is offerred by Redis? AOF ? We will provide similar durability guarantees with better performance than AOF. We already provide snapshotting that can be 30-50 faster than of Redis.

omg, can I marry you?

Sorry, missed your second question. Redis is using fork-based save for snapshotting and full replica sync. This means that memory pages are duplicated on write. That consumes more memory than DF implementation that implements algorithmic point-in-time snapshotting. In addition DF was designed to use less overhead memory in its keys table. As for the why, you can read about it here: https://github.com/dragonflydb/dragonfly#background

Ok you got us. We chose dragonglydb and not dragonflystore just because the former sounds better on tongue :)

Having said that we carefully choose to write everywhere in the docs thay we are in-memory store (and not the database).

Btw, I reserve full rights to provide full durability guarantees for DF and to claim the database title in the future.

Thanks!

I've wrote in in the background section of Readme.

Basically, I worked in a cloud company in a team that provided a managed service for Redis and Memcached. I witnessed lots of problems that our customers experienced due to scale problems of Redis. I knew that these problems are solveable but only if the whole system would be redesigned from scratch. At some point I decided to challenge the status quo, so I left the company and..and here we are.