reconditerose

I'll get on my soap box then. The redis client ecosystem is fractured and awful, and it's mostly Cluster's fault. Getting cluster right is really hard, and most clients do it inconsistently and get something wrong, if they tried at all (hiredict doesn't for example).

> I think there was some mention at some point about our two forks working together on maintenance of the protocol specification independently of Redis Ltd, which would be a good way to ensure that the clients remain broadly compatible with both of our forks.

Yeah, I'm still fully aligned with this.

> Interestingly, there have only been two commits from a single developer to the Redis repo in the last two weeks since the license change. A huge decrease.

It's just a guy from Redis. It's not even one of the three former maintainers (Oran, Yossi, or Itamar). The number of open PRs is also dramatically down (it was around 550 when I last looked before the fork, since I've gotten a lot of notifications from old PRs of people refusing the CLA).

I think there is a few things I would like to see in the mid to short term. We're trying to make a lot of the core datastructures more efficient (both in terms of memory and performance) as well as the main dictionary. Valkey 8.0 (or whatever first major version we have) will have lower overhead per key-value pair. Multithreading performance is nice, but as you mentioned most people don't need it. It gives folks a lot of runway if they need to scale but don't want to use clustering, and can also be a "quick fix" for certain types of P99 or higher latency spikes. Clustering is also really hard to use today, and a lot of the current folks want to fix that. It's a huge community pain point. Observability is another pain point I would like to improve. (Disclosure, I work at AWS as well) We see a lot of customers ask about "why did we see a performance issue", and Redis really doesn't provide a lot of introspection to diagnose those issues.

Another big area, which maybe will work for redict, is we want better integration with other OSS projects, especially CNCF projects like envoy, open-telemetry, K8s. There are a lot of self-developed projects floating around, we're hoping we can pull all of this together to make a more cohesive project for people to use instead of what we see today.

I think another big issue will be clients. I'm concerned Redis will try to inject a poison pill into the clients they own to make it so they can only talk to "official" Redis versions. Ultimately a small community will struggle to maintain a lot of clients, so I think we need a larger (and likely commercial) investment into keeping clients open. We will likely passively support redict with our clients, so they'll get that for free.

We want to do all the cool feature stuff to (timeseries, JSON, bloom, RAG), but I would like to keep the core pretty clean.

So, we thought the best approach was to stick with the license and get us into a vendor neutral foundation. We haven't ruled out LGPL, we just didn't make the change unilaterally and as a foundational part of the fork. We just want to keep building :)

This is partially true, but is missing the fact that all of the recent major contributors were coming from other IT companies (AWS, alibaba, tencent, huawei, Google). None of them would have contributed with this license change. Most of the major features (TLS, ACLs V2, sharded pubsub, Functions, multi-slot dictionary) were all funded by companies, not individuals. It's hard for an individual to contribute complex features these days, since systems are so complex. Even those that do, it required a lot of help from folks like me or other contributors to get it merged.

Disclosure, I was the pervious maintainer of Redis and am employed by AWS.

The old development community is regrouping under https://github.com/placeholderkv/placeholderkv. As of right now we don't intend to change the license, we just want a more balanced governance that prevents a corporate entity to make this type of choice. We are currently planning a five person governance, with everyone coming from a different company. (We would love some independent maintainers, but there weren't any at the time

No, you can't have this both ways. I'm the main contributor from AWS, and I've worked many times on weekends because I care about open source. I like helping people, I don't need to be paid to do it. Many of the AWS folks that made changes were normal engineers that were excited to be part of Redis. https://github.com/redis/redis/pull/10419 and https://github.com/redis/redis/pull/8621 are both examples of features someone from AWS built in their free time. We're all upset about this. Not because Redis deserves to get paid, it's that they acted like they were being good stewards of the open-source community and then they changed their mind.

The main innovation that makes it interesting is that it uses a common rust core, built on top of redis-rs, to implement the heavy lifting of cluster topology and connection management. This code is much harder to implement correctly compared to the RESP protocol for simple commands. Higher level language wrappers interact with this core over a protobuf based protocol.

The design is flexible, in the future it could support alternative backends (like memcached) and should be easily extensible for for new languages. I know the team is looking for folks interested in contributing.

I think with Redis, you are starting knowing that your use case needs to be fast. You start with the access patterns and fit that into key/value lookups (or search with modules). You're typically denormalizing the data, duplicating it or aggregating it from other places, so that you can serve operational data fast. Otherwise, just use in-memory SQL.

Seems way off the mark for why people use Redis. Developers use it precisely because it's not like their classic databases.

The most useful measurement I've seen is you pick a latency target, and evaluate how many QPS you can send to the server that meets that latency target. That gives a fairly simple dimension to compare to.

Do you run stand alone Redis or Redis cluster? Multithreading is a strategy, but it isn't the only strategy to improving raw performance.

> is there somewhere i can read more about that? curious about the server needs to do that. i worth key/value clusters a little larger then that. if possible it would be cool to use redis for it.

Redis scales to about ~1000 nodes with each node supporting about 200k qps, math gets you to around 200m qps as the practical upper bound for Redis these days. Multithreading would potentially help us push clusters to the billion RPS boundary.

The conversation happened here, https://github.com/redis/redis/issues/8340, and it's not like the most pressing issue for the project. It's also not as complex as what was implemented for dragonfly, which basically has native support from the ground up for concurrent programming during command execution. It would be hard to do in C as well.

There was an optimization built for Redis 7 where we actually start return copied memory pages back to the kernel, https://github.com/redis/redis/pull/8974, I wonder if the testing provided on Dragonfly includes this optimization.

Glad to see you guys made a lot of progress, although a little disappointing you chose to go down the path of building yet another source available DB and not contributing to open source.