Redis Stack | Hacker News Reader

Redis Stack

redis.io

158 pointshadrianpaulo69 comments

This is pretty much the direction of all specialty database providers (MongoDB, Redis) are moving into. Everyone wants to be able to query datasets across enterprise. I bet is Redis Stack will soon querying against support Parquet or ORC files. Then connector to traditional databases like PG, MySQL, Oracle, etc.. Throw in some AI modeling. In a way, this is a marketing play to stay relevant in the enterprise space.

Redis as a pure in-memory store (caching) engine just doesn't make as much money as "analytic system". I don't have a problem with this as they have to make money somehow.

What a crazy mess of services, so sad to see. redis was such a beautiful, elegant and pragmatic code base that was extremely effective at what it does. It looks like they are trying their hardest to make it more enterprisy to squeeze every last cent out.

Open source has changed a lot recently. Most people/companies do open source to make money. The initial phase is just to make the software popular enough so that a money-making business can be built around it.

Frankly, I don't see anything bad with it. Those who don't want to spend money get it for free and those who can afford will pay for it.

This is I-Can’t-Believe-It’s-Not-Open-Source™, not the real thing.

Is it? I mean, how? The core Redis software is still open source as it's always been, with the same (well, improved over time) functionality, whereas the closed-source for-profit components are not even 'paywalled features' but clearly separate higher-layer products building on the core (visualisation, ML, time series, full-text search, graph structures, &c). To me this is a paradigmatic example of well-done open-core.

Seems great to me. The AGPL style license is a bit much, but they have to make money somehow.

If you read the license carefully you'll notice it's much more permissive than AGPL, and allows you as an end user (as long as you're not developing a database) to do with it whatever you want.

GP likely meant AGPL-style as "MIT/BSD with limitations on usage, like how AGPL is MIT/BSD with a limitation on licensing of client code".

It doesn't change Redis. redis-server will stay simple as it always been, redis-stack-server extends redis-server with addition functionality for those who need more low latency data access more than Redis K/V.

Nothing is stopping you running vanilla Redis... And none of this touches that codebase, as they're modules you could opt to install/load manually?

Isn't this just a collection of addons/plugins on top of the "actual Redis"?

If so, what's your objection? To me this amounts to more or less just expanding the Redis ecosystem, which feels like a win-win.

It's the hot take culture. Some people see internet forums as a vessel to unleash their toxicity.

I don't see how it's 'toxic' to express dissatisfaction with mission creep in a piece of software. It's a totally normal, reasonable criticism, and from the author's point of view is far more useful than their staying silent.

i would agree that this was "mission creep" if it those were not plugins. you can still use 100% vanilla redis without any of this.

Yea I agree, I love redis. I have to admit seeing this group of services was a little strange, but your comment about enterprise money seems on the nose.

Are you paying for redis? I know I am not. Also, you comment is unfounded because redis is still redis, this is a stack build on top of it.

> redis was such a beautiful, elegant and pragmatic code base that was extremely effective at what it does

Doesn't seem like that has changed to me

True, you can always run vanilla Redis.

Having said this, it really gives me the sensation that Redis is trying to solve too many use cases, turning into some sort of Swiss knife.

The reality of Swiss knives is that they don't excel in anything of the things they do.

We feel like the most important part required from a low latency DB, Redis "Vanilla" and Redis Enterprise already cover. Now, we can leverage this amazing platform to provide a real low latency multi model DB on top of it.

you can really do a lot in terms of building structured query engines on top of a simple performant key:value store

It's always been a Swiss Army Knife. It's a single-threaded-in-memory caching, pubSubing, streaming, distributed-locking beast.

>The reality of Swiss knives is that they don't excel in anything of the things they do.

Except the one thing that do excel at: being swiss army knives.

Here's their blog post about it: https://redis.com/blog/introducing-redis-stack/

You might also want to watch this video https://www.youtube.com/watch?v=mUNFvyrsl8Q

You can really build pretty much anything on top of a K-V store, but I would hate to be vendor-locked into Redis ecosystem.

We tried to keep it as vendor natural as we could, for example in the Graph API we picked the openCypher query language and in the JSON API we used JSONPath.

Just develop for an interface, instead of an implementation.

Migraring data is no fun, but much more straightforward than rewriting a good part of your code when you decide to switch database.

Is there any example of this ever working in practice in the history of computing? As in, we built a large, stable service with datastore X, but we were forward looking enough to make sure that we only "developed to an interface" and we successfully swapped to datastore Y without significant code changes?

The project I'm currently working uses AWS DynamoDB as primary DB. Since it's proprietary, we ensure the AWS-specific APIs are contained in small and specific part of the codebase.

For instance, to create a "User" we need the PutItem API from DynamoDB [1]. Similarly, to retrieve a "User", there's the GetItem API [2].

Instead of making references to these APIs all over our codebase, we have a single `db-interface` module, which implements `get_user` and `create_user` functions. Each of these functions has an interface: they expect specific arguments with corresponding types. This interface is modeled against our data domain, not DynamoDB data domain quirks.

Inside the `db-interface` module, we implement the conversion from our data domain to Dynamo's. We also have general-purpose functions, like `create_item` and `get_item`, so that `get_user` and `create_user` are pretty much wrappers and only serve the purpose of defining the interface for interacting with the "User" data object.

The rest of our code only interacts with our internal interface, never with DynamoDB APIs directly.

If we were to switch database, we only need to re-implement the general-purpose functions (e.g. `create_item`, `get_item`).

May sound like a lot of work, but it took only a handful of days to implement the entire interface mapping everything we needed from DynamoDB.

Hope this helps clarifying a bit the application of this concept (develop for an interface, not an implementation) in the context of interacting with databases.

[1] https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...

[2] https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...

That's interesting. Have you tested this with another DB? The design makes a lot of sense, but I feel like it's the kind of thing that's hard to get right unless you actually test it. At the company I currently work, they switched DB some time ago. They already used interfaces but even that was not enough to make the switch, due to some differences in how the DBs worked.

If you can map the differences in your code and your interface does not leak any implementation detail, you won't have problems.

But if the current database has features that can't be mapped to the new one, the interface won't save you.

Example: DynamoDB has a "time-to-live" feature [1]. You add a specific property to any item with a timestamp and it will auto-delete this item at the timestamp determined.

Few databases will provide this out of the box. If you choose to use it and later need to migrate, you'll have to implement your own kind of "TTL monitor" to perform the same task. The interface can't possibly save you from this.

But this would be an issue whether you decide to use a central interface or not, anyway...

[1] https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

Thanks, that "time-to-live" feature is a great exemple of what I was talking about. At work if I remember correctly it was something about not being able to read a write just after making it, and having to wait one second. I guess that's the kind of things where you either need people to be very careful when developing the implementation, or have people that have already been burned by that to think about that.

I understand the concept, I'm just asking if anyone has ever actually done it. DynamoDB has various guarantees, features, edge cases etc that you have to be aware of in your app code, and any DB that you want to switch to will have a different set of guarantees, features, edge cases etc that you will have to support throughout your app. So it's more than just an interface.

The only thing I hate about Redis is that all data has to stay in memory...

Well I know that's obvious, but Redis is nearly the perfect solution in terms of performance. Unlike disk-based databases, Redis has consistently optimal performance on the order of millions of queries per second. And horizontal scaling is trivial with Redis Cluster (it involves running multiple Redis instances and many Redis service providers don't support it - I hope to have an easier way of running cluster though).

Given that memory is cheap it's quite plausible to store an entire database in a single Redis instance. I do hope that Redis finds its way to replace conventional databases. I think we just need an asynchronous `FLUSH` command, which blocks until the next `fsync`, and that should get us closer to ACID compliance.

What is the logic behind separating those from the core server? Binary size? Automatic configuration?

"Wow this has a lot of features" vs "wow what are all these extra things I need to sift through to figure out what I need", no?

All the above :)

I wish product would be small sharp tools that does something well and sticks to it, iterating on making it faster,more stable, better documented, easier to setup and configure.

Instead a lot of product start out that way and then if it becomes popular scope creep rears it ugly head and it is extended to be mediocre at many things.

I find far too many developers once they start investing time in a product they want it to do all sorts of other things that there are allready tools that do it well.

I have used Redis TimeSeries a few years ago and I found it easy to use and very performant for my needs. I think it is still chugging along at my last employer. The development team is already familiar with Redis and therefore found TimeSeries simple and easy to use. None of us wanted to use ( and learn) a dedicated TimeSeries database API and business did not want an expanded footprint to maintain. So Redis TimeSeries was an excellent solution for us

Around two months ago I was excited to learn about RedisJSON, only to get to know that there is no ARM-based Docker image available for installing it on a Raspberry Pi.

What is preventing you to contribute and create it?

Probably some combination of knowledge, experience, time and interest. Frankly it sounds like kind of an intimidating ask. Do you have special knowledge that makes this seem simple, or is it actually not challenging once you really get all the pieces involved? I'd love to be confident that I could see a statement like the one you replied to and immediately execute on it, so if you have any tips to share please do!

Well, to start, have you tried to send information in their issue tracker?

Maybe my answer sounded more harsh in tone than what I meant, sorry for that.

I red the message as "Hey they didn’t cover my specific case!", and I felt like this was unfair to ask an opensource project to cover each corner case upfront and not even consider a lacking feature as something that can be reported or maybe even contributed back.

Nope, I wasn't the top level comment.

Yes, it came off sharp, it would have discouraged me from contributing. Sometimes people don't really get that the process is open to all and need to be invited, I think it can be good to try and see stuff like this as an opportunity to extend that invitation.

I think you misread this one because it is basically "excitement about new thing -> implied disappointment at not getting to use new thing", and I can't find an ask in that.

The redisgraph functionality is really cool, it makes it an interesting choice over SQLite for small data science projects. The search functionality also is helpful for data science projects.

Does anyone know what this RSAL license’s impact is for internal tools (e.g log analysis)?

RSAL 1.1(b): "[Licensor hereby grants to You a ... license ... to:] use the Software, or your Modifications, only as part of Your Application, but not in connection with any Database Product that is distributed or otherwise made available by any third party"

As long as this log analysis is part of the production environment of your application, and you don't resell it as a tool, that should be fine.

Redis Stack sounds like it provides the same features that ArangoDB does. Anyone done a comparison?

I want to do a sentiment analysis on Redis-related HN comments pre and post antirez departure.

That can be a very interesting project :) I would even consider using RedisAI to run the model...

Here is a video that outlines how to build a real-time stock watcher using Redis Stack: https://www.youtube.com/watch?v=LaiQFZ5bXaM

Worth a watch for sure

I don't understand why there is timeseries solution there, usually people who use timeseries work with many Gigas of storage, are we gonna pay for RAM to do that?

First it all depends on your latency requirements and TP, if you need your TimeSeries DB to support high ingest TP and low query latency a RAM based is almost the only way you can go.

Second, many Gigas is what you get from any small size server... And, on top of that Redis/TimeSeries supports an almost linear scale out which allows your memory to increase dynamically according to your needs. Last, Redis Enterprise (& Cloud) add Redis on Flash support extending your RAM to your local Flash storage.

I have run Redis clusters, both with and without Redis on Flash, with hundreds of GBs of data. If you need the performance, it's doable and not prohibitively expensive.

It’s a shame they don’t offer first class Ruby support. I could see myself using some this stuff.

Is this something Redis are likely to offer in future does anyone know?

> they don’t offer first class Ruby support

It's a datastore. It has clients for many languages, including a recommended one for Ruby. What support are you missing here?

To use the extra features that come with Redis Stack, you need a client that supports those features. According to the site posted, the Ruby client doesn't support these features.

So it appears that you are the one that is missing something.

> So it appears that you are the one that is missing something.

They were asking what library support was missing for that person's use case (which your question answers, but I think the snark is based on a misunderstanding).

You're right, we would love to provide a much better experience in Ruby it was just a matter of priorities... BTW, all the modules already have ruby based clients developed by the community.

e.g.

* https://github.com/npezza93/redi_search

* https://github.com/vachhanihpavan/rejson-rb

* https://github.com/dzunk/redis-time-series

You can see on each modules docs the list of clients developed by the community. e.g.

https://redis.io/docs/stack/graph/clients/

Thank you, that'll be a good place to start. Cheers.

The first link on the page: Getting started gives 404, page not found.

Hopefully the doesn't happen with the product itself.

Which link? on which page?

https://imgur.com/a/JWwmb1x

thanks!

Someone's been silly with their links. They've linked the "getting started" text you linked to "https://redis.io/docs/stack/getting-started", however in their navigation they have a separate "get started" link which points to "https://redis.io/docs/stack/get-started"

It would be great if ElastCache supports this

You can still get redis-stack on AWS see Redis Cloud https://redis.com/try-free/

This is made of separate codebases with new licenses, instead of being part of the MIT-licensed Redis, in order to prevent AWS from doing that.

Redis Stack is not Open Source, so that wouldn't be possible. AWS would have to reimplement all of the functionality, make their implementation Open Source, and keep adding new features to keep up.

That time series though...

I would love to hear your feedback

Reply on news.ycombinator.com