In-memory database Redis wants to dabble in disk
theregister.com
theregister.com
This sounds to me like not understanding your place in the market or why people use your product. I can name half a dozen classic SQL databases off hand. I can’t name a tool that competes with Redis at their niche. Why aim for an already oversaturated market when you already have a good profitable niche.
Presumably because there are a lot of things that straddles the two, and where you risk losing people to databases or hybrid setups where the Redis use will be simple enough that people won't want to pay for support.
E.g. cases where you can't afford to keep it all in memory where you today might end up resorting to using Redis as just a dumb-ish in front of a database, or even avoiding Redis entirely in favour of in-memory tables or aggressive cache settings for your database even in cases where you might prefer Redis if it could intelligently handle a larger working set more cheaply.
And also to create a buffer given that the level of oversaturation in the classic database market may make their niche a tempting target to try to expand into for some of those to set themselves apart.
Recently DHH of Rails fame gave a talk about Redis and how at their company it's being replaced in favor of long term storage on the now much faster and cheaper SSDs.
The takeaway was that the long caching it enables yields far better real world results in their products. He even had some charts.
I think redisql has been supplanted by zeesql - https://zeesql.com/
If your dataset fits in memory and you want relations then just use PostgreSQL/MySQL. Why would you pick the unique data structures and performance of redis sets/lists/pubsub/HLL and want SQL in front of it?
After working with MongoDb, CosmosDb and DynamoDb, I’m very happy to be back in a proper SQL world with Postgres.
Author of zeesql/redisql here!
Unfortunately I was not able to figure out a clear business model for the module.
If any of you guys need it also the PRO version, just write me.
In the end we went with Aerospike which actually while backed by NVMe, outperforms redis backed by memory. Aerospike in memory is completely absurd speeds and throughput. Sadly the setup is a bit weird and query model is a bit wonky but we worked around it. A bit sad on how expensive the enterprise version is.
> "We can take the lesser-used data that hasn't been touched in a while and shuttle it off to flash where it can sit for a while. When the user comes back eventually, it's very easy for us to seamlessly move it from flash back into memory. And that allows the company to save costs," he said.
> Redis is now planning to extend the concept to disk-based memory to offer support for a three-tiered architecture.
I would expect it to be a very reasonable middle ground that requires almost no work and it is already enough and ready for most use cases.
Surely you need the sorted set in memory to derive rank/manipulate it etc?
Or would you have to shard it and add other elements on top in app code to focus on only a subsection of the data?
If you store it as an augmented balanced tree, then you only need to load the path from root to leaf to get or change rank. For example, if each node stored the number of leafs of its subtree, then you can avoid visiting most branches by simple logic: if you want to find element with rank=14, and left subtree only has 10 elements in total, ignore it and go straight to the right subtree, searching for element with rank=4 there.
we recently implemented this to grow our caches to >50TB
Seems way off the mark for why people use Redis. Developers use it precisely because it's not like their classic databases.
If what you want is persistent k/v, then there are already tons databases that can do that. Redis has a different use case, so what are you trying to say? That you want to misuse tools?
Are there? What's a good, mainstream, persistent k/v for cheap single-node instances? PostgreSQL's KV support isn't good enough, Cassandra and MongoDB expect to be set up as a cluster and have all the overhead of that, BikeshedDB is alpha and I won't touch their upstream for obvious reasons, CouchDB is pretty much unmaintained, ....
LMDB.
FoundationDB, etcd, consul in single node mode.
I’m always shocked that people use Redis for anything other than an ephemeral cache.
I think with Redis, you are starting knowing that your use case needs to be fast. You start with the access patterns and fit that into key/value lookups (or search with modules). You're typically denormalizing the data, duplicating it or aggregating it from other places, so that you can serve operational data fast. Otherwise, just use in-memory SQL.