Hacker News Reader

Comment by paraschopra | Hacker News Reader

Redis actually looks pretty good alternative to traditional RDBMS. This tutorial really shattered quite a few myths about key-value databases' real world application. Anybody having experience with Redis? Anybody knows practical implementations of key-value dbs on a large scale?

View on HN

Hello, there are guys using Redis in production with 350 million keys for server (64 GB ram for server). Also there are several other startups using Redis in production. Of course I'm using it in production myself.

If you search 'redis' in search.twitter.com you can see a lot of guys hacking with Redis in paywork environments. We are ready to release Redis 1.0 stable, it'a a matter of one month now. Then we'll have two development trees, one stable and one instable.

To deliver a rock solid product is one of my top commitment with Redis.

Question: (newbie here) if we assume that with 350 million keys for the 64GB ram server we have reached the peak of it's capacity, if we double the number of keys, do we have to get new hardware, like a second server or double the ram?

Or is it something simpler? Like storing the current keys on disk (somehow) and keep going with the server as it is.

To rephrase my question: keeping all in ram... is "all" like ALL ??? Would this solution work for the "real" twitter, with the billion twits, if 350 million keys were created every hour?

If you have more data than a single server can hold the best thing to do is hashing. For instance the Redis-rb client supports consistent hashing.

yes "ALL" in Ram :) this will work for twitter, facebook, everything: what they are doing is to use only the RAM even if they have MySQL, with tons of memcached around. With Redis of course the memcached layer goes away.

> If you have more data than a single server can hold the best thing to do is hashing. For instance the Redis-rb client supports consistent hashing.

Hopefully with a secret salt! It would be interesting if someone nasty managed to take down a node by e.g. predictively registering certain usernames.

The issue with key-value store isn't that you can't create an application, its that the tools to control data integrity and access your data in unexpected ways are missing or will be alot of work to create. A simple example is that two way key-value pairs for the UID and username in this article. Now any update to one value (or is it key?) must update the other.

If your main concern is scalability, then Redis and the like can look very attractive, but we should be clear on what is being given up: data coherence and query flexibility. That may be fine on your app, or even many apps, but its important to point out what is missing.

I agree in general, SQL is like magic (this is why it can't scale). You write abstract tables and then automatically you can do incredible queries against it, and this is very cool indeed.

But there are some kind of problems that Redis can fix in a very natural way. For example the idea of PUSH and LRANGE solves the get-last-N-items problem in a trivial way compared to MySQL.

Also Set intersection is able to solve tagging easily compared to the complex queries needed in a SQL db.

So SQL is very flexible, but strangely enough some trivial data access pattern is hard to model and slow even if it is as direct as take the last N items from a list.

I actually first heard about Redis from a talk at the latest Ruby meetup in our area. The main things I took away from the talk is that it's really fast and is a pretty good way to manage sessions.

You can find more information at the Redis Google Code page:

http://code.google.com/p/redis/

antirez, the author, occasionally hangs out here, and I know that he's using it in production.

Reply on news.ycombinator.com