Implementing a Social Graph using Redis | Hacker News Reader

Implementing a Social Graph using Redis

blog.meltingice.net

16 pointsabraham3 comments

Because Redis requires the entire dataset to reside in memory.

It doesn't: http://code.google.com/p/redis/wiki/VirtualMemorySpecificati... .. Well, in the classical "in memory" sense. Virtual memory is a sort of memory.. :-)

In practical terms, under scale, it's pretty much the case. If you don't have enough memory to have your entire dataset in memory, the performance degrades to a point where it'll be too slow to do anything, so it might as well not work at all.

Similar problems with MongoDB. Anytime your indexed data starts hitting disk, kiss your site goodbye.

Sharding seems like the proper workaround though, so maybe you could take the same approach of flockdb and shard the redis social graph.

    the performance degrades to a point where it'll be too 
    slow to do anything

Do you have any examples? In my experience, Redis is pretty fast even when it lives partially in VM.

You can get boxes with a ton of RAM these, days, though, and if you go beyond it, you can always shard/distribute the database, like you mention.

Reply on news.ycombinator.com