Hacker News Reader

Does anyone know how the different storage systems are utilized, and why each system is utilized for that purpose? The presenter mentions using memcached, Cassandra, and PostgreSQL, and mentions the same type of data when discussing each (votes, for instance). I would definitely benefit from a more in-depth understanding of how each system is utilized, and why.

View on HN

Each tool has a different use case. Votes is a great example.

Memcache has no guarantees about durability, but is very fast, so the vote data is stored there to make rendering of pages as quick as possible.

Cassandra is durable and fast, and gives fast negative lookups because of its bloom filter, so it was good for storing a durable copy of the votes for when the data isn't in memcache.

Postgres is rock solid and relational, so it was a good place to store votes as a backup for Cassandra (we could regenerate all the data in Cassandra from Postgres if necessary) and also for doing batch processing, which sometimes needed the relational capabilities.

That makes a lot of sense. Were the majority of your systems using this "durability chain" so to speak -- memcached -> Cassandra -> Postgres? Additionally, in retrospect do you find this type of chain to work well, and would you use it again (perhaps you already are over at Netflix)?

Reply on news.ycombinator.com