Hacker News Reader

Comment by ihodes | Hacker News Reader

I'm curious to know why they chose to use MongoDB? Is it because its free & fast? Or is it truly faster than a propriety SQL system?

Either way, it's nice to see that computation in science isn't stuck in the 90's, which is sometimes how it feels.

View on HN

They go into it in the actual paper, if you're interested. They wanted a document store because their data is flexible (all different sizes and shapes), complex, and they needed secondary indexes. They didn't need transactions and MongoDB was very fast.

They use it as a trivial caching system of pre-aggregated data.

Correct me if I'm wrong but couldn't any number of other caching solutions (such as memcached) be used in this situation?

In other words, are there any features specific to MongoDB that they are actually making use of?

They don't give much details, but it sounds like a very superficial need that could be accommodated by Redis, Memcached, AppFabric, and on and on. Kind of surprised that they chose MongaDB as it's far from the fastest competitor.

Just speculation, but the built-in map/reduce is probably quite useful.

Perhaps, but the description essentially said that there is a custom python layer that aggregates the data, and they punt it into MongoDB with some SQL extraction pattern simply to store it into that silo.

e.g. If I run a query for NAME LIKE 'BLAH%' it stores that resultset. If I run NAME LIKE 'BLAP%' it stores that resultset. If someone else comes and runs either of them, only a direct match will pull my prior results.

It's okay, but it's no big win or case study.

They needed to be able to do some queries on various indexes, so key/value stores wouldn't work. Also MongoDB tends to be only 10% slower then memcache, so not sure why you think its not fast...

Reply on news.ycombinator.com