Hacker News Reader

Comment by tlianza | Hacker News Reader

I find these kinds of stories interesting, but without some feel for the size of the data, they're not very useful/practical.

I've heard of Bump, and used it once or twice, but I don't actually know how big or popular it is. If we're talking about a database for a few million users, only a tiny percentage of which are actively "bumping" at any time, it's really hard for me to imagine this is an interesting scaling problem.

Ex. If I just read an article about a "data migration" who's scale is something a traditional DBMS would yawn at, the newsworthiness would have to be re-evaluated.

View on HN

They celebrated 80 million installations as of 2 months ago (up from 50 million 8 months ago). http://blog.bu.mp/introducing-bump-pay-a-new-project-and-app...

That's a growth rate of 5 million installs a month; if they kept up that pace, they're at 90 million installs.

To put that in perspective, Instagram "only" has 50 million users. http://www.quora.com/Instagram/How-many-users-does-Instagram...

More bump data here: http://bu.mp/static/images/infographic_9-2011_6.pdf

I'm not a user, but it seems like they have serious data.

Even at 90 million users, with anything approaching a reasonable level of activity, we're not talking about serious data.

90 million rows of denormalized data isn't a big deal, and if I had to guess, their ops per second is probably no higher than what a dedicated single, or maybe a small master-slave postgres deployment could handle.

Again, something a DBA would yawn at.

And I say this as someone who scaled up an API for a service that plugged into multiple ad networks concurrently for a total of billions of impressions per month with a high level of reliability. Using NoSQL and an RDBMS combined.

People who want to preach the NoSQL message should probably have some actual experience. Otherwise, it just makes very viable NoSQL solutions look really bad.

90 million users certainly does not equate to 90 million rows of data.

I was being lackadaisical, deal with it.

I'm not sure what exactly qualifies as respectable scale, but the Mongo master was running out of space and IO capacity with 24 SSDs and 90 million user records, and was replaced by a sixteen-node Riak cluster.

I'll happily share any other statistics you're interested in.

Edit: the Riak cluster actually contains lots of other data (communications, object metadata, etc.); we didn't need sixteen boxes for the user records.

90 million users is a great datapoint, yes! In my book that's more than respectable.

The only other stat that I'm curious about is the total size of the DB. Certainly databases with tens of millions of records can be held completely in RAM these days... but that also depends on how big each record is.

All-told the users database when we started migration was about 600 GB on disk, so not the most easily stored database in RAM, but not impossible if you get enough large machines.

In fact, we use Redis a lot at Bump, although almost exclusively for queueing and transient state, and not as a persistent database. For a period of time we did store long-lasting metadata in Redis, and as we became more popular instead of throwing engineering effort at the problem we threw more memory, culminating with a handful of boxes with 192 GB of RAM each. We've since moved that entire database to Riak. :)

i was sort of hoping you'd tell us why you wanted to move away from mongo.. but it was your mongo master deamon that was the bottle neck; right?

To be clear, you are comparing a sharded Riak cluster against an unsharded MongoDB installation?

I was actually going to make a joke about "if the number of people I know who actually use Bump is any indication, it's not clear they even need a large data store."

Reply on news.ycombinator.com