Realtime Hadoop usage at Facebook: The Complete Story

This is extremely interesting stuff. The posts that continue tease the problem of having to process millions of events in real time. Here is the direct link to the paper http://borthakur.com/ftp/RealtimeHadoopSigmod2011.pdf incase anyone doesn't feel like clicking through. What I'm very drawn in by is facebook's continued and heavy use of mysql. They didn't run off to some nosql based datastore when things got tough, they learned to scale mysql, they leveraged its strong points while trying to minimize using its weaknesses. And then they looked at alternatives that compliment their existing systems. This is scaling at its best and as a system administrator its the kind of challenge I find both fascinating and exciting.

Interesting, as much as I don't really like facebook.

I only see Hadoop being used even more widely now - that, or spinoffs/improvements, etc., as needed, unless Hadoop really can scale as much as possible.

At least, that's what this article tells me.

http://www.infoq.com/news/2011/06/hortonworks

Is Facebook contributing these changes upstream?