Really Understanding the MapReduce Model

benjisimon.blogspot.com

10 points·fogus··5 comments

You should just read the (very readable) MapReduce paper, rather than content-free blog posts like this one.

(playing devil's advocate here) there is a big overhead in printing out a 10-14 page 2-column academic paper and actually bothering to read it, no matter how easy-to-read it might be ... blog posts could be good teasers to generate interest and get people motivated to read the real papers.

And you should avoid "tiny" and other homebrewed Map/Reduce implementations for any serious work. This advice according to Prof. Jimmy Lin during his "Data-Intensive Text Processing with MapReduce" tutorial that I took at the NAACL/HLT natural language processing conference: http://www.umiacs.umd.edu/~jimmylin/cloud-computing/NAACL-HL...

Tiny mapreduce implementations are for learning. I find that to really understand fancy concept X, or fancy framework X in new language Y, I need to port it to a language I know well or re-write parts of it.

Its interesting how things become trends and the next "big thing". The same concept has been around for a long time (divide and conquer), even in CS (though perhaps more related to hardware) in tree machines.

Not to say that google isn't doing great things with the concept, just interesting to note that we've already studied some really complex problems and come up with smart, efficient solutions in the last 50 yrs and a lot of what we think is innovative is anything but.

Reply on news.ycombinator.com