Why functional programming matters (aka MapReduce for humans)
gist.github.com
gist.github.com
I think this is a better basic example (in clojure) of what makes functional programming and map/reduce powerful (skipping Hadoop, while we're at it):
(defn wc-map [str]
(let [words (split str #"\s+")]
(map
(fn [p] [(first p) (map (fn [_] 1) (second p))])
(group-by (fn [x] x) words))))
(defn wc-reduce [h]
(map (fn [p] [(first p) (reduce + (second p))]) h))
Simple word count function, completely functional, and using the map/reduce concept.This function allows you to infinitely parallelize word counts: you could simply run `wc-map` on a file and `wc-reduce`, maybe even on multiple servers, merge (merge-with concat) the results, and run it through `wc-reduce` again, ad infinitum.