As part of our work we have done extensive comparisons of Spark on various workloads, clusters and cluster sizes comparing with Hadoop Map Reduce, Naiad and several other frameworks. We've found Spark to be temperamental, hard to configure, and with wildly varying performance, suited only to a small set of computations for which in-memory state reuse is beneficial (mostly it isn't).

In nearly every test Naiad has beaten Spark.

More info on Naiad: http://research.microsoft.com/en-us/projects/naiad/