In this example, did we learn what is faster, Java or Scala? Nope. But we learned a lot digging for explanations why the results are different, which will hopefully result in having better cohesion between languages and the underlying platform.
Performance is often FUD when it comes to languages. Clojure, Scala, Haskell, and even Common Lisp are plenty fast enough for most purposes. Hell, Python and Ruby are fast enough for most applications. Besides, runtime performance has more to do with how the code is written than anything else. You can write very fast C++, if you spend a lot tweaking it and hire experience and extremely expensive ($300k/year and up) programmers, but typical C++ isn't any faster than well-written code in other languages. (I've seen C++ projects fail for performance reasons related to maintainability issues that arguably wouldn't exist if Haskell were used.) However, invoking performance is a great (if unreliable, given who it brings on to the field) way to scare decision-making business people (toddlers with guns) into taking your side on an issue they know nothing about.
This is a place where it'd be better if programmers were a little more politically savvy. (Bringing The Business into a technical dispute is not politically savvy. It ruins everything, in the long run. Never invite executives, also known as toddlers with guns, to anything. As many a Chinese noble learned about opening The Wall and letting the Mongols in to fight one's battles, it's impossible to get them out after it's done.) Let's say you have a team of 5 programmers who want to write something in Python, which is (for most purposes) fast enough. One of them stands up and says, "oh no, we can't do this in Python because if we end up running this on 100,000 boxes it will be too expensive, so we can only use C++" (premature optimization). If he were more politically savvy, he'd build the thing in Python and then, if the software were to run on 100,000 boxes, rewrite performance-critical pieces in C++, and justify a bonus for himself by pointing to the 20,000 CPUs that were just deprovisioned. And a year later, a month before bonuses are disbursed, he can rewrite another performance-critical component. This is good for him (he actually gets recognized, instead of being that annoying guy who bludgeoned a team into writing C++ and taking 4x as long to deliver an MVP) and for the business (only performance-critical components, with price tags large enough for him to care, get rewritten).
Oh, and if you think that part is constructive and boring, and you came here for a language holy-war, I do think both Java and Scala suck as programming languages, because they both allow me to write stupid programs with performance problems.
I assume this is sarcasm, but I actually worry about the fact that Those of Us Who Care About Languages are too divided over minutiae like Haskell's syntactic whitespace and Clojure's parentheses, and that may be why The Business comes in, mushroom stamps us and says, "I'm sick of your shit, programmers. Everything has to be in Java."
(Actually, if you've watched Orange is the New Black, you know that The Business has taken to calling software engineers "inmate", but that's another discussion.)
I like Haskell and I like Clojure for very different reasons, and they are very different languages, but the day-to-day real-world differences between them are small compared to the very real risk of The Business overhearing our flamewar and saying, "fuck you guys, Java all the way, now lick my SCRUM or it's minus-5 story points for you."
One of the things that worries me about Clojure is that, while I'd argue that it's the best (for a definition of "best" that includes short-term business viability; this enables me to exclude obscure niche languages that may be better on paper, but that are just too numerous for me to know anything about) dynamically-typed language-- a great language on its own right, but sitting on top of the JVM and having access to those libraries-- there's been a mind-share split between Clojure and Scala. And while Scala/Java and Clojure/Java interop aren't bad, Clojure/Scala interop is a mess. On top of this, while Odersky's brilliant, I think Scala has taken in a little too much of the Java culture for it's own good. Scala's a fine language to write in, but large Scala codebases are generally things that I'd rather not risk my sanity and career by being anywhere near them. If Scala wanted to be "Haskell with Java libraries" it would been a different and harder fight... but then again, it might not have taken off at all without the "slightly better Java" crowd, so maybe the way things happened was the only possibility.
The mind-share split between Clojure and Scala scares me, because it generates a very real risk that the intellectual energy that I'd like to see benefitting both languages, or at least consistently benefitting one of them, might fall back down the tree onto Java. The real risk, to me, is "Clojure vs. Scala: Divided We Fall". But I don't know exactly what to do about it.