> Let me know of any other runtime platform that even comes > close to the things JVM does.
Some of the more advanced Lisps would qualify.
Some of the more advanced Lisps would qualify.
[1]: https://twitter.com/ChrisGSeaton/status/619885182104043520
[2]: http://cr.openjdk.java.net/~pliden/slides/ZGC-Jfokus-2019.pd...
[3]: https://craftingjava.com/using-java-flight-recorder-with-ope...
But it's not just benchmarks. It's capabilities.
He should please provide references that compare other runtime platforms' capabilities side by side with JVM's capabilities.
How many weeks work do you imagine that would be?
While no language / platform / os / etc is perfect, JVM as a managed runtime platform is hard to beat. More than two decades of research in its JIT compilers, and multiple GC implementations with various knobs and dials for tuning and monitoring. Battle tested.
Someone mentioned SBCL. While I am a fan of Common Lisp, and generally all Lisps, I would cringe to see SBCL with terabytes of heap and hundreds of CPU cores running a serious workload -- and see how well it holds up.
Just an example: You can Google for this, but in 2012 Twitter switched from Ruby on Rails to Java. They have YouTube videos explaining their change over. Basic reasons: performance and scalability. They have to handle BILLIONS of tweets per day and route each one to multiple places with notifications in near real time.
That's nothing SBCL can do, but one can save an executable and start that in a subsecond without the need for a terabyte of RAM.
Generally there is a lot of stuff in the JVM which makes it less attractive as runtime for Lisp: more complex code loading via 'class loaders', lack of support for memory representation of Lisp's data types (like CLOS objects, which are more dynamic than Java classes), lack of TCO, lack of easy AOT compilation, lack of easy image dumps, lack of resumeable exceptions, ...
And in 2019 something like Eclipse still is a bit clunky to run on my 8 core Xeon - part of the reason could be a less than great GUI implementation. In IntelliJ I need to restart my IDE for every simple plugin... probably features like live-updating haven't made it yet into popular apps.
A few tens of magabytes maybe and the JVM runs a complete Hello World in about 60ms, but yeah, Clojure does a lot of work on startup. It's more of a Clojure issue than a JVM issue, though. It's true that HotSpot is mostly designed for long-running applications. Other JVMs -- like Substrate VM (AKA Graal native image, which isn't quite a JVM yet) and Excelsior JET give you AOT compilation (no warmup).
> makes it less attractive as runtime for Lisp
I think Clojurists (which probably outnumber all other professional Lispers combined several times over) would disagree. I'm sure there are use cases where you'd like to use Lisp and HotSpot is not the right choice, but it seems like those cases are vastly outnumbered by the cases where it is. Domains where HotSpot is certainly not the right choice are usually either client-side web or domains where C/C++ dominate.
> lack of TCO, lack of resumeable exceptions
Delimited continuations and eventually tail calls are coming as part of Project Loom (https://wiki.openjdk.java.net/display/loom/).
> lack of easy image dumps
HotSpot's monitoring, management and observability are almost unmatched (maybe Smalltalk is somewhat better in some regards, and BEAM in some others). Whatever you may be lacking on that front is probably offset by something probably even more valuable (these capabilities have been designed over the past decades to suit the needs of millions of developers running huge backbones).
I'm not saying that HotSpot is always the best tool for the job, but it's the best tool for the job for a huge portion of the software industry.
> And in 2019 something like Eclipse still is a bit clunky to run on my 8 core Xeon - part of the reason could be a less than great GUI implementation. In IntelliJ I need to restart my IDE for every simple plugin... probably features like live-updating haven't made it yet into popular apps.
You don't need to use a Java IDE if you want to write Clojure, although the Java IDEs are also pretty unmatched -- a matter of getting used to.
So I need to switch to exotic JVMs to get better performance?
> I think Clojurists (which probably outnumber all other professional Lispers combined several times over) would disagree
Does the number matter, since most of them have a) never used Lisp nor b) do they know anything about Lisp implementation? Things like 'no TCO' in the JVM are no question of the number of people using it, it's simply a fact.
If the JVM would be a better match, then Lisp implementations on it would have a better start-up time - like most Lisp runtimes have.
There are many reasons to use the JVM, but good support in the JVM for Lisp implementations is not one of the strong points.
> Delimited continuations and eventually tail calls are coming as part of Project Loom
Which means, after reading the linked doc, that neither resumeable exceptions nor TCO are in JVM, neither currently nor in the near future. Now the usual arguments about TCO are: a) one does not need TCO b) it makes tracing difficult c) it's a security problem d) we can do recursive self-calls e) a future version of the JVM will provide annotatations for tail calls f) using TCO would provide interop problems with JVM code... etc etc.
The fact remains: TCO is not supported by the JVM, while this capability is a part of many other runtimes.
> probably even more valuable (these capabilities have been designed over the past decades to suit the needs of millions of developers running huge backbones).
That's all great, but I don't develop huge backbones all the time.
> I'm not saying that HotSpot is always the best tool for the job
Your original claim was this: 'if someone is going to assert that another platform is competitive with JVM, then not only is performance important, but capabilities as well.'
We are now talking about capabilities like full TCO and the JVM simply does not provide it. It currently provides no TCO at all.
> You don't need to use a Java IDE
It's not about what I need, it's that capabilities like extending the IDE on the fly are still not available in an IDE like IntelliJ - why? After downloading a plugin, it wants to restart the IDE. Something which a typical Lisp system on a Lisp runtime brings for free, because of its capabilities.
Depends what you mean by "better". HotSpot is optimized for peak performance (which is higher than that of other Lisp runtimes), and to suit the needs of server-side development. Other VMs target other uses -- fast startup, embedded etc.
> If the JVM would be a better match, then Lisp implementations on it would have a better start-up time - like most Lisp runtimes have.
It's already a better match, because that's what most Lispers prefer. If you mean it could be even better than it is, then I agree. I also agree that it's not the best match for, say, Racket. Now, it's perfectly fine if you prefer Racket over Clojure, but most people prefer Clojure to Racket, partly because overall the runtime gives them more of what's most important to them. Would it be cool if the JVM had more features that would allow it to be a better match for Racket? Sure.
> Does the number matter
Yes, because Clojure is the most popular Lisp, the features it has are probably the ones most important to Lispers.
> Things like 'no TCO' in the JVM are no question of the number of people using it, it's simply a fact.
Yes, but how much not having TCO matters can be answered by popularity. That people use Clojure much more than they use Lisps on runtimes with TCO suggests that other things matter to them more.
> neither currently nor in the near future
Currently, no. Near future -- probably. I'm the technical lead for that project, and it's very likely that delimited continuations would arrive soon (in fact, you can already build the prototype today, and early access builds will be available in a couple of months).
> Now the usual arguments about TCO are ...
Well, as it's being addressed only now rather than, say, 10 years ago means that it's not a top priority, but judged to justify some reasonable effort at this point in time.
> TCO is not supported by the JVM, while this capability is a part of many other runtimes.
True, but the fact remains that the totality of capabilities of the JVM is considered more attractive by more people than that of most other runtimes.
> That's all great, but I don't develop huge backbones all the time.
I am not saying that you, personally, should definitely use the JVM, only that it seems that the JVM is a great platform for many languages, including Lisps, and that the use cases it targets are very desired.
> Your original claim was this ...
I believe I also said that I mean capabilities for the domain HotSpot targets. Obviously, other things may be more attractive for, say, embedded software.
> It currently provides no TCO at all.
There are other things it doesn't provide. But by comparing capabilities I did not mean one by one, but the totality of those of one runtime vs. those of the others.
> It's not about what I need, it's that capabilities like extending the IDE on the fly are still not available in an IDE like IntelliJ - why? After downloading a plugin, it wants to restart the IDE. Something which a typical Lisp system on a Lisp runtime brings for free, because of its capabilities.
No, that capability is definitely there. I'm guessing the IDE authors just didn't think it's worthwhile to use. NetBeans, for example, doesn't require a restart after downloading plugins (or, rather, rarely does).
What relationship have continuations to making Java exceptions resumeable?
> But by comparing capabilities I did not mean one by one, but the totality of those of one runtime vs. those of the others.
Not sure how you compare a totality, other than actually having a list and compare them one by one.
My feeling: you make a claim, but you have never used one of the Lisp runtimes like Allegro CL and you simply don't know what they provide and when it comes to specific capabilities, they are not important to YOU, because JVM users don't use them - given that they have no choice.
The JVM isn't the predominant enterprise platform because it is the best Lisp platform. Very few people in companies care whether it runs Lisp well or not. People have chosen it as their platform for their niche language, because of its domination in the specific enterprise market - not because it is technically or capability-wise a good platform for their niche language. Actually even though the JVM now exists for a long time, none of the dynamic languages have their best / most performant / dominant implementations on the JVM. Not Smalltalk, not Lisp, not Scheme, not Python, not Ruby, ...
We don't want resumable exceptions for Java. But if you want to implement a Lisp that has them on the JVM, you'll be able to use continuations for that.
> but you have never used one of the Lisp runtimes like Allegro CL and you simply don't know what they provide and when it comes to specific capabilities, they are not important to YOU, because JVM users don't use them - given that they have no choice.
I have never used Allegro, and, as I said, some capabilities that are important to some people may be missing on the JVM. But it seems that most Lispers prefer what the JVM has to offer, which is high performance, low-latency GC, low-overhead monitoring, and a huge ecosystem.
> not because it is technically or capability-wise a good platform for their niche language.
True, but most people don't care about what's good for their language, they care about what's good for their program (and users). It seems like more people prefer Lisp on the JVM, even if they have to give up some features (and they'll need to give up fewer features with Loom).
> Not Smalltalk, not Lisp, not Scheme, not Python, not Ruby, ...
Python and Ruby are actually most performant on the JVM nowadays. Look up TruffleRuby and GraalPython. They're just very new. Even TruffleJS is almost as good as V8.
So the Java libs I'd have to use would not support them. Not so great.
> I have never used Allegro
So the totality of your opinion of the JVM being better for Lisp comes from exactly what knowledge? Oracle marketing?
> But it seems that most Lispers prefer what the JVM has to offer, which is high performance, low-latency GC, low-overhead monitoring, and a huge ecosystem.
You mean Clojure users recruited from Java? I have never seen many Scheme, Common Lisp, Javascript, Julia, ... users on the JVM.
> Python and Ruby are actually most performant on the JVM nowadays. Look up TruffleRuby and GraalPython. They're just very new. Even TruffleJS is almost as good as V8.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Really? Looks to me that it's not particular fast.
[GraalVM CE, Java 1.8.0_202] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Please note: This Python implementation is in the very early stages,
and can run little more than basic benchmarks at this point.
Probably the largest Python implementation in the world...
Well, resumable exceptions are not so great... We really don't want them in Java (remember that Java is a language originally designed by Lispers).
> So the totality of your opinion of the JVM being better for Lisp comes from exactly what knowledge? Oracle marketing?
I've used Racket (and PLT Scheme before it) quite a bit (I was a Schemer when Java barely even existed), and Guile, and, of course, Clojure. Also, Oracle marketing is quite terrible.
> You mean Clojure users recruited from Java? I have never seen many Scheme, Common Lisp, Javascript, Julia, ... users on the JVM.
I started using Scheme in the mid-nineties. Unfortunately, I haven't seen many Scheme and Common Lisp users anywhere. Clojure is the first in over three decades to be used on any noticeable scale in industry. There are plenty of people that run JS on the JVM (probably ten times all non-Clojure Lispers), and Julia is still new.
> Really? Looks to me that it's not particular fast.
It's very fast. Here's some benchmarks with early prototypes: http://thezhangwei.com/documents/oopsla113-zhang.pdf
Beats PyPy.
I thought they are?
> We really don't want them in Java (remember that Java is a language originally designed by Lispers).
Who of the Oak team were Lispers? Not even Gosling has ever seen or used a competent Lisp implementation - he wrote a 'Mock' Lisp without lists ;-) for an Emacs written in C on Unix. One of his many projects. Stallman later implemented a slightly better Lisp using Goslings C code as a start.
> I've used Racket (and PLT Scheme before it) quite a bit (I was a Schemer when Java barely even existed), and Guile
None of them have especially interesting runtimes. Racket now gets a new runtime based on Chez Scheme from Cisco. Chez Scheme is actually great. You should have studied that. Well, source code is now available from Cisco. It has a nice AOT native compiler.
> I started using Scheme in the mid-nineties. Unfortunately, I haven't seen many Scheme and Common Lisp users anywhere
You were too late. SUN had its own Lisp offerings in the 80s. Like DEC/DIGITAL, IBM, Apple, HP, Apollo, Texas Instruments, Xerox, ...
Java outperforms SBCL by a very wide margin -- 1.25-4x faster -- except in the short-running benchmarks (and except the spectral-norm test, where SBCL is 5% faster), as the game does not perform warmup, and includes Java's compilation time (you can see that no Java result is under 2 seconds).
> you can see that no Java result is under 2 seconds
That's because the workloads were chosen to push Java results above a couple of seconds.
> as the game does not perform warmup, and includes Java's compilation time
How many 1/10ths of a second do you imagine that takes?
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
I have looked at some example programs in the shootout game where different implementations use completely different low-level algorithms, including code that's very unidiomatic, so those benchmarks are of low quality. And yes, Java does have a disadvantage here (I work on OpenJDK, BTW, so I don't need to imagine).
Anyway, I’m not sure which side you’re arguing, because the benchmark game did show a rather drastic win to Java (TBH, I would have been surprised if that were not the case). I simply qualified that win because the shootout game is not a high quality benchmark.
Which programs?
> including code that's very unidiomatic
Which programs? (And don't you expect code written to be fast to be different?)
> I work on OpenJDK, BTW, so I don't need to imagine
So why haven't you told us how few 1/10ths of a second?
> I’m not sure which side you’re arguing
I don't care one way or the other, I'd just like your comment to do better than name-calling.
All these things make the inter-language comparison as presented nearly useless. Crucially, the site itself doesn't even state what it is that it purports to measure or compare, like a paper without any hypotheses, claims or exposition. That alone is enough to place it in the low-quality benchmark category (regardless of the merits of the problems themselves, which I haven't looked at). But that's where most non-professional benchamrks are. ¯\_(ツ)_/¯
Now, I would guess that there is some meta-hypothesis to the game pertaining to markets, but that's not what's being analyzed or even presented.
> Which programs?
I tried to look at programs that report big performace differences and I'm looking at C++'s regex-redux, and I see it uses a library that isn't in the standard library. Or look even at "Java" vs "Substrate VM" pidigits. These are two VMs for the same language, yet the programs are completely different. I'm not sure what I could learn from that about HotSpot vs Substrate VM.
Or, the example that caught my eye when I reached the game when looking for SBCL benchmarks -- SBCL's reverse-complement, that is reported to outperform C by more than 10x. Well, not only does it use a completely different low level algorithm (doesn't use parallelism), it doesn't outperform C or Java at all, but crashes, and what appears to be some junk result is reported. Had it not crashed, it would still have used a very different algorithm.
> So why haven't you told us how few 1/10ths of a second?
Because that would be misleading. Now, I admit that what threw me off at first was Java's performance compared to SBCL's reverse-complement, but that's just a mistake and has nothing to do with compilation. Real programs can take many seconds to compile, but looking at the benchmarks, they do look quite small, so you are correct that they will be compiled rather quickly (less than a second). However, I see that Java is run with tiered compilation, and obtaining good optimization, even for such small programs, can still take many seconds (say 10 or 30). As a general rule of thumb, we always warmup for at least a few seconds, even for microbenchmarks.
It is true that the difference would often not be large compared to the inter-language differences reported by the benchmark -- and as I said above, it's unclear what it is they compare -- but they can certainly be in the 5-10% range, and maybe more. This is a big difference for what I call "high quality" benchmarks, benchmarks intended to really give a precise performance measurement. For example, Twitter has a whole machine learning system that tunes and re-tunes their JVMs just so they can get a 7% performance improvement. Even the game's own examples that you linked to and seem to say, see, warmup doesn't matter, report differences of 10-200%.
> Which programs?
Now I'm looking at Haskell's fannkuch-redux
> And don't you expect code written to be fast to be different?
Yes, and if the comparison was for the same language (and compiler) then this would be interesting, but what does this tell you when comparing different languages?
Browsing some more I see that the binary-trees programs for Java and Substrate -- again, same language -- use completely different algorithms. Not to mention that just the "Java" programs alone use such a weird range of parallelism constructs, that I have no idea whether there was any reason behind it, the programmers simply didn't know about proper parallelism constructs, or the program is so old that it was written before newer constructs were introduced.
The program with the fastest measurements on OpenJDK did not have the fastest measurements on Substrate VM.
Someone with sufficient curiousiy might choose to investigate — Why?
Does it? How do you know that the same programs were run?
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Is that what the presentation emphasizes most?
"Will your toy benchmark program be faster if you write it in a different programming language? It depends how you write it!"
"Which programs are faster?"
"Which programs are fast?"
"Non-motivation: We are profoundly uninterested in claims that these measurements, of a few tiny programs, somehow define the relative performance of programming languages."
2) Or look even at "Java" vs "Substrate VM" pidigits.
Currently 2 Java pidigits programs are shown.
The program with the fastest measurements on OpenJDK uses GMP and something different needs to be done to make that program work on Substrate VM.
3) …but crashes, and what appears to be some junk result is reported.
What is reported? Measurements at the largest workload both programs worked.
4) examples … you… seem to say, see, warmup doesn't matter
See, for programs with runtimes of 0.24s, 1/10th of a second is a lot!
See, for programs with runtimes of 4s and 20s, 1/10ths of a second… !
5) Now I'm looking at Haskell's fannkuch-redux
And… ?
6) … if the comparison was for the same language (and compiler) then this would be interesting, but what does this tell you when comparing different languages?
That seems to be a subjective statement about your interests.
Someone else might think that neither "tell you" much until you investigate Why?
Yes, because that's what the site shows most prominently: comparisons between languages.
The fact that the explanations to the benchmarks basically say that these comparisons mean nothing doesn't make the comparisons mean much more than nothing, especially as there's no explanation to what the comparisons do (or are supposed to) mean. So in addition to having no hypothesis, no analysis and no exposition, there's a notice saying, don't use these comparisons for so and so. Then what are those comparisons for?
Again, I don't think the game's comparisons are worse than many other benchmark comparisons out there, but that doesn't make it good. What's unfortunate is that unlike those other benchmarks, they have a lot of data that could be used for some really cool stuff (not to compare languages).
It's possible that the language competition is meant to encourage more people to submit and show how their favorite language wins, but there's no information about the market aspects.
> See, for programs with runtimes of 4s and 20s, 1/10ths of a second… !
I run a toy benchmark that's the same size as the game's, and HotSpot's C2 gives me a 30-50% boost after ~10s. It really varies by program (not by problem). This tends to be worse for programs that use a lot of parallelism and aren't configured to run compilation in blocking mode, because then the application threads compete with the compilation threads, but again, there isn't really a good rule of thumb here. In any event, it is simply wrong that C2 would never yield significant performance boosts after a few tenths of a second, even on small programs (it may be true for some specific benchmark entries and maybe even all current ones, it may vary by JDK version, and it definitely varies on whether HotSpot uses Graal or C2). Although, given all the other serious problems that render the comparisons mostly meaningless (different algorithms), my guess would be that this effect is smaller by comparison.
Still, there's a lot of cool stuff the game could do with the current programs (and historical ones), but comparing languages is not really one of them.
Program after Program after Program.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Of course, it's really not possible to show any computer program which is not written in a programming language.
> … basically say that these comparisons mean nothing …
Obviously not: please don't put your words in others' mouths.
> … a lot of data that could be used for some really cool stuff…
Take the data and do that really cool stuff.
https://salsa.debian.org/benchmarksgame-team/benchmarksgame
> … but there's no information about the market aspects.
What do you mean by "market aspects" ?
> …it may be true for some specific benchmark entries and maybe even all current ones…
So it would be wrong to try to dismiss those measurements, based on nothing more than extrapolation about how some other toy benchmark program might perform.
> … cool stuff the game could do …
You're allowed your own personal opinion about what's cool and what isn't (and no doubt others will have their own personal opinion).
It's certainly not obvious to me what they comparisons mean (is it obvious to you?) and the authors of the site don't even state, let alone support, a hypothesis. They just say what the comparisons don't mean. So unless it's a work of art, I think observers can be forgiven for thinking it doesn't mean anything.
> What do you mean by "market aspects"?
I mean a hypothesis about "free market competition" driving people to write really fast programs.
> You're allowed your own personal opinion about what's cool and what isn't
True, but I meant something that could be used in a more rigorous/meaningful way (e.g. comparisons where only one variable changes). Meaning is perhaps also subjective, but I don't even know the subjective opinion of the meaning behind the game, because the site doesn't say.
> https://salsa.debian.org/benchmarksgame-team/benchmarksgame
Cool.
Anyway, now that I've taken a slightly closer look, I realize I spoke too soon when I said that the SBCL vs. Java benchmark is "better than nothing." If, say, Java uses parallelism while SBCL doesn't, then if Java is 4x faster that may not be a "better than nothing" indication.
Program X measured less than Program Y.
> … comparisons where only one variable changes…
Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK).
> I spoke too soon
You chose to summarize data: "Java outperforms SBCL by a very wide margin -- 1.25-4x faster".
Do you mean that you made mistakes when you made your summary.
That's like asking what does a program do and being told, "it does what the code says." Technically true, but not what we mean when we ask. Also, I don't think that's really what the site's authors thought. After all, when I click on the Haskell page, I don't see a comparison of Haskell programs, but a comparison of the fastest Haskell program with the fastest Java program. It's not obvious to me what that means, or even that it means anything at all.
> Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK).
That's exactly right, but because that's rather tedious, I would present just meaningful comparisons rather than non-meaningful ones, so no inter-language comparisons. I'm not saying they're always meaningless, but they can't be made with the data currently available on that site. Unfortunately, we cannot manufacture (non-fictitious) meaning out of data as we like.
> Do you mean that you made mistakes when you made your summary.
Yes. By "better than nothing" I overestimated the significance of results, at least in this case. After a closer look I realized that different programs use different algorithms, and so the comparison tells me nothing about the relative speed of those languages.
No it is not. Program X may not have measured less than Program Y.
> … meaningful comparisons…
Earlier you complained there was "code that's very unidiomatic" but now you say "writing all the Lisp programs as if they were being written in Java" would be exactly right?
1) You seem to be contradicting your own argument.
2) Your desire for "rigorous" comparison is in-conflict with your desire for inter-language comparison.
> … different programs use different algorithms…
For some programs, for some definition of "algorithm".
> … the comparison tells me nothing about the relative speed of those languages.
What is the speed of a programming language?
I may have misinterpreted your comment. By "as if they were written in Java" I meant using a similar algorithm. Alternatively, there could be a rule about idiomatic code (enforced by experts), which would compare very different things, but would also be at least somewhat meaningful.
> For some programs
I don't know. Whenever someone refers me to a result on that site that looks weird, I see very different algorithms.
> for some definition of "algorithm".
For a very well-accepted definition of an algorithm in the context of real performance comparisons.
> What is the speed of a programming language?
I'm not sure (although perhaps I could come up with some definition given enough thought), but I didn't make a website that compares languages based on performance. In fact, I said that the website is totally meaningless. But the people behind it may have something in mind, and if they do, they're not telling us. All I can do is think of ways to use the site's data in ways that can at least be meaningful.
Will someone else look at those similar algorithms and "see very different algorithms"?
> … a result on that site that looks weird, I see very different algorithms.
Do regex libraries for different languages in-fact always implement the same algorithms?
> What is the speed of a programming language? I'm not sure…
You did say "Java outperforms SBCL by a very wide margin -- 1.25-4x faster".
Saying "… so the comparison tells me nothing about the relative speed of those languages" suggests you are looking for something that tells us about "the relative speed" of those languages?
I would think so.
> Do regex libraries for different languages in-fact always implement the same algorithms?
I don't know. But I do know that when something is presented as an answer, it's good to know what the question is.
> suggests you are looking for something that tells us about "the relative speed" of those languages?
I can think of a couple of ways the two can compare. One is the relative speeds (and other metrics) of programs of a similar nature, written simple idiomatic code. This gives a general "feel" for what kind of workloads different languages may be suitable for. Another is comparing the performance of the compiler/interpreter on some microbenchmark of the same algorithms, preferably written by people who coordinate, and preferably without mixing too many variables (e.g. measuring parallel scheduling separately and regex separately). Either one of these is of some interest.
The benchmark game does neither. If there is some idea behind the site's comparisons, it's certainly keeping it a mystery.
So when you say "the relative speed of those languages" you actually mean the relative speed of programs?
> … programs of a similar nature … preferably without mixing too many variables…
Seems vague, no-longer rigorous.
We can make mystery for ourselves — no one can force us to understand.
No, I mean the relative speed of programs with some carefully chosen relation between them. For example, there is a big difference between "relative income of individuals" and "relative income of social groups," even though the latter could be reduced to the former. Whether a comparison of individuals is meaningful as a comparison of groups depends on how the individuals are chosen and aggregated. Clearly not every choice of individuals would make for a meaningful comparison of groups and not every choice of programs makes for a meaningful comparison of languages. The choices made by the game do not seem at all meaningful for the comparison of languages, but if there is some rhyme or reason to the choice, the website doesn't say what it is.
> We can make mystery for ourselves — no one can force us to understand.
True, but when a website chooses to present data in some aggregate way that seems to defy any familiar forms of aggregation and reasoning without so much as stating what the meaning of the presentation is, it is itself mysterious. I agree that it is possible that the comparisons are not worthless, but without any statement of a hypothesis, it is the most reasonable conclusion. I am sure that the authors of the website would agree that if they don't even attempt to say what the meaning of the comparisons is, even in their own minds, it is reasonable to assume it means nothing.
> > So when you say "the relative speed of those languages" you actually mean the relative speed of programs?
> No, I mean the relative speed of programs with some carefully chosen relation between them. For example, there is a big difference between "relative income of individuals" and "relative income of social groups," even though the latter could be reduced to the former. Whether a comparison of individuals is meaningful as a comparison of groups depends on how the individuals are chosen and aggregated.
You can choose which program measurements to compare.
> > We can make mystery for ourselves — no one can force us to understand.
> True, but when a website chooses to present data in some aggregate way that seems to defy any familiar forms of aggregation and reasoning without so much as stating what the meaning of the presentation is, it is itself mysterious.
You have not shown that is what that website does.
I agree that it is possible that the comparisons are not worthless, but without any statement of a hypothesis, it is the most reasonable conclusion.
Yet another tendentious assertion!
Program X measured less than Program Y.
I am sure that the authors of the website would agree that if they don't even attempt to say what the meaning of the comparisons is, even in their own minds, it is reasonable to assume it means nothing.
Once more, you are putting your thoughts and words in others' mouths.
Yes, but the website also chooses, and it is the website's choice that I think is bad. Also, it doesn't seem like the game even has comparable programs in some/many/most cases.
> You have not shown that is what that website does.
How can I show what a website doesn't do? If you think they say what the hypothesis is, show me. I haven't been able to find it.
> Program X measured less than Program Y.
First of all, when something means nothing other than itself, we say it has no meaning. If the comparisons on the website are meant to suggest nothing more than just their own definition, then they are meaningless.
Also -- come on. The website presents comparisons for specific choices of X and Y. What is the meaning of the choice and the resulting comparison? Your position that it's not meant to mean anything (other than itself), sounds disingenuous because the website still makes some very specific choices that seem to hint at a meaning (e.g. why not compare the second fastest programs between languages instead of the fastest?) yet the meaning is never explained.
Anyway, we're going in circles.
For someones definition of "comparable".
> How can I show what a website doesn't do?
You can try to show your claim — "… chooses to present data in some aggregate way that seems to defy any familiar forms of aggregation and reasoning…" — has some basis in reality.
> … when something means nothing other than itself, we say it has no meaning.
That may be what you say. Show why we should think that's what people interested in "Why did Clojure gain so much popularity?" would say.
> … why not compare the second fastest programs between languages instead of the fastest?
Inexplicable! Or obvious: there would need to be two programs for each, rather than one.
> Anyway, we're going in circles.
I agree, you repeat your assertions without establishing anything.
As Rich Hickey says: because it has _reach_
The JVM is widely used.
ClojureScript on JS is because JS also has reach.