Benchmark: Java, Python, PHP, C, JavaScript, Swift, Ruby, Perl, Go, Lua, Rust | Hacker News Reader

Benchmark: Java, Python, PHP, C, JavaScript, Swift, Ruby, Perl, Go, Lua, Rust

benchmarksgame.alioth.debian.org

101 pointsfilipmandaric128 comments

Years ago i was naive enough to believe these benchmarks. Now i know these are Potemkin-village level stunts and political maneuvering by language activists and staff of that site. If you really want actual benchmarking you have to do it yourself and read the .asm output for comparison. Another caveat is that some languages are genuinely faster in one hyperoptimized case(even faster than C) but fail horribly when things go more complicated and abstraction creep makes them much slower in most cases.

You provided zero evidence for your claim. I'm not saying you're necessarily wrong, but so far you have only made an assertion.

If you think some language is misrepresented in that benchmark, you can submit your own solution.

The problem is subjective and thus more difficult to provide evidence for.

I don't think the parent doubts the runtimes or is claiming that the code for his pet language wasn't written properly and therefore the site is a sham. The issue is: what is reasonable code?

Benchmarks invariably become a game of massaging the code to produce the optimal assembly. And that's the problem. The reason that we invented higher level languages was so we could stop thinking about assembly. The point of most higher level languages is not to optimize CPU cycles, but to optimize human brain cycles. If we only cared about speed (and not at all about development time), we would just write all code in assembly.

I am not particularly interested in how fast meticulously fine tuned code in a given language runs. I have no doubt that, given enough time, an excellent programmer can finely craft a block of code to run very fast in any half decent language. And this site is evidence of that.

What I want to know, and what benchmarks never show, is which languages make writing performant code simple, idiomatic, and natural. Which languages give me good performance, without having to really work for it?

This is an amazing site, but it is a stretch to think that it compares languages. It compares programmers and their implementations. The implementations are more like their own art form, like Chess or something.

>> I don't think the parent doubts the runtimes or is claiming that the code for his pet language wasn't written properly and therefore the site is a sham. <<

We could always take the parent at their word -- "Now i know these are Potemkin-village level stunts and political maneuvering by language activists and staff of that site.".

Have you looked at the winning submissions code? Every single one I saw (not all of them) was just a straightforward implementation of the algorithm / solution. I don't see much weirdness.

Yes. For example I noticed the typescript one. The implementation is straightforward, yes, but it is just plain javascript. It doesn't need TSC at all. So why call it typescript? If you're going to say it is typescript at least use types, no?

Or the ruby ones. Not much like ruby I have ever seen. It's not particularly difficult to write C code that can be easily called from ruby which is why I guess most ruby programmers wouldn't bother to write code like this in ruby. If you want C, just write C!

I think all the implementations are straightforward as implementations. But few are the most idiomatic way to express the alogorithm in that language.

Specifically which "typescript one"? (Currently 15 typescript programs are shown.)

Specifically which "ruby ones"? (Currently 37 programs are shown.)

>> the most idiomatic way <<

One person's idiomatic is another person's idiotic.

On how the benchmarks should be interpreted http://benchmarksgame.alioth.debian.org/for-programming-lang...

http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus...

I had the same observation. I'm a Perl fan, but I can see some submissions were solely motivated to make Perl appear competitive. Like using threads in Perl, which isn't often done in real life.

I'm not sure how much evidence is needed. It seems obvious that people who are fans of language X are going to submit code with the sole intention of bumping up performance at any cost.

The site is still interesting, but probably useless for real world decision making.

Doesn't matter, if you don't use threads in Perl when you could, that's your fault and benchmarks like this serve to remind you should. There are folks who use threads with their Perl code in real life btw

It's not even enabled in several major distributions. Yes some people do use them. Most don't...it was broken for several releases.

I don't find the .asm output to be meaningful at all. The realistic comparison for me is if I write a solution the easiest/most convenient way in each language, which language is naturally more efficient.

>> Now i know these are Potemkin-village level stunts and political maneuvering by language activists and staff of that site. <<

That's just name-calling.

How do you read an asm output for PHP, python, erlang, Lua, Java, JavaScript?

For Node.js http://www.mattzeunert.com/2015/08/19/viewing-assembly-code-...

Via the +PrintAssembly switch

https://wiki.openjdk.java.net/display/HotSpot/PrintAssembly

Hey, the benchmark game got a new website (at some point in the several years since I looked at it last)! Almost simplistic, but nice to read.

As someone that works predominantly in a language long considered passé (Perl), I always kinda relish seeing that it is still faster than Python and Ruby for many problems. As Python and Ruby have gotten faster, I'd sort of assumed they were both notably and clearly faster than Perl, by now, but that's not the case though the difference is now quite small and probably debatable.

Python was never really designed for speed. It was designed for simplicity and clarity of behaviour. The beauty of the age we live in is that we have such a broad array of tools to apply to different problems. Performance is often not the problem that needs solving, but when it is, you've got many options too!

I'm not disparaging Python (or Ruby). Merely taking small pleasures where I find them. I work in Perl because of a huge array of existing code in Perl, not because I hate Python or Ruby (I've built stuff in Python and Ruby, too, and like them both).

Perl is only faster here in nbody because I tricked it. I used a tricky compile-time optimization for faster array index access. You wouldn't do that in the real world, and in the real world PHP, ruby and python are faster. cperl instead is only beaten by PHP. But the benchmark game is not open for faster alternate engines like luajit, nim, crystal, pypy, truffle-graal, ... or even slower ones like Perl6, which limits the impact of those better engines.

Decades ago it was open and written in Perl. Then it was rewritten in Python and closed down. Go figure.

>> a tricky compile-time optimization for faster array index access. <<

And you even comment on that usage in your code.

>> Decades ago it was open and written in Perl. Then it was rewritten in Python and closed down. Go figure. <<

At best your comment is disingenuous.

You can download the Python measurement scripts, use them with luajit, nim, crystal, pypy, truffle-graal or Perl6 programs and publish the measurements.

You don't. Go figure.

Perhaps this is merely a reflection of the difficulty of finding where to download the measurement scripts? I found them (here: http://benchmarksgame.alioth.debian.org/play.html ) but, it took some clicking. Maybe a github link on the front page, or similar, would alleviate some of the frustration the previous poster had about it.

I assumed the sources were available, somewhere, but until I went looking for it, I wouldn't have been able to say so with confidence. That's not an obligation, of course...volunteers should feel free to do whatever they want with their projects. But, if you wanted to keep the community involvement the game had historically, linking source in the lingua franca of the day (github or gitlab, or whatever, seems pretty standard today) would go a long way.

Perhaps, so I just googled "download benchmarks game scripts" -- 4th URL on first page of results.

Perhaps it's merely a reflection of wanting some one else to do work we choose not to do.

(You'll find that there already are duplicates of the benchmarks game repos on github).

"Perhaps it's merely a reflection of wanting some one else to do work we choose not to do."

Entirely valid position to take. There's only so many hours in a day, so many days in a year, and so many years in a life.

rurban (and frik and wulfklaue) do not seem to feel that it is a valid position for me to take.

Yeah, they're wrong. Offering polite suggestions is one thing, demanding volunteers do things in some specific way is entirely another.

Do it the way you want to do it. Unless someone is paying you or you want their specific help with something going forward, I don't see any obligation on your part to listen to demands.

If course I have the benchmarks, because that's how everybody benchmarks it. But rather with the old perl scripts, not with your new python scripts. They suck. Before it was trivial to bench, but not trivial to setup all the environments.

The problem is not that someone else can do it, the problem is YOU need to do it. You got the name reserved for it, you got the machines to do it, you decide to rewrite it and throw them out, no one is looking somewhere else for better and more benchmarks.

The big competitors with better and faster engines have no incentive to match performance when their name is not on the list, and they just publish their own results. Users don't look somewhere else.

>> Before it was trivial to bench, but not trivial to setup all the environments. <<

That's what it's like now -- trivial but time consuming.

There were pain-points with the old perl scripts (and nested make files) but I would have continued to put-up with the problems and would have continued to use the old perl scripts.

But (back in 2008) I couldn't see how to set-affinity with Perl and I could see how to do that with Python.

That's why I rewrote the measurement scripts in Python.

>> They suck. <<

https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writ...

>> You got the name reserved for it <<

Wow! Think-up a name!

>> you got the machines to do it <<

I have one too-old machine.

>> no one is looking somewhere else for better and more benchmarks <<

55% of page views are from "organic search".

Dude. What's up with attacking someone who does things voluntarily (I presume) for not doing them the way you want them to? That's pretty damned presumptuous.

Open Source stuff really only ever has one guarantee: If it breaks, you get to keep the pieces.

You can put it back together any way you want, but you can't demand they give you a new one that meets your expectations. Unless you want to pay them to do so, and they're willing to accept the money and terms.

>> new website <<

Dec 2015. Pleased you find it nice to read.

Are you the maintainer? Thanks for keeping it running all these years, if so.

I don't go there often, but it's nice to know there's a way to get a vague understanding of relative performance for languages for those occasions when one needs to know "is it fast enough?"

>> Thanks for keeping it running all these years <<

Thanks to all those who contribute their expertise and their programs.

So PHP is faster on most of those test than python or ruby ? I'm I missing something ? I thought everyone hating PHP ( also) because it was slow.

Nobody that I'm aware of has ever hated PHP because it was slow. They hated it for a variety of other reasons.

Many of them covered in: https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

Just like JavaScript, PHP used to be slow. Then, large amounts of effort and/or money were spent into making them not so.

My somewhat glib comment was meant to point out that even had PHP been very fast back when PHP hate was a popular past time it would still have been hated for other reasons.

Ruby was (very) slow back then (notably and measurably slower than PHP, Python, or Perl) and people still loved it. The warts on PHP that got the most bad press were rarely related to performance.

So, yes PHP used to be slower than it is today, by a lot. But, speed wouldn't have immunized it against the criticism leveled against it. All the effort going into fixing performance for PHP coincided with fixing some of the bigger warts of the language, as well...so much of the hate for PHP has dwindled in recent years.

The fact that people building for the web are much less often required to work in PHP also reduces the hate.

It was low performance, but not slow, its simple model meant it was performing enough for the task (DB IO was the bottleneck most of the time).

Since PHP5, the lead team made progress into making it solid. And in PHP7 some eastern/russian guy hacked the VM a lot. Adding new traits and improving memory usage and performance a lot too.

The "hatred" comes from old days (v3,v4):

  - massive inconsistency in naming
  - global side effect was a norm
  - bugs, bugs, bugs

Combined with the nascent web boom, tons of new kids (that includes me) started having fun with html/php. A large mass of the web was written with it, carrying all the bad design decisions in forms of security bugs and impossibly bad code.

ps: I audited one famous wordpress plugin and it was 10 pages long of redundant code in 3 big loops, dead code everywhere, commented code released in production. All this to add a few links here and there.

It's not necessarily a performance issue, unless you're Facebook (traffic wise) you might never worry about that, but it's more about the developers who use PHP[0] it's a mixed bag of below junior level to senior level developers. It's hard to know which type of codebase you will get your hands on until you do. There's other issues I personally have with it, though I use it if it's not code I can easily reinvent without investing way too much of my own time on it e.g. a forum or a blog engine.

[0]: http://thedailywtf.com/articles/switched-over

"unless you're Facebook (traffic wise) you might never worry about that"

Facebook hasn't spent a lot of time worrying about their choice of PHP either. Facebook's primary language is Hack, a mostly compatible derivative of PHP. Prior to Hack (~2014) Facebook used traditional PHP, which they reimplemented a couple times for performance reasons. It seems that Facebook has spent its time worrying about how NOT to leave PHP.

Makes one wonder about the value of this daily new programming language era we're in at the moment.

PHP 7 is extremely fast.

There was some legitimate criticism to be had about PHP 5.x being slow, especially if it wasn't paired with an opcode cache system / accelerator.

PHP is fast, faster than Node.js even.

But in practice, the Node.js ecosystem is all built around evented I/O and PHP has to execute them one at a time.

(Well, actually with libraries like Kraken and ReactPHP and amp, you can have an event loop, but there are not that many I/O options for you.)

> I thought everyone hating PHP ( also) because it was slow

Most of the replies here are stating that PHP is fast. It can be for these kind of benchmarks.

But, it does have the reputation you're suggesting. That reputation, though, is based on the old CGI model. Most php apps are built in a way that all of the initiation and setup code is run for every single request. That makes it comparatively slow.

Some PHP apps get around that by using a shared memory cache, like apcu, to box around that. That helps a lot, but isn't the same as a typical golang or node.js app where startup code is truly run once, and new requests go through a much shorter path.

All of that, though, is less a "language" thing and more of a runtime thing. You can, for example, use Facebook's HHVM+Proxygen and run PHP code in the exact same way as node.js.

php7 was threatened by hhvm and was completely rewritten by keeping backcompat. It's now the fastest major old scripting language. js and luajit are in the very fast ballpark, php in the middle and the other 3 far behind.

PHP also has the best language features and IDE integrations, the only remaining problem is the legacy stdlib and devbase. Ruby with an inliner and method cache would catch up, but I don't see it happen. Python is doomed by its architecture. Perl has a faster interpreter loop, but data structure bloat and similar dev problems as the other 2.

Pypy is considerably faster than CPython, so I don't think Python is doomed to be slower?

pypy changed not only the vm, but also the internal data structures. It's totally different. pypy's problem is that is only optimizes loops and often called functions, not the general interpretation, as e.g. PHP7 or lua did. For normal, simple small scripts it's not that big a benefit, and you loose all the extensions, (though that's being worked on).

That has been less true for the last few years: C extensions are still a concern, although many of the major ones work now, but I've had a significant number of programs run faster simply by using pypy. It's a full JIT compiler and one nice area that improves is regex handling so all of my log-related work got faster with zero effort.

Speed is not the problem. Amongst many other issues, php has a terrible security track record compared to other languages VMs/compilers.

The languages with the most security issues, by far, are C and C++.

No PHP is hated because it's too easy to use.

The means that a lot of rather bad (inexperienced?) programmers use it, which means there is a lot of bad PHP code out there.

People see that bad code and assume the language is bad.

The language has some warts and inconsistencies, and people are eager to point them out as justification for their hate. But for real programming most of them hardly matter.

this is something a bit naive - basically all dynamic languages have the same low barrier and can produce the same crappy code, see for example javascript or ruby (with RoR). Python too, considering the fact that it's used by a lot of people that their main job is not software engineering.

The biggest difference between PHP and the other dynamic languages, though, is that PHP came out with pretty popular stuff like wordpress, joomla, magento or drupal that they all share crappy code coz they all have a large codebase started before PHP managed to fix some stuff starting from version 5.3

Python is as easy to write as PHP, but deploying it as a web app is a whole different story.

That comparison is complicated: it's definitely easy to drop a .php file on a server but once you start talking about frameworks, using a newer version than your OS distribution shipped, adding C extensions, etc. it's easy to find counter-examples, and the Go proponents are not wrong to observe that deploying a single file is easier than either.

This is also becoming less of a distinguishing factor in the container era where the answer for a number of cases is “Extend a base Docker image”.

At least PHP offers that option at all, and it's well-supported and well-documented. It's also very easy. Good luck doing the same with any other language.

If you really, really try it's possible in Python, C# and Java too, but ... wtf.

Note that I didn't say it wasn't a selling point for PHP. It's just that I can count on no fingers large PHP projects I've seen where deployment is still that simple – and that's generally not a problem because with modern tooling (e.g. Ansible) and deployment options (e.g. containers) you're under less pressure to make that a deciding factor for your application.

No, PHP is actually really fast. Especially HHVM[1] which is crazy fast.

[1] - http://hhvm.com

Ever since PHP 7, there's not much difference anymore.

HHVM is not PHP. The sheer number of bugs and unimplemented features means it might as well be another language.

Today.

Compare it to Nodejs and Go.

It's slow to develop with, which is a much more costly problem.

As slow as it might be to develop something from scratch in c++, perhaps. But, if you're using a framework and/or good libraries, all of that underlying work and ecosystem is standardized for you. Now, the ecosystem with things like Symfony are really great.

This comment is so patently false that I don't even know how to express strongly enough how polar opposite of true it is.

Any benchmark that includes a JVM language always makes me look for a disclaimer that they allowed for proper JIT/warming time by running the benchmark a few times in the same VM instance before collecting the numbers. I may have missed it in this article, but I didn't see it.

Yes, startup time is a concern with the JVM, but that's not what it's built for - it's meant for long-runnning repeatable code paths.

> that's not what it's built for

There was nothing in the original design that said "only long running server processes". In fact, Java was originally aimed at set-top boxes, then web applets, then cross platform GUI apps. It was only after it failed in all of these that the server was discovered, despite the fact that "write once, run anywhere" is almost completely pointless when you control the hardware it runs on.

The long startup times are a limitation of the implementation technology, not a principled design choice. See also: Jitterdämmerung ( http://blog.metaobject.com/2015/10/jitterdammerung.html )

Any comments that "startup time is a concern with the JVM" always make me look for evidence that the person has made some attempt to investigate startup time for the specific circumstances they discuss.

http://benchmarksgame.alioth.debian.org/sometimes-people-jus...

He doesn't do warmups for JITted languages.

...which means the measurements are meaningless. They will include the time to analyse the hot paths and then compile them.

They will include a few tenths of a second.

Note the timings are seconds and tens-of-seconds.

A program written in C will execute faster if you run it multiple times, and slower if it has not been run recently. Presumably all of the benchmarks consist of many thousands of repetitions of the same process.

The difference is not as dramatic as with the JVM.

The difference is not as dramatic with tiny tiny programs that run for more than a few tenths of a second.

Oh, yes it is. I've benchmarked code I've written in C++ vs Java before and allowing hotspot to optimise the code makes a huge difference

How "huge" on a 20 second run of nbody 50000000 ?

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

The measurement shown is 21.54 secs.

The average JMH shows is 23.367 ± 0.062.

I see that the point of this site is supposed to be that a change in implementation (keeping the language itself held constant) greater affect on performance than rewriting in another language.

My problem with this site it makes poor support of that claim. The data is hidden behind loads of links, and none of the data is visualized at all. Even a few simple bar charts, keeping the hierarchy of the site otherwise unchanged would vastly aid comprehension. The points being made would come across much more easily.

This sort of data is the poster child for visualization, yet this site relegates the data to tables of numbers.

What on the website tells you that is the point?

>> Even a few simple bar charts <<

http://benchmarksgame.alioth.debian.org/u64q/which-programs-...

>> would vastly aid comprehension. <<

Is it possible that you are just wrong?

Is it possible that simple bar charts do not aid comprehension but encourage thoughtless instant conclusions?

The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory usage and performance on every test except regex-redux (edge case?).

http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...

> The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory usage and performance on every test except regex-redux (edge case?).

Why would it be stunning that a programming language compiled to assembler can beat a broken-by-design interpreted language which doesn't even have integers or arrays? The stunning part is that the people behind node.js have managed to get that much performance.

That's not a very fair or accurate description. JavaScript implementations use a JIT which is different than what people usually mean by interpreted. Types can be inferred without explicit type declarations. JavaScript has Array and integer semantics can also be inferred.

Also JS has received many times the investment of Go.

Ummmm, that's a somewhat harsh and arrogant reply. I actually enjoy coding in Node.js myself. I just wasn't aware how much faster go is (according to these benchmarks of course).

I program in Go and JavaScript, but don't enjoy it.

I assure you that JavaScript is an interpreted language, and it does not have arrays or integers. This is not a harsh statement but a factual one.

The JS language as specified is interpreted and doesn't have ints or first-class arrays; the code run by the VM that runs Node.js code is compiled and runs on smallints and arrays and so on.

(Not sure if you already knew this, but if so, you're not being factual, you're being selectively pedantic to make something sound worse than it is. Lack of ints or arrays isn't a big factor in JS performance for non-pathological cases.)

(edit: reasons for downvotes are appreciated!)

There's plenty to not like about JS, but being interpreted and not having integers or arrays is not high on its negatives list. The little bit of good far outweighs the whole lot of bad.

If you don't enjoy JS, it's because you haven't taken the time to learn what's great about it and have gotten hung up on the stupid things other people do with it or you're copying their mistakes.

> If you don't enjoy JS, it's because you haven't taken the time to learn what's great about it and have gotten hung up on the stupid things other people do with it or you're copying their mistakes.

oh come on now, it's perfectly ok for people to dislike a language for any number of reasons. I don't hate JS but I don't particularly enjoy it, and that could be said for a number of popular languages. it has nothing to do with "taking the time" to learn anything - some people just don't jive with JS and that's totally cool.

That last paragraph is a logical fallacy, because it posits that one's distaste for JavaScript could only come via ignorance. I will offer my own anecdotal experience to refute that assertion. I've been forced to use JavaScript since the 1990s, and have done significant amounts of work in Node for several companies. I dislike Node and JavaScript because JavaScript has a weak standard library, ugly syntax, is slow, is a pain to scale, and a pain to write complex code with. I will absolutely give credit to the Node community for improving JavaScript, in both speed and syntax, and admit that JavaScript is better now than it ever has been; however, so are most popular languages. Personally, I've removed JavaScript from my resume, and turn down jobs that use it. If I have to use it for some reason, I can, but right now, I typically avoid it altogether by using Elm, and real server-side languages on the backend. I can't speak for anyone else, but I know JavaScript and still don't like it.

I don't know, I rank not having integers pretty high on the list of faults. Wasn't there just a huge debacle with node being unable to represent inodes properly?

The inodes thing was about lack of 64-bit ints specifically. As I understand it, it was a bug where Node converted inode values to floats (instead of using strings or pairs of 32-bit ints or whatever). One could argue that the bug wouldn't have happened if JS had 64-bit ints, but anyway it's not a foundational thing.

The performance implication of JS not having ints is a different matter. In practice, if you write code that treats a variable like an int, modern JS engines will correctly compile that into bytecode where it really is an int, and performance will be as expected.

So the lack of an int type doesn't really hurt performance, though one can argue that it makes it harder to write performant code.

We're discussing programming language benchmarks. If one was forced to write all of a C or C++ or even Go program using only doubles, and not integers, and only hash tables, not arrays, undoubtedly it would become slower. The impressive thing about Node.js is that it is as fast as it is despite JavaScript not having integers or arrays.

No, we're not discussing language benchmarks anymore. The parent I replied to claimed that not having integers is not a big deal -- I disagree. It's a huge deal.

They go on to say:

> If you don't enjoy JS, it's because you haven't taken the time to learn what's great about it and have gotten hung up on the stupid things other people do with it or you're copying their mistakes.

Which is... wrong. Many, many programmers dislike JavaScript and node.js because of its flaws. I don't think Stockholm syndrome is an appropriate prescription.

I think you're in the right here, but that's a little too far. I don't enjoy programming in JS, because I really like strong typing. And that's ok. You seem to be implying that everyone who doesn't like JS is deluded or arrogant, but that's not the case.

> There's plenty to not like about JS, but being interpreted and not having integers or arrays is not high on its negatives list.

We are having a discussion is about programming language benchmarks.

A benchmark in this case is a measure of how fast a computer program can run. A computer program is a series of instructions for a computer. These instructions are written in something called a programming language. A programming language which is interpreted is inherently slower than a programming language which is compiled to assembly language. A programming language which does not have integers but must represent all numbers as floating point numbers is inherently slower than a programming language which has integers. A programming language which does not have arrays, but must represent arrays using hash tables, is inherently slower than a language which does have arrays. Thus, Node.js, which is interpreted and does not have arrays or integers, would be expected to be have much lower benchmarks than Go's, and yet node.js has exceptionally high performance.

JavaScript is compiled to assembly language in modern runtimes like v8. You are confusing a language with the language implementation.

For example --

http://benchmarksgame.alioth.debian.org/u32/compare.php?lang...

that's a very simplistic exploration of a very complex subject

(edited for negativity and ad-hominem)

>regex-redux (edge case?)

It looks like the JS version is using the built in runtime's regex engine.

I don't know exactly why (maybe differences or missing features in go's regex) but the go version is actually using the c FFI to call pcre.h, and without profiling it I'd guess a huge hit is that alone.

> I don't know exactly why … and without profiling it I'd guess a huge hit is that alone.

Don't guess. Look at the measurements. Look at the source code. That Go PCRE program is faster than --

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

Maybe faster still to use some of the techniques from this other Go PCRE program --

http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...

I think Go's regex lib is native Go, but simply naive. In any case, this particular benchmark isn't useful unless your application is regex as a service.

Ruby's fastest time for pidigits was 3.14, haha.

With this new revision this great benchmark got less useful.

Why can't I compare:

  Lua vs LuaGIT

  PHP5 vs PHP7 vs HHVM vs Hack

  Perl5 vs Perl6

  GNU GCC vs Intel C vs CLANG

This new rule "one implementation per language" is so stupid!!!

@debian: change it back

"If you're interested in something not shown on the benchmarks game website then please take the program source code and the measurement scripts and publish your own measurements."

Does anybody know why Go binary-trees is so much slower than Java's?

Binary-trees is allocation bottlenecked, and Java has bump pointer allocation where Go has slow allocation. The root cause being that Go's GC is non-moving so it can't do bump pointer allocation without fragmentation, so it has to do a whole bunch of computation to find a place for each node.

I see, thanks!

Wow, I expected Erlang to score better, considering the claims of Elixir community how fast it is... compared to Ruby. OTOH maybe that tells more about Ruby.

Erlang makes no claims about being fast in these sorts of tests. They aren't using what it was designed to do.

Also some languages like, Ruby, do really well on some of these tests because the C-code underneath has been written well (example regexes are just as fast under ruby as they are anywhere else because the underlying C-library kicks ass though I forget the name of it).

You're probably thinking of PCRE, although Ruby actually uses Onigmo: https://github.com/k-takata/Onigmo

Erlang's claim to fame at least for me is parallelism. When you scale out to thousands upon thousands of clients. Think about the simple fact it was using all cores on a processor out of the box by design that a computer had available. There's also the whole zero downtime aspect. In Erlang you should be able to design software (again downtime from the software side, not much you can do about hardware failure or server maintenance updates to your OS) that if it needs an update it can update itself without killing off anyone connected to it. There's also it's functional aspects.

Erlang isn't built for computational throughput. It's built for concurrency and fault-tolerance.

This is why benchmarks really don't mean much. They are a myopic view of the complex nature of language selection. For example, I would never recommend Java for a bulletproof app that had to scale, because OTP is so much better at scalability, fault tolerance, live debugging, and hot code deployments. At the same time, I would never opt for Elixir for doing screen-scraping, because it is slow at it. Sometimes, developer productivity is more important than speed and scalability. It is important to know the strengths and weaknesses of various languages, and the most important aspects of your project, in order to pick the right tool for the job, I think. Performance benchmarks are but a small part of the selection process.

where's D-lang? it would be interesting to compare D with C, C++, and Rust

Good luck on seeing D on that site. There seems to be a active war between the sites maintainer and D. There used to be D benchmarks on the old site and people tries to release new code to the site, but for some reason the current maintainer has a issue with D.

And he thinks its no unfair that D is not resented when a lot of people look at his site and think: "If your not here as a language, then the language must be bad".

Good luck on changing his mindset because i have seen topics going back years regarding D not being there.

If you want a more impartial benchmark set:

https://github.com/kostya/benchmarks

And where is D on those tests ;)

> There seems to be a active war between the sites maintainer and D.

No.

> There used to be D benchmarks on the old site

Back in early 2008.

http://web.archive.org/web/20080730204710/http://shootout.al...

Look how many other language implementations were also shown back then and have not been shown since.

> … for some reason the current maintainer has a issue with D

No.

> Good luck on changing his mindset…

Make the measurements you want to see and publish them for everyone else to see.

I readed python was slow, but i had no idea it was relativly That slow!

they should have included luajit it's almost a dropin replacement for most projects

It was included in the past [0]. It was removed to simplify the process and due to a "one implementation per language" rule.

[0]: http://i.imgur.com/0xxOfVm.png

No Scala anymore?

No success getting programs updated for newer versions -- so, no Scala anymore, done.

No Lisp? :-(

They used to have Lisp & Clojure. No idea why it's been removed. Some is still accessible:

http://benchmarksgame.alioth.debian.org/u32/lisp.html

http://benchmarksgame.alioth.debian.org/u32/clojure.html

>> No idea why it's been removed. <<

Lisp hasn't been removed.

(Clojure has. So's Scala.)

As if Lisp needed more exposure..

A CL Common Lisp and Racket are included.

Go to the home page

http://benchmarksgame.alioth.debian.org/

and click "Lisp".

Thanks.

If you're a competent programmer, one should be able to make any real-world task take about the same time in any two languages.

Therefore, any large-margin differences are benchmarking the programmer not the technology.

> one should be able to make any real-world task take about the same time in any two languages.

I don't think that's a practical claim. Interpreted or JIT compiled languages depend on the interpreter or VM. JavaScript has become much faster over time as JavaScript VMs have improved, independent of the details of any particular JavaScript program. Ruby has become faster over time for the same reason. I've seen Ruby programs I've written speed up significantly without any significant code changes when I moved from an older to a newer version of Ruby.

Reply on news.ycombinator.com