Hacker News Reader

Comment by HarHarVeryFunny | Hacker News Reader

This is a rather meaningless comparison, since the differences are going to be dominated by:

1) The choice of libraries/datatypes used for strings and word->count map

2) How the source file is split into words - probably library function again, although in C/C++ one could choose to implement a super-optimized low level version that would blow the others away

IMO a performance comparison between languages is only meaningful if the runtime is not dominated by library functions, or if one admits it's really a standard library performance comparison, not a language comparison.

View on HN

I understand what you're saying. However if you look at it not from the point of the performance figures per se, but rather which language, when written idiomatically, can solve a realistic problem performantly, then I think it makes more sense.

True - there's still value in these types of comparison, although the click-baity "C++ vs Python" title rather detracts from that.

I'm not sure how much value there is in this specific benchmark though since in the real world you'd be using a utility to do this (e.g. linux wc - word count) and it's not obvious what's dominating the runtime (I'd guess reading file and/or splitting into words), so what you're takeaway should be (idiomatic X is good for what, exactly?) if wanting to extrapolate this to some other task.

For that matter, in the real world people don't really choose language on a task specific basis... It's either use language X because the project demands it, or write some limited use utility in a scripting language if performance isn't a major concern.

Agreed. It’s a common desire to boil benchmarks down to a single number.

Some of those languages, however, are far more complex then the others. And as any complex tool, it requires certain methodology. If you are to use heavy machinery like C++, forget iostreams and character-level processing on your hot paths.

Do you have a particular example problem in mind that might demonstrate language performance better? One of the Benchmark Game programs maybe?

What is there to benchmark between C++, C, Rust, or any other low level compiled language though? With any one of them you can essentially achieve the same assembly code generation, if you try hard enough.

In the end it either boils down to compiler optimisations or library implementations.

I think the interesting question there (and really any language comparison) is to compare the result of idiomatic implementations. Unfortunately, there is no clear answer on what is idiomatic, even with something like Python that is more opinionated than most on the question.

When there is "no clear answer on what is idiomatic", we are left with uninteresting "yes it is" / "no it isn't" Dead Parrot assertions.

http://montypython.50webs.com/scripts/Series_1/53.htm

I think the best way forward on the idiom question is to accept that we can't have a competitive/adversarial benchmark suite where idiom is a factor. I think you really need one author, or maybe a group of colleagues who trust each other, to write the whole suite and make their own choices about what idiom means to them. Folks who disagree with those choices can either write an article about how much performance you gain from making what changes, or just produce their own whole suite. Having a very high level comparison like "this is how languages stack up with these idiomatic choices, but this is a different chart with different choices" would be interesting, even though it would take more work to interpret it.

Seems that idiomatic is a matter of personal taste, used to arbitrarily accept or reject.

One way forward would be to ignore claims based on idiom, but measure how long it took to write a program that produced correct output.

I think it's more interesting to try to measure the languages that are designed to be "fast but also garbage collected", like Java and Go.

With C I rarely use libs outside of stdlib and when I do I pay attention to performance more than convenience.

I think even if it's dominated by libs and data types I think it has value as it might reflect what actually happens out there in the wild rather than a purely academic exercise.

exactly, is mostly comparing different C libraries implementations.

My language of choice, LiveCode, is a superset of HyperTalk. The concept of "words" has been built-in (not a library) since the '80s.

Reply on news.ycombinator.com