Hacker News Reader

Comment by Mithaldu | Hacker News Reader

In fact, i know i do not understand how the table on this page is generated:

http://benchmarksgame.alioth.debian.org/u32/which-programs-a...

That is precisely why i am suspicious. I cannot say for a fact that it is deceptive, but it certainly seems deceptive.

On that page Ruby is being shown as 10% faster than Perl. Yet on the direct comparison page things look quite different:

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...

On that page, for all benchmarks that can be compared, Perl has used an overall time of 9255 seconds, while Ruby has used an overall time of 10662 seconds. As such Ruby is actually 10% slower than Perl.

Where does this difference come from?

View on HN

>> but it certainly seems deceptive <<

You go too far -- your lack of understanding is simply your lack of understanding ;)

What are you told the table shows?

>> Where does this difference come from? <<

Check the same thing for 2 other language implementations were the arithmetic should be easy. For example, Java median 2.04 and Smalltalk median 21.22 -- the direct comparison shows 11x as the rounded median of the Smalltalk/Java program times.

So, basically: Because Perl is considerably slower in one single comparison, even though it is faster in 7 others, it gets judged as slower overall?

Seems like your graphs up top in the language versus language comparison need to be reworked to make it clear how bit the difference in reality between x3 and 1/3 is, because right now it is deceptive.

>> Because Perl is considerably slower in one single comparison <<

What are you looking at? Perl is shown slower on 3 tasks.

Note the word "considerably". There is only one single task in which perl takes a considerable amount of time longer than Ruby.

PS: The source doesn't show here: http://benchmarksgame.alioth.debian.org/u32/program.php?test...

Another addendum, i'm not sure i'm getting what's happening here:

If i calculate the average of the time perl took divided by the time ruby took, i get this:

((226/724)+(5.35/16.8)+(3/9)+(2750/3960)+(1120/1368)+(3236/3837)+(30.5/35.8)+(939/618)+(263/135)+(662/214))/10 = 1.07

Which i understand to mean that perl on average took 7% longer.

-----

However if i turn this around i get:

((724/226)+(16.8/5.35)+(9/3)+(3960/2750)+(1368/1120)+(3837/3236)+(35.8/30.5)+(618/939)+(135/263)+(214/662))/10 = 1.58

Which i interpret to mean that Ruby took, on average, 58% longer.

------

These things contradict and i made some mistake here. Can you clear up what i should've been doing?

>> i made some mistake here <<

The table you don't understand shows the median but you seem to be calculating the arithmetic mean.

Ok, i don't get it. Can you show me the formula i should be using?

Or even better: Try to explain in detail why on the overview page perl is claimed to be slower than Ruby, when in a direct comparison it is not.

You don't know how to calculate the median?

http://www.robertniles.com/stats/median.shtml

English is not my primary languages and especially english maths are hard for me to grasp. That's why i am asking you to demonstrate, using the actual numbers for Ruby and Perl, what calculations should be performed to gain the numbers your site is showing.

Also, in addition, after reading your link, the situation seems even worse. Using the median Perl outperforms Ruby by ~15%, but the main site does not reflect that at all.

Reply on news.ycombinator.com