Hacker News Reader

Comment by kevingadd | Hacker News Reader

jsperf test cases like this don't produce valid results.

The amount of work done in the test case is very small (actually dwarfed by the setup part of the test), making it subject to huge variance due to factors like task switching, GC pauses, etc.

The result of the benchmark is also never used so it is possible for a valid JS runtime to partially or completely optimize it out.

Benchmark actual code doing actual work. Code reduced to this level of meaninglessness does not allow you to draw meaningful conclusions about actual JS performance.

View on HN

What ways of testing performance do you recommend? I need to test different regexes and string manipulation. I was about to use jspref but I am very open to any suggestions,

It's not valid to see that do while is tenfold slower than while, across the board? Or the total lack of native forEach performance?

These are the sort of results that I'd think would be good to keep in the back of your mind, and little else.

'while' is actually faster than 'for'. But in this example the while loop does extra work (mutating the array). while loops are always faster than forEach, as they simply do less work.

There is an example that doesn't mutate the while. For still seems to be faster in certain browsers (Safari.) I think the bigger takeaway here for me is that mutation has little to no performance hit. In fact, the while shift() example was faster than the non mutating while version.

> jsperf test cases like this don't produce valid results.

I once went to a meetup presentation where the presenter was a supposed JS performance expert. His presentation was based on JS perf results, and Chrome was consistently showing 749.8M ops per second for complex side-effect free js code.

The only thing I learned from the presentation was that Chrome is good optimizing out code that does nothing, and the presenter's laptop ran at 2.25 Ghz.

Generally agree with you (benchmarking isn't what it used to be ;-).

However the following examples all do nothing (except increasing i) and get different results.

http://jsperf.com/codethatdoesnothing/2 http://jsperf.com/functionwithwhilethatdoesnothing

They don't increase i at all. It is impossible to count to 10,000 at a rate of 200Mhz. That implies a 2000Ghz processor.

JS Perf is fundamentally broken. It doesn't force a side effect, and from the debugger, the first step it does is something like:

    while(100,000,000--) {
      // your code here
    }

The chrome compiler then recognizes the loops can be swapped, and eliminated, thus it runs your code once, yet jsperf thinks it ran it hundreds of millions of times.

Reply on news.ycombinator.com