No single benchmark for the web | Hacker News Reader

No single benchmark for the web

hacks.mozilla.org

67 pointsdherman11 comments

Firefox Aurora 16a2 is blazing fast on my computer to the point where I don't really care about the browser speed war anymore. Congratulations to the Firefox developers who completely transformed the browser from where it was in 2009.

Indeed, it's a shame many devs have been lured over to Chrome in the past (myself included), while every day it becomes more aligned with Google's corporate interests.

Let's not forget the good Mozilla has done in making a free and open web. Also, Fantastic Mr Firefox recently becoming a worthy competitor on technical merit alone, is a truly inspiring milestone.

I'd also be skeptical of a benchmark created by a company that has a pretty big dog in the fight.

Yes, and no.

As OP points out, in order to optimize, there must be something to measure. This is Google saying the benchmark is what they'll be using as a measuring stick internally (and likely, in their marketing against competing browsers).

Does all of this make the benchmark a useful tool for Google? Yes.

Is it the best benchmark for cross-browser comparison? Probably not.

Isn't the blogger the same guy who also started Emscripten as a project?

The benchmark for the modern web is what feels fastest in every day use. Chrome.

Ever compare Chrome to Firefox on a netbook or low-powered PC, especially with many tabs open? That seems to be where Firefox shines for me at least. I love them equally but for different uses.

Looks like Mozilla tries to compensate for the slowness of Firefox compared to Chrome each time a new JavaScript benchmark is released.

Have you read the blog post? If so what don't you agree with?

Mandreel is produced by a third party with no ties to any particular browser. Emscripten is mozilla's own project. So rather than google's benchmark being biased in favour of chrome, it seems more likely the ones he compares to are biased in favour of firefox.

To be fair, Mandreel has done a lot of work with Google, and you can see practically all their games are on the Chrome Web Store and made for it. So unsurprisingly Google has optimized for them. There is nothing wrong about that, of course, just pointing out that they are not "a third party with no ties to anyone".

Second, while Emscripten is a Mozilla project, you can see that it works with Chrome as well: Links to bugs filed by an Emscripten dev (me) appear in that blogpost, in fact - those bugs are filed to help Chrome be faster, and nice progress has been accomplished, one of those bugs was just closed in fact (it's cool stuff, read it for more info - Chrome was much slower than Firefox on that benchmark, and now it's faster on it).

Finally, the point of the blogpost is that all benchmarks need to be treated skeptically - there is no such thing as a perfectly unbiased benchmark. So we need to just be aware of that, and not consider any specific benchmark to be "the" benchmark of the modern web. That's all.

Reply on news.ycombinator.com