The HTML5 test | Hacker News Reader

The HTML5 test

html5test.com

56 pointsdmn00134 comments

Scores are as follows, all latest stable releases: (edit: incorporating other responses)

    OSX stable builds:
    Chrome:   288  +13
    Opera:    258  +7
    FireFox:  255  +9 
    Safari:   228  +7

    OSX dev builds:

    Windows 7 stable builds:
    Opera:    258  +7
    FifeFox:  240  +9
    IE9:      130  +5
    
    Windows 7 dev builds:
    IE10 PP:  130  +5

    Linux stable builds:
    Chrome:   273  +13
    FF 3.6:   155  +4

    Linux dev builds:

I'm surprised that firefox scored as (relatively) poorly as it did, although I suspect lack of support for closed formats had something to do with that. Similarly, I'm a bit surprised at Safari's poor showing given Apple's insistence that "HTML5 is the Future".

I'd be interested to see similar tests on Windows/Linux, with ie9 in there. Also, it'd be interesting to see how the beta/dev builds rate.

Don't have Opera and Safari installed on this machine but these are the results on Windows 7 32-bit:

  FF 4.0.1:   240 + 9 bonus points
  IE9:        130 + 5 bonus points
  IE10 PP:    130 + 5 bonus points

Opera 11.10 on Windows 7 scores 258 + 7 bonus points

Interesting that FF scored worse on Windows than OSX. I can't say I'm surprised by IE's performance, however.

The 15 point gap is because Athtar doesn't have up-to-date graphics drivers for WebGL. It's otherwise the same.

Chrome 10.0.648.127 on Linux: 273 + 13.

Firefox 3.6.11 on Linux: 155 + 4. (Ouch...)

(By "Linux", I mean 64-bit Crunchbang 9.04.)

The current production version of Firefox 4 was released about a month ago and it's approaching 90 million downloads.

http://glow.mozilla.com

You are using old Firefox numbers. A lot has happened in the year since Firefox 3.6 was released.

Agreed. I get 240 + 9 bonus points on Firefox 4.0 on Ubuntu.

I know. I just included it because I happened to have it installed (it actually came by default). My main browser is Chrome.

Chrome 12.0.725.0 Ubuntu 10.10: 293 + 13

FWIW, WebKit nightly (OSX): 271 +9

edit: Can anyone familiar with Chrome development shed some light on how it is "ahead" of the WebKit nightly? Regressions? Is there rendering code in Chrome that doesn't get pushed back to WebKit?

No need to post your scores, just check this page: http://html5test.com/results.html

For me, the most surprising results in that table were the ones for mobile browsers. Blackberry and Opera Mobile scored higher than all Android and Apple devices listed.

The Firefox Mobile (Android) scores a necessary point (235+9 bonus) to be the leader.

Thanks. Serves me right for doing work before checking to see if anyone else had already done it.

If you're a web developer, the site http://www.caniuse.com/ breaks down exactly which browser support which "HTML5" technologies.

You can also compare 2 browsers (e.g. IE10 and Firefox 3.6: http://www.caniuse.com/#compare=y&b1=ie+10&b2=firefo...) and it shows support by current browser market share (e.g. 53% of users have browsers with CSS3 border-radius).

The tests are not thorough (not everything is testable), and I'm worried that it encourages browser developers to do shoddy job.

Case in point: for a while form validation in WebKit (in "stable" Chrome and Safari!) was only implemented on DOM side and had absolutely no UI, which caused invalid/incomplete forms to "mysteriously" fail to submit, leaving users confused and preventing server from offering fallback.

But it passed automatic test!

In the interest of completeness, I've tested the following browsers:

  Internet Explorer 5: no score
  Mozilla Phoenix 0.1: 16
  Mozilla Firefox 1.0: 12
  Mozilla Firefox 2.0: 37
  Netscape Communicator 4.5: crashes
  Netscape Navigator 6.2: 12
  Netscape Navigator 7.0: 16

They should write code to detect whether each HTML 5 feature is native.

Ok, mod the humor down now.

I know this will inspire some backlash but I am sick and tired of all these different browsers and what they can and cannot support. I would actually pay good money every year for a browser if they would all just do the same job.

As a developer all this crap gets boring, day after day, endless checking broswers. And to top that off, Microsoft are thinking of releasing another. IE6 support ends in 2014, are Microsoft trying their best to have 10 active versions before then??

I think, just leave one of the companies to it and let them get on with it, I know competition inspires innovation but its getting stupid now. I think leave either Mozilla, Google or Opera to the job. Operating system supplier, get on with what you are doing and make operating systems!

Microsoft, if you are reading this, please give it up. I use your products every day but I have not actively used Internet Explorer for 6 years, I only use it when I want my blood to boil when I test a website in IE6, IE7, IE8 and IE9. Only to see that it works in every single one but IE7????!!?!?!?!?!

Tests like this shouldn't need to exist. It should just be a case of, "Oh there is a new standard out, {Browser Manufacturer} supports this already and we are good to go".

I wish I could say, "Rant over". But it's not, i'm going grey at 27 and it's all because of browser testing. Yes Microsoft, I'm looking at you.

Honestly, if you can't handle developing for different systems with minor different bugs that you have to work around, I think development is not the thing for you. It's like people expect things to be perfect and they whine and moan if they have to do a little extra work because it's not. Get out in the real world of non-computer engineering and product development and you have to deal with all sorts of non-ideal situations, and people just deal with it.

So you are happy to accept that there is a standard that people do not adhere to?

It's not like I have to do a "little extra work", it's a lot of extra work. We can't do transparent PNG's because IE6 doesn't support them. We can't use PNG fixes as it overlays buttons that are absolutely positioned as it gives an overlay on them.

I'm not asking for things to be perfect, I'm asking for a standard and for people to stick to it. Microsoft are useless when it comes to browsers. Yeah they more or less invented ajax with OWA but beyond that they are just a pain.

I think you have given it to the situation, it doesn't have to be like this. There are many browsers that don't have these issues. When I test my website in Mozilla, I don't find myself checking Safari, Chrome or Opera, I know they work.

If you looked around the real world you would spot there is a whole campaign to bring IE6 down, its not just a few people "Whining and moaning", its a whole world annoyed with a stupid browser that will have been active for 13 years when it departs. It's with Microsoft having the worst update policies and coming out with stupid comments like "let's make HTML 5 native to Windows 7". Native for gods sake. The guy should have been shot when he came out with that comment. Sacked and forgotten, that is the mindset that has got Microsoft where it is today with IE6.

With Internet Explorer's native support for HTML5, I thought it would score higher.

Personally I think this test is not set up properly.

It does appear that IE 10 has made improvements on HTML5 support over IE9. Going off this test however, they would receive the same score of 130.

Unfortunately, these HTML/CSS/Javascript tests tend not to be about correctness on the large. ACID3 was probably the best example of something that was meant to specifically show how IE was poor. It tested specifically those items that IE was deficient at. It didn't weigh all aspects of the standard equally.

An ideal test would go through and enumerate the standard and attempt to weigh each feature with expected usage and potential to workaround. I've yet to see anyone even attempt it.

The W3C HTML Working Group is working on a complete testsuite for HTML5. The WebApps working group are doing similar things for their specs. many of which are often loosely referred to as "HTML5". The goal of these testsuites is to help browsers achieve interoperability, so the emphasis is on getting high quality tests that cover the whole spec and include difficult cases.

If you want to help out, contributions are very much appreciated; see [1]. Indeed, contributing tests is probably the most effective way to reduce browser interoperability problems and hence pain for future web developers.

If you want to take the tests and start adding subjective weightings based on importance and ease of working around fails, the liberal (BSD-style) license will let you do that (although obviously you can't claim that your derived work is the official W3C testsuite anymore).

[1] http://www.w3.org/html/wg/wiki/Testing

It looks like they're not very far along. Do you know if there is an ETA on when they think it will reach signifigant?

I find it surprising to see they only have 925 tests. It seems to me that a spec of this nature is the type that would really benefit from a very test-based approach (as a s section is being written, so are conformance tests).

Despite Microsoft mentioning "native" HTML5 dozens of times when IE 10 PP 1 was released, there are actually no changes in HTML 5 support in IE 10 PP 1. The only changes were either CSS or Javascript. So having the same score as IE 9 would be correct.

Calling this an HTML5 test seems overstated. Of the 400 possible total points, 131 are from "related standards". Seems like you'd want to break those out to have 269 for HTML and 131 for related.

Even though the related specs are not part of the actual W3C HTML5 spec, they are considered by many people to be HTML5 in the broader sense.

That being said, many of those related specs were part of HTML 5 in the past, but spun of to there own spec. Others were proposed on the WhatWG working group, part of the WhatWG Living HTML spec or extensions to HTML 5 elements.

Safari Version 4.1.3 (4533.19.4) on Mac OS X 10.4 scored 228.

Chromium on Linux scored 276 and 13 bonus points.

130 by IE9 and 288 by Chrome. Impressive!

Opera Mini 6.0.24095 on Nokia E63 '35'

Reply on news.ycombinator.com