IE10: How does it really stack up? | Hacker News Reader

IE10: How does it really stack up?

lucidchart.com

93 pointspughbri66 comments

The real problem is that while there will be a new and better Chrome and in a month and a half, and similar for Firefox, we'll have this build of IE10 for 2 years or maybe more.

How they compare at the start of the cycle is not typical, how they compare at the midpoint will be more average, how they compare at the end (where IE7 or IE8 is now) is the worst case.

Yeah, at first I thought, that's impressive that they can catch up so far, even if they fall back again. But then I thought about how much further ahead Firefox and Chrome would be if they'd effectively abandoned their users a couple of years ago and hid in some research lab to plan the next big version. That's a whole lot less testing, bug-fixing, releasing and reworking etc.

Completely agree. IE10 can't even beat Chrome when it's launching once every 2 years. This means Chrome will simply expand the performance gap during this 2 year period.

Now if only Chrome for Android had version/feature parity with Chrome for desktop, because it's a full 5 versions behind right now - and it shows. It's not even competitive anymore on mobile because of this huge development gap. Google needs to take Chrome for Android as seriously as they take Chrome for desktop, if not more so.

Internet Explorer will start automatically updating.

http://blogs.windows.com/ie/b/ie/archive/2011/12/15/ie-to-st...

The question is how often. Adopting the firefox 2/3 model (albeit less nagging a-la-chrome) is a great step forward from the IE6 one we have had up to now, but they really need to match the rapid release schedules of mozilla and google today. At a minimum quarterly updates (features/engine updates, not just security fixes) to stop hindering the web.

Less naggy? I find Firefox to be one of the most naggy browsers about upgrades. Or have they changed this recently? With Chrome you just see a notification on the settings menu, and a restart brings you up to the next version. I really hope IE adopts this model.

He said, "less naggy a-la chrome". Firefox nags, chrome doesn't.

I'm pretty sure recent versions of Firefox (say, since the early teens) have had nagless updates.

I find myself having to manually check the About dialog for updates now, because they're about the only reason I ever restart my browser, and if I don't manually check occasionally I'd have no way of knowing an updated existed.

That article is interesting, but rather vague.

> "With automatic updates enabled through Windows Update, customers can receive IE9 and future versions of Internet Explorer seamlessly without any "update fatigue" issues."

How often will these updates come out?

And what about the usual tying of browsers to OS. IE 10 will come out for Windows 7 ... which means that people running XP (or Vista) won't get it. So IE9 will still be in play for a while, and by the time this fades into irrelevance like IE6...

Will IE11 be Windows 8 only? What about IE12? Or is there going to be an IE10.1 with new features in only six months time. We don't know. It does not dispel the (currently correct) perception that for the foreseeable future, some version of IE will be the boat-anchor holding the web back.

We gathered timing information for the parts of the benchmark that it did complete

Which parts didn't complete? Did you try the beta or aurora?

Code that can consistently crash firefox would probably be elgible for a security bounty.

Interestingly, the part that crashed Firefox was one of the simpler tests--typing a bunch of text content in, then resizing and rotating the shapes that contain that text. Firefox didn't give any details as to why it crashed.

The only thing I could even measure that would lead to a crash is memory usage. I just re-ran the test and watched the process's memory usage. It hovered in the 300-400MB range during the first third of the test. Then, during the test I describe, memory usage rocketed to around 1GB, at which point the browser crashed hard.

I'm not sure why Firefox would exhibit this behavior when other browsers don't, but we know a few people at Mozilla (they're customers of ours) so we'll probably reach out.

Why not remove the part of the test that crashed it and post results of the rest of the benchmarks? Those are 3 tests out of 16, I would still like to see how it compares in the other benchmark tests.

Seeing "We're sorry. Firefox had a problem and crashed." as well as its exclusion from the performance summary made it feel like there was a bit of a bias against Firefox, where even in your conclusion it's stated that it performs "quite well" - that sentence does not reflect on the rest of the article at all.

resizing and rotating the shapes that contain that text

Maybe it was hardware acceleration in the particular machine with the particular video driver you use?

Can you try the test on an entirely different video setup just to prove it can get through, even if the numbers cannot be compared?

Well, let's agree that IE10 does well whatever it is designed to do well. Cool. Now what? We have an IE10 browser that compares well with Google Chrome, Opera and perhaps Firefox too (I don't know the exact details about your tests that were making Firefox crash as often as you've claimed in your article).

So is this the point where Microsoft takes on the real challenge? Will it go beyond its typical high-speed sales pitch for the new browser and attack the more relevant problem that is to really get those IE6/IE7/IE8 and perhaps even IE9 users to adopt IE10 immediately?

Just a few days ago, I saw on HN how IE6 has become a political election issue in South Korea!

Pushing the lazy and entrenched users out of the hole is a much more important (and harder) problem to solve than to win medals over browser speed and performance tests. Even if tests like these help push the cause of IE10, it would be appropriate to give out all details of the tests and realize why Firefox was 'disagreeing' to your test at all?

IE 6, 7 and maybe 8 users are probably running XP so can't upgrade.

IE 9 users on Vista are similarly out of luck.

I won't be recommending IE 10 even to my clients on Windows 7 or 8 until they have an silent auto-updater and a frequent release cycle. It's the only way we'll stop IE's user base holding back the web.

They do have auto-updates for IE now: http://blogs.windows.com/ie/b/ie/archive/2011/12/15/ie-to-st...

But it's not quite there yet because it relies on Windows Update and it only shows the option if you haven't previously declined an upgrade to a new version of IE. But at least it's a start.

This is the one thing which always mystified me about MS and IE. Why do they continually dead end their browser to a specific OS release?

I can't imagine someone in their marketing department actually thought by doing so would get people to upgrade their OS so they could have a more up to date browser.

Chrome doesn't work on older versions of OS X either.

Developers don't like to develop for legacy systems.

The newer versions use features that didn't exist in older Windows. Features which are core components to how the new browsers work, both in terms of rendering and security. Someone made the decision that having these features was more important than backwards compatibility; it wasn't a marketing decision made after-the-fact. If there were no marketing department, IE10 still would not be able to run in Windows XP.

Chrome and Firefox both work fine on my copy of XP. So those new features can't be very important.

Here, learn why so you can form an informed opinion. http://arstechnica.com/information-technology/2010/04/why-mi...

IE10 and Windows 8 are even stronger on the security points. Think of PC security as somewhat akin to vaccination. It only works for everyone when enough people are immunized. Herd immunity, it's called. Your sticking with objectively less secure browsers on objectively less secure platforms hurts us all. It's not something Microsoft should be encouraging, not just from a business or marketing standpoint, but a moral one as well.

Oh, and IE10 is objectively faster than Firefox too, thanks to the integration with DirectX 10 which is already compositing everything else on the screen in Windows 7 and 8.

Haha, I use a modern Linux 95% of the time these days, but keep an old box around for testing. I run a very tight ship and that box has never been owned (except for the one morning in ~2004 when the dcom exploit hit). Your implication that I'm a security threat is amusing.

The nice technologies the article mentions are operating system level and don't "have to" be tied to the browser. They are using fancy terms to describe security features from the 80's (on more professional systems). "Browsers can't write to the OS to install rootkits"... wow.

MS decided not to backport to further corporate goals, fine. But, the idea that IE has to be part of the OS was debunked a decade ago.

How do you suggest Microsoft goes about convincing the millions of East Asian users running IE6 on a pirated version of XP to upgrade to IE10?

Release IE10 for WinXP, and don't check authentication.

Given that IE 10 (and 9 before it) builds on core Windows APIs only introduced in Windows 7 I'd consider that unlikely. And they definitely won't invest the time and effort of backporting DirectWrite, Direct2D and whatnot to XP (this might even include DirectX 10 and/or 11 along with WARP) which is already out of mainstream support and only get security patches.

You gotta suspect they'll be forced into upgrading eventually. Developers simply can't continue supporting those archaic versions.

But then I suppose the operative word there is 'eventually' which can't come soon enough.

I think they should see if they can release a strip down demo version of windows. No frills or anything just the basic required functionality with clear path for upgrading. That version should be able to work on WinXP level hardware and be easy to upgrade.

I would probably be hard to offer something like this without cannibalizing some of their sales but in Asia, they have tons of users who will never purchase a legitimate version of windows but are sizable enough that they have to cater to them for other services.

If they aren't your (potential) customers and/or those of your advertisers, do you really care?

All the countries in North America, Europe and South America ex-Chile and and Venezuela are at less than 1% IE 6 usage.[1]

7 & 8, which are better but still not great developer-wise, unfortunately still have a ways to go before passing that threshold. But at a minimum you should think long and hard before pouring time into IE 6 compatibility.

[1] http://www.ie6countdown.com/#list

I think part of the problem is that IE is the browser of choice for the Dept. of Defense. If you've ever worked for the DoD, you know that updates, etc. take forever because of security concerns. I wonder if that relationship informs IE's decision to hold back on their browser capabilities.

I'm very familiar with the DoD's byzantine and prehistoric policies. You're absolutely right. And it's ironic, because changing their patch management strategy would provide better security. The DoD is stuck in a compliance world, rather that one in which they manage risk.

Yep. They're getting left behind when they should be on the cutting edge. It's sad, really.

Yea but IE9/IE10 are number one at stopping malicious downloads which happens to be the easiest/most prevalent way to take over a computer. Plus IE 8+ is the only browser that has TLS 1.2.

Only on Win7 (since IE depends on SChannel), and Opera has TLS 1.2 too.

There is the risk of having your computers compromised and there is the risk of being fired for making someonewho writes policies look incompetent.

attack the more relevant problem

You know, one way to attack this problem is to make IE10 a more compelling upgrade.

This should really be titled "IE10: How does it really stack up in one benchmark."

Not just one benchmark, but one apparently completely undocumented benchmark run under Windows 8 on a single laptop.

They don't mention how many times the benchmark ran which makes me suspicious that the answer might be one??

Unless they release the benchmarking tool then this result seems pretty worthless to me.

>we’ve developed a benchmark that runs inside of the Lucidchart diagramming application //

So maybe that line for Firefox could as well read "Lucidchart crashed"?

So maybe that line for Firefox could as well read "Lucidchart crashed"?

No, I'd say that JS should never crash the browser, no matter what.

OK fair enough but if I code something wrongly it doesn't work and fails badly then I'm at least partly to be blame, no?

It's moot anyway as it appears the test is proprietary and so can't be investigated further?

There is one more strange thing about IE10 that I should mention. IE9 often maxes out one CPU core when a Lucidchart diagram is open, even when the user is completely idle. IE10 is somewhat better, but on our test machine it still used about 50% of one core when the user is idle. No other browser has any detectable CPU load under the same conditions.

Has anyone else seen behavior similar to this in their own javascripting (is that even a word?)? I hadn't been paying special attention to IE's CPU usage but I'd like to think I would have noticed this in any of my own stuff when I was doing browser compatibility testing.

According to http://blogs.msdn.com/b/ie/archive/2012/06/13/advances-in-ja..., IE has separate GC and JIT threads- if I were to guess, I bet it's the GC

IE10 had to skip a bunch of frames and use half a core to not quite keep up with chrome and roughly match Firefox... Well, I suppose its an improvement still, but they had a pretty low bar to begin with.

In IE10 the JavaScript is multithreaded... there's a separate thread for JS parsing, compiling, and garbage collection. As far as I know this is the only browser with a threaded JavaScript VM. So the skipping frames is probably because the rendering can be done independently from running JavaScript, and the core in use is probably some quirk like the benchmark creating a few objects on a timer, causing the GC to run on another thread -- that kind of thing can be fixed pretty easily, either by tweaking the benchmark (if it were publicly available) or by Microsoft fixing IE.

I haven't actually used IE10 yet, but I think you'll find this new architecture makes overall browsing more responsive and smoother than Chrome.

Opera used to market itself almost exclusively on how it was faster than any other comparable browser. I just had a look on the Opera website and now it says "Faster & Safer internet" yet it performed worst on this test. As the underdog in the browser market, despite being what has always seemed to me as a pretty nice piece of software, I feel a bit sorry for them. I hope they can do whatever it takes to catch up with Chrome on the performance front.

We were a bit surprised on Opera's performance as well, since it did reasonably well in our last benchmark (vs. Chrome 6, FF4, IE9 beta, etc).

http://www.lucidchart.com/blog/2010/09/16/ie9-ff4-beta-in-re...

>a look on the Opera website and now it says "Faster & Safer internet" yet it performed worst on this test. //

Which page were you on, the front page of the website doesn't say that to me?

My first instinct would be that they're referring to their "turbo" implementation and their large mobile install base, cf http://www.opera.com/browser/turbo/.

> Which page were you on, the front page of the website doesn't say that to me?

It's the contents of the <title> tag so it displays on the title bar of the browser you are using to read it with and in the tab if you are using a browser with tabs.

Lol, I don't normally have a title bar (turned off for main FF window) and the relevant part was most likely hidden by having a dozen or so tags open. Thanks answering.

I use Opera on my netbook (Atom 1.6Ghz, very slow). Chrome is actually faster at rendering or interacting with pages - which is what benchmarks generally measure. What they don't measure is how responsive the browser is. For me, Opera is far faster than Chrome at switching tabs, opening new tabs, having lots of tabs open in the background, having more than one tab loading at once, and eats less of my system resources than Chrome to achieve them.

Were all of these tests done on a Windows8 machine? I only ask because the latest version of Safari isnt available for Windows (you may be able to install WebKit nightly as a Windows user though).

Id like to see how nightly build of browsers stack up as well.

That's right, these were all done on a Windows 8 machine. We could of course test on Safari 6 on a Mac, but then the numbers wouldn't be directly comparable (since it's different hardware).

I think your article would benefit from making this piont clearer. There's been discussions on HN before about how performance of Apple software on Windows differs greatly compared with performance of Apple software on Apple OSes.

The same version of Chrome across both platforms can be a control proxy.

Others have already mentioned the probably slow pace of IE updates. Another thing I'll mention is that testing IE requires Windows, and I haven't worked at a place in years where anyone on my team was using Windows. Years. I've worked at Ask.com, Cloudkick, Rackspace, and now a new startup. Until I can test IE on a Mac, I wont bother to support it unless customers mention something is broken, and they rarely do. Because few use IE.

You may have heard of this new thing called “Virtual Machine”.

My development environment runs on a VM. You can't run multiple VMs on a host. I actually didn't want to reply to you because of your sarcasm. Also your comment history is very windows focused.

The only time I ever had problems running multiple VMs at once was when I used two different VMMs to do so. In fact, testing IE on Windows is equally painful as you need the VMs for the older versions anyway (unless you happen to have a few real machines lying around).

Depending on your site's demographics you may choose to ignore IE testing, I understand. But often that demographic is rather disconnected from your own development OS so you may not always get around it.

Also your comment history is very JS-focused (but how does that matter in any way?).

My comment history is more varied I think. My development environment runs a lot of stuff, so even if running two vm's at once were realistic, it wouldn't be.

My skills are a little rough and this is probably more than a company can devote to a browser war study but it would be interesting to see this benchmark run multiple times or as a Monte Carlo simulation.

Also - does anyone know what core runs at 50% in IE even when the user is idle? Not surprised to hear it but curious if anyone knows what it is.

While idle, the profiler tells me that the function with highest time is yp at line 5198 in chart.js. But no idea whether that's really the culprit.

Hi, Lucid CTO here. We've spent a lot of time in the IE profiler, and it never appears that any of our Javascript is consuming much CPU at all. There are a handful of functions, like the one you quoted, that poll frequently on a setInterval. But none of them are doing much more than checking a variable and returning almost every time. It's really bizarre.

The benchmark is "internal only". How convenient. So basically those are just meaningless pretty graphics.

Benchmarking Safari under Windows pretty meaningless.

Wait, did you make the other browsers your default before running the tests? I was under the impression that IE is the only first-class browser and the other browsers would work as third-party apps with limitations on things like the amount of processes it can open, etc. I ask because I supremely doubt that Mozilla Firefox couldn't pass your benchmark, although I don't use it myself.

There are no limitations on desktop applications in Windows 8 whatsoever.

Do you have any article source for this claim?

Reply on news.ycombinator.com