LLVM based JavaScript JIT that beats V8 on Asm.js benchmarks
trac.webkit.org
trac.webkit.org
V8 doesn't explicitly support asm.js so it's easier to beat than Firefox.
A powerful JIT like LLVM will tend to do best on large long-running benchmarks, like asm.js.
"Some benchmarks use time (lower is better), and some use points (higher is better). We orient all graphs so lower is better, visually."
> JavaScriptCore uses LLVM for a top-tier high-throughput optimizing JIT, which we call the FTL (Fourth Tier LLVM).
Because if they're saying the new webkit jsc (using asm.js) is faster than V8 (not using using asm.js), then it's not really a fair comparison.
However, V8 doesn't have a separate mode. The "'use asm';" pragma is ignored.
The Octane benchmark suite does contain Emscripten and Mandreel benchmarks though. So, they are tracking it and they also try to make this kind of code run faster.
That doesn't follow. They certainly care about it; it's now in the Octane benchmark suite, which means they think it's representative of code that should be optimized for, but it's only part of the suite, which means it's just one more style of code that should be able to run efficiently in a modern JS engine.
Okay, I know what you mean, but still why wouldn't it be a fair comparison? V8 is free to "support" asm.js, whatever "support" means.
Nobody will see any benefits from JSCore being "faster than V8 on asm.js".
In fact, if JSC is slower on non asm.js code, the comparison would also be extremely misleading too, because 99.9999% of web code is NOT asm.js.
At the moment it's like saying "JSC is faster than v8 on a synthetic benchmark almost nobody really uses in the actual world, and v8 hasn't bothered optimizing for".
Whether it matters is an interesting question, but V8 developers seem to disagree with you, because they included asm.js benchmark in Octane benchmark suite, which V8 developers use to tune V8.
Also obviously I covered that, when I wrote "99.9999% of web code is NOT asm.js". So who are these "people"? All 10 of them?
>Whether it matters is an interesting question, but V8 developers seem to disagree with you, because they included asm.js benchmark in Octane benchmark suite, which V8 developers use to tune V8.
If anything, it shows that they agree with me. See how I compared asm.js to a "synthetic benchmark" in my comment?
V8 doesn't have any specific optimizations for asm.js, opting instead to focus on optimizing JavaScript as a whole (which asm.js also benefits from). That's their choice, and this is the result of that.
Because nobody (for large values of "nobody") cares for asm.js. 99.999% of Javascript code out there is not asm.js, and nobody really uses it at the moment for production/business. Some PoC games mainly.
So the fact that it beats V8 on something:
a) V8 does not implement, b) requires specifically written (actually "compiled" from things like emscripten) JS code, c) is irrelevant at the moment for JS performance on the web,
makes it pretty irrelevant itself.
The most important games companies - Unity, Epic, etc. - care very much about asm.js. Unity is going into beta very soon. Expect to see games after that.
> So the fact that it beats V8 on something: a) V8 does not implement
jsc also does not "implement" asm.js, in the sense that both v8 and jsc do not detect asm.js code specifically using the asm.js type system. Yet, jsc beats v8.
asm.js code is going to matter because games matter. Those games are using plugins now, but plugins are going away, and asm.js is the best option after that - that's why the games industry is adopting it.
asm.js is a bizarre hack. You're sending someone assembly code for a CPU that doesn't exist, as text.
It would have been far superior to just figure out CPU family specific sandboxes as NaCL did, and then accept that only two CPU architectures matter these days: x86 and ARM. If you can sandbox both, you're done and if someone wants to invent some super-duper new ISA they can damn well make code easily sandboxed when they design it.
"use asm";I use it for non-game PoCs. Why do you care about V8 trying to be fast? Looks like JSC and Ion are beating them. JSC has no special "use asm" path.
JavaScript is what the web "runs on", and if Firefox might in the future provide the fastest way to run JavaScript. Obviously for most cases that isn't going to be that relevant, but I can imagine a situation where you have Photoshop-in-the-browser and it runs better on Firefox due asm.js.
It's neat that V8 is fast on long-running code that's not asm.js, but it's wrong to extrapolate that to most real-world web code.
Apple may not care about such use cases. The fact that they could get ASM speedups for free is pretty cool. : )
Google could implement such optimizations as well, instead of spending their resources into NaCL and Dart VM.
1. A generic JS JIT that runs generic JS faster than another generic JS JIT.
2. ASM.JS JIT that runs ASM.JS faster than another ASM.JS JIT.
3. A generic JS JIT that runs ASM.JS faster than another generic JS JIT.
It's not a significant milestone, as SpiderMonkey's ASM.JS implementation outperforms this one, and V8 isn't implementing ASM.JS in any way. As in at all. So of course it'll be slower.
The purpose of ASM.JS is to allow a separate ASM.JS runtime to run ASM.JS code. The reason it happens to be a JS subset, is so you can have a fallback for backwards compatibility.
Not to say this LLVM engine doesn't show potential. But implementing ASM.JS in LLVM is not the hard part of implementing a generic JS JIT.
ASM.JS is entirely static, so it's comparatively trivial to JIT, it's literally an ASM to ASM transpiler.
In the broader context, this shows that the JSC approach of taking LLVM and using it as a JIT - called FTL - can beat Chrome/v8 on large long-running computational code. That means that LLVM is significantly better than v8's backend JIT at some things. This is not too surprising since LLVM is much larger and sophisticated - it has a more powerful register allocator, supports things like float32, etc.
Those two examples weren't random - Firefox has added an LLVM-like register allocator and float32 support (not just for asm.js code, in general). So there are two paths here: Safari takes LLVM, and Firefox implements some bits from LLVM in its own JIT. v8 so far has done neither, but hopefully has plans to do one of the two, or some other cool thing.
In addition there is the issue of asm.js. Safari doesn't detect it, and beats Chrome but still trails Firefox. But it's far from clear that is the last word - Safari's new JIT just landed, it can be tuned a lot more, presumably. So it might still end up showing that a general JIT can beat custom asm.js optimizations, time will tell.
What is clear though is that the Safari's FTL JIT took far more work than Firefox's asm.js optimizations. If the LLVM-based JIT mostly helps asm.js code anyhow - it shows little benefits in other benchmarks so far - then that raises interesting questions (if the benefit is mostly for asm.js, why not focus on asm.js specifically, if it's simpler and more effective?). But again, FTL just landed, perhaps with more tuning it will show general benefits.
Exciting times!
I object to "all the dynamicness of the rest of the JS is what V8 excels at". V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset.
Typescript benchmark included in Octane benchmark suite is a good example. In Google's own words: "This test measures the time TypeScript takes to compile itself and is a proxy of how well a VM handles complex and sizable Javascript applications".
In particular, Crankshaft is not effective for this benchmark. (Check yourself by running it with Crankshaft disabled: the score actually improves because you save compilation effort.) And as you can check yourself or check on AreWeFastYet, JavaScriptCore beats V8 by 2x on this benchmark.
I admit Crankshaft subset of JavaScript is large and interesting (certainly compared to asm.js subset) but it is not true that V8 excels at dynamic JavaScript, for sufficiently large value of dynamic. JavaScriptCore does.
JSC might be warming faster or simply have a better cost / benefit ratio at lower tiers. I doubt FTL kicks at all for this benchmark given how short running it is. V8 also deopts quite a bit on it, so it might be some pathological case... This is definitely something worth exploring and fixing on V8 side.
As I understand it Crankshaft's claim to fame is the runtime type specialization, which does well if the code is hot and ends up using same types in all cases. Is this what you are referring to?