Benchmarking JS
mrale.ph
mrale.ph
His blog http://mrale.ph/ is full of hidden goodies like Object.create vs function constructor [1] and hidden classes vs jsperf [2]. Lots of good stuff on his SO too.
It'd be great if he (hi if you're reading this btw!) could write a bit about TF and the changes being made to v8 internally from an overview point of view.
[1] http://mrale.ph/blog/2014/07/30/constructor-vs-objectcreate.... [2] http://mrale.ph/blog/2013/08/14/hidden-classes-vs-jsperf.htm...
I might write something about TF but right now it's still not used to compile normal code and the team is just starting on stuff like adaptive optimizations, so it's a bit too early.
Also given that I don't work on V8 anymore I always think it's a bit unfair for me to do it --- somebody from V8 team should do it instead, cause my perspective is bound to be slightly different from theirs :)
I can only find the videos from 2014.
This talk is a combination of two separate talks I have given before plus two new examples, based on my recent endeavors.
Talks merged here are LXJS 2013 one[1] where I talk about microbenchmarking pitfalls and WebRebels 2014 one[2] where describe how bugs in VMs can affect benchmarking results.
If you have any questions about the slides just ask them here or send me an email --- I will try to answer as soon as I can.
[1] https://www.youtube.com/watch?v=65-RbBwZQdU
[2] https://webrebels.23video.com/crooked-mirrors-of-performance...
Thanks again!
edit: to clarify. In C++ I write dumb but readable code knowing that the compiler will always perform specific optimizations. In JS it seems that there are many factors resulting in different optimization levels. How do I ensure that the VM does not execute my 'dumb but readable' code literally? Without that guarantee it seems that I have to frequently resort to optimized yet unreadable code.
For a single test case where V8 goes off the rails there are millions of lines of code across the globe which V8 optimizes correctly.
On a funny note: I actually do have a version of this talk where I show GCC going slightly of the rails and producing a code that is 3 times slower than it should be because it hits an (infamous) partial register dependency stall --- see StackOverflow question[1] for the gory details.
> How do I ensure that the VM does not execute my 'dumb but readable' code literally?
Well, as I do say in the talk: reasonable code should be reasonably fast. If it is not the case --- file bugs with VM vendors.
Keeping your code relatively static / monomorphic is the best way to achive performance in any language.
In any case I think it's much much much more important to optimize algorithms not their concrete implementations.
[1] http://stackoverflow.com/questions/26585977/64-bit-code-gene...
I guess a lot of Javascript micro-optimization comes down to don't change field type and give type hints in assignments. Like:
var integerType = 0|0;
integerType = 45|0;
integerType = (integerType + 1)|0;
Emscripten can be used to compile C++ into Javascript. Perhaps it could help you?
Sign up to their newsletter at http://gotocon.com/chicago-2015/
And I had no clue that js-engines has come this far, optimizing code during runtime this efficiently(if you ignore the bugs.. :)).
Some of the optimizations compilers/jits will perform would blow your mind.
LICM is a very simple thing to do.
There are more advanced things, like http://www.cs.rice.edu/~keith/512/2011/Lectures/L06CLZ-1up.p...
etc
That's just on the code movement side too.
Most good static compilers can symbolically evaluate even linear systems of equations/recurrences/etc that are solvable, and transform loops that calculate them into constants, etc.
Yeah, it sometimes surprises me how much value the most simple optimizations have and how much they break people's attempts to measure performance.
There is another side to this medal which is best expressed in a quote I picked up from a relatively old paper:
"A survey of the literature on optimization uncovers many techniques that are hard to understand, harder to implement, more general than necessary and of marginal value. Our approach was to do the most profitable machine independent optimizations in the most profitable way" (A Portable Optimizing Compiler for Modula-2 by Michael L. Powel)
In some sense the more microbenchmarks an optimization breaks the bigger its impact on real world code is :)
At some point, you have to step back from what you are doing and actually design and engineer optimizations coherently, not just "do this little thing and that little thing"
LICM is highly dependent on the quality of your trace recording or method inlining.
Not disagreeing with what you wrote at all, just wanted to provide an anecdote.
The set of things you are worrying about it actually precisely the set of things CLZ solves :)
I think the set of things I worry about is slightly different, because they operate in a static environment and I operate in a dynamic one. For example, they don't seem to be worried about the transformation from:
loop {
if (pred) {
// o and o.f are loop invariants
Guard(o is X);
v = Load(o.f);
Use(v)
}
}
Guard(o is X);
v = Load(o.f);
loop {
if (pred) {
Use(v)
}
}
I did optimize JavaScript in the past, but it always worked different in different JS implementations and could change in different releases of the same JS engine. So I gave up on it, and it actually makes me more productive not having to think about with method is faster.
You should perhaps not optimize trivial things like methods vs function, because there is no good underlying reason for them to be much different. You should know when to optimize and what. I love the longer version of Knuth's quote - "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.". The overall architecture, the way you write and organize your code, in my opinion, and in case of JavaScript is likely to be in the 3%.
But I also agree not caring too much about it improves productiveness. Most applications really don't need to worry about performance in such depth. Even when they do, it often boils down to keeping one or two core components optimized.
The only worry I always have about it, is that when you notice your code style and architecture doesn't play well with JS engines, it's often too late to change it. A friend called me a day before a prod release asking what to do about his program taking a huge chunk of memory for what the profiler described as a "compiled code". It was a large corporate application, with dynamic code loading. We digged into the framework internals just to notice there was no built-in way of deregistering a class once it's created. Since it didn't play nicely with the JS engine, there were many more hidden classes than business models in the system, and most calls were polymorphic. All this compiled code had to be kept somewhere. At this point there was no way he would replace the framework and even quickly fixing it was a challenge.
The problem is that there are many different engines out there.
Unless of course you are talking about server-side code. But for the server-side, there are more efficient solutions than just javascript.
upd. done, scaled down to 1024x768