Comment by kragen | Hacker News Reader

Comment by kragen | Hacker News Reader

Parent Full thread

It's disappointing that they don't include any performance results from Sparkplug itself, only from V8 using Sparkplug some unspecified and unknowable percentage of the time. While it's true that Octane doesn't provide optimal performance targets for engineering V8, it does provide a legible approximation of the relative speeds of different operations.

From what's in the post, we can't tell if the interpreter is 128× slower than optimized code while Sparkplug code is only 32× slower, or the interpreter is 32× slower and Sparkplug code is 16× slower, or the interpreter is 32× slower and Sparkplug code is 4× slower (which is my guess as to the reality of the situation.) I'm more interested in this is-it-4×-or-32× question than the possibility that microbenchmarks might misrepresent a 32× slowdown as being a 48× slowdown or a 24× slowdown.

https://docs.google.com/document/d/13c-xXmFOMcpUQNqo66XWQt3u... does offer more performance numbers, but they're for "an early prototype", and more importantly they don't break out the compilation speed vs. the execution speed.

The description in the post makes it sound like Sparkplug output is pretty similar to direct-threaded Forth code, just a sequence of subroutine calls to other methods and implementations of primitive operations, interspersed with the occasional open-coded primitive (=== maybe) and control flow. And typically that means an overhead of around 4×, although maybe it's more with JS's aggressive implicit coercions and whatnot. It sure would have been nice to see some disassembly listings, too.

Author here: we do have various numbers collected here, with varying levels of "I'm sure enough about these numbers to share them in a public blog post".

Compile time is on roughly the same order of magnitude as Ignition compilation (just AST to bytecode, so excluding parsing), and roughly two to three orders of magnitude faster that TurboFan. The relative performance to the interpreter, as the sibling comments point out, varies _wildly_ by workload, but around 4x is probably a decent approximation for something not entirely dominated by property loads.

Thank you, that's great! How fast are property loads?

Those vary widely too, depending on what (and more importantly, how many) object shapes they see. They'll be roughly the same cost in Ignition and Sparkplug though, since they effectively call the same handlers for those loads (and way faster in TurboFan, since that one then can generate optimised code tailored to that type feedback)

Oh, so code that's entirely dominated by property loads will be much slower in Sparkplug than in TurboFan, by a factor of more than that 4×, perhaps closer to the 32× I was SWAGging for Ignition's overall relative slowdown? I guess at this point I ought to be running microbenchmarks instead of asking you to do it for me... ;)

Sorry, the 4x was my estimate of Ignition vs. Sparkplug (I guess it's closer to Iain's estimate of 2-4x). Sparkplug vs. TurboFan is crazy workload dependent. For microbenchmark-y tight loops, integer attorneys, and clean single-shape ("monomorphic") property loads, TurboFan is orders of magnitude faster. For small functions that just have non-inlineable function calls and heavily polymorphic property loads (with lots of shapes so you can't skip the property lookup), like you see in a lot of web code, they're close to on-par (because there's nothing TF can optimise away). If you put a gun to my head, I'd say it's on "average" maybe 10x? Very much a Fermi approximation though.

Oh, thanks! That's surprising! And precisely the kind of question I was interested in.

As an educated guess: Sparkplug is very similar in design to SpiderMonkey's baseline compiler. I ran a quick test a few weeks ago to see what the performance delta was between SM's baseline compiler and baseline interpreter, and going from the interpreter to the compiler was about a 2x speedup. Based on that, I'd expect that you're looking at a ~2-4x speedup from the interpreter to Sparkplug.

Edited to add: Which is a very good result! Congrats to the V8 team for landing this!

I doubt guesstimating based on interpreter loop overhead is going to get you that far, because Sparkplug looks like it has ICs. I would expect their impact to dominate on many, many workloads.

ICs? Integrated circuits?

Sorry, Inline Caches.

From what I understand, there are basically 3 things that provide the bulk of performance gains in compilers: inline caches, register allocation, and inlining. The latter two don't really apply to interpreters or baseline compilers.

Compilers do many optimizations beyond those, but generally only for incremental improvements.

Oh, sorry, that should have been obvious. Are you suggesting that the performance improvement from inline caches (whether PIC or by-the-book monomorphic Deutsch & Schiffman) wouldn't show up in microbenchmarks, or would be misleadingly effective in microbenchmarks? IIRC Octane has a benchmark or two specifically for this...

I think constant folding, specialization, and loop hoisting (especially of bounds checks) commonly produce more than merely incremental improvements. If you don't have some kind of inlining you don't get those (usually) but inlining by itself is usually only a minor improvement.

You're right, inlining by itself doesn't do much. I think inlining + register allocation is the big one, but you might be right about the other optimizations that are enabled (or at least massively boosted) by inlining.

ICs steal a lot of specialization's thunder. (Alternatively, since the point of ICs is specialization, you could say it's the other way around!)

Well, just as inlining is necessary for constant propagation or loop hoisting in a lot of cases, specialization is necessary for inlining in a lot of cases. Despite the nominative similarity, inline caches don't enable inlining in the same way specialization does, because they just enact the transfer of control more quickly; the code they transfer control to is just the standard generic version of the code, not an inline copy that can be optimized further according to the arguments at that callsite.

Reply on news.ycombinator.com