IainIreland

It's hard to predict statically which branches will be dynamically unpredictable.

A seasoned hardware architect once told me that Intel went all-in on predication for Itanium, under the assumption that a Sufficiently Smart Compiler could figure it out, and then discovered to their horror that their compiler team's best efforts were not Sufficiently Smart. He implied that this was why Intel pushed to get a profile-guided optimization step added to the SPEC CPU benchmark, since profiling was the only way to get sufficiently accurate data.

I've never gone back to see whether the timeline checks out, but it's a good story.

As I understand it, V8 keeps track of an ElementsKind for each array (or, more precisely, for the elements of every object; arrays are not special in this sense). If an array only contains floats, then they will all be stored unboxed and inline. See here: https://source.chromium.org/chromium/chromium/src/+/main:v8/...

I assume that integers are coerced to floats in this mode, and that there's a performance cliff if you store a non-number in such an array, but in both cases I'm just guessing.

In SpiderMonkey, as you say, we store all our values as doubles, and disguise the non-float values as NaNs.

We do streaming baseline compilation as fast as the network can hand us bytes, but we also kick off a more optimized compilation in a background thread.

A lot of optimizations can be baked into the generated wasm, but you still need to spend some time doing eg register allocation.

Generally we say "real users" internally as a comparison to benchmarks, as in "this improves JetStream by 5%, but I don't think it will make any difference to real users".

JSOP_SUNSPIDER was apparently a running joke back in the day inside SM.

To be clear, Firefox doesn't care about this benchmark either. It just so happens that changes we made for Speedometer also improved our performance here, which is a fun coincidence but not important in the grand scheme of things.

(Speedometer is the benchmark that most closely approximates messy real-world code.)

Yeah, I've been making this argument for years too.

Speedometer 3 is in the works, and it's moving in this direction. On multiple occasions we've done performance analysis of the latest draft version, noticed a particular chunk of code is hot, considered what it would take to optimize it, and then changed it because we don't think it reflects real-world usage.

EDIT: And I guess the answer for "why not fix SunSpider specifically?" is that once you've done the work to turn it into something worth optimizing, it no longer bears any resemblance to the original. It probably looks a lot more like Speedometer, honestly.

It's possible. We have plans to do so. There are a bunch of finicky corner cases, though. For example, destructuring will call the `return` method of the iterator when it's done (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...), so we have to be careful if somebody has defined `return` on the Array iterator prototype. You can supply a default value when destructuring, which is lazily evaluated, so we have to be careful about arbitrary side-effects taking place in the middle of the destructuring. We need to be able to fall back to something less optimized if we suddenly see a generator, or a user-written iterator. And so on.

Fundamentally, it's the same thing that impedes most optimizations: there are hundreds of different patterns we could optimize, it takes a fair bit of thought to get it right, and we have finite resources, so we have to pick and choose. This particular pattern turns out to be idiomatic in a lot of React code, which is why we're looking at it now. (Or so people tell me; most JS I write is gross little testcases.)

To be clear: we track it in the sense that we still have it running as part of old dashboards. It's cheap, and every so often it catches an unintended regression.

The SpiderMonkey team is very much on board with ignoring microbenchmarks. We care about Speedometer, because it correlates reasonably well with real-world performance. We pick and choose which parts of Jetstream2 we bother with. Anything older than that is strictly a measuring stick, rather than a target: it's nice when the numbers go up, but it will only happen as a side-effect of work we're doing for other reasons.

Notably, all the SunSpider wins on this graph are side-effects of Speedometer work we've been doing this year.

In general, there's no difference between the stack of the C++ program and the stack of the JS program. When SpiderMonkey just-in-time compiles a JS function, the result is native code that creates a stack frame on the same stack as the C++ code that implements SpiderMonkey. JS code and C++ code can be semi-arbitrarily interleaved on the stack: JS calls C++ calls JS calls C++...

The one exception about sharing the same stack is that the first few iterations of a function run in an interpreter implemented in C++, and that interpreter has its own stack that we heap-allocate. This particular bug occurred during the transition from the C++ interpreter to JIT code.

I wrote that comment. One other possibility: if you use spread syntax to call a function with a large array (eg `foo(...Array(20000))`), then all those arguments will be pushed on the stack.

(I didn't dig into the specifics of the Google code because, as weird as it is to have 20000 stack values, we really should be able to handle it. This was, at the end of the day, a bug in our stack probing code.)

Eh. I worked on IBM's COBOL compiler for four years or so, and I think you're overselling COBOL. COBOL is the best language for extending the functionality of critical legacy systems already written in COBOL. I have a soft spot for the language, in the same way you might for a puppy that is big-hearted but ultimately not very bright, but I can't imagine a greenfield project where COBOL is the right implementation choice. (Idiomatic COBOL does end up being surprisingly fast, mostly because the idioms were established in the 60s where things like "dynamic memory allocation" and "a call stack" were too expensive.)

The market for COBOL on .NET is limited by the fact that existing COBOL code is usually tightly integrated with the rest of the mainframe ecosystem (CICS, etc). The average COBOL shop has a low appetite for significant change. Even just recompiling the codebase with a newer version of the IBM compiler was often a big lift. (IIRC, when I left, COBOL 6.1 was generating code that was nearly twice as fast as COBOL 4.2 for CPU-bound code, although admittedly a lot of real-world COBOL isn't CPU-bound. It was still difficult to get people to migrate.) Anybody who wasn't change-averse and tied to the mainframe probably stopped being a COBOL shop years ago.

Edit to add: None of this is to say that Otterkit isn't a cool project! I just don't expect it to sweep through the world of banking.

By your definition, webassembly doesn't use a JIT, because it doesn't use any runtime information.

In practice, the JIT/AOT distinction boils down to whether compilation time counts as part of execution time, or not. If cycles you spend optimizing your code are cycles you don't spend running it, you make different design decisions.

(Note also that the Emacs native compiler is built using libgccjit.)

A possible non-Rust example: SpiderMonkey uses a trick we call NaN-boxing to represent JS values, which takes advantage of the fact that doubles have 2^52 different bit patterns for NaN to store 48-bit pointers with 4 bits leftover for a type tag. Provenance for the ptr->int->double->int->ptr loop might still be manageable using a "base of the GC heap" pointer and with_addr(), but there are also cases where built-in objects store arbitrary non-GC pointers in private slots (for example, a Map object has a raw pointer to its heap-allocated backing array). In that case, there isn't an obvious way to reestablish provenance (although we could hack it for e.g. Cheri by adding a layer of indirection through a GC allocation that stores the real pointer).

As a Canadian: this response should not be downvoted. It is correct and important. The War Measures Act used during the October Crisis was a different law; the Emergencies Act, which was just invoked, was passed as a response to concerns about civil liberties and abuse of power under the WMA. Temporary laws made under the Emergencies Act are, unlike the WMA, subject to the Canadian Charter of Rights and Freedoms.

More information on Wikipedia: https://en.wikipedia.org/wiki/Emergencies_Act

It may have changed since I left, but my understanding as of a few years ago was that IBM was replacing the front-end with Clang but continuing to use TPO (at least for now).

> Then pwn2own decided not to include Firefox in another year hacks because Firefox made no improvements at all in previous 2 years.

Uh, [citation needed]?

Firefox was a valid target in Pwn2Own 2021: https://www.zerodayinitiative.com/blog/2021/1/25/announcing-...

Nobody attacked Firefox successfully, but that's hardly a point against it. I don't know what "made no improvements at all in previous 2 years is supposed to mean", either, since we had just released a major overhaul of the optimizing compiler a few months before Pwn2Own: https://hacks.mozilla.org/2020/11/warp-improved-js-performan....

I spent ~5 years working on a COBOL compiler. In practice, production COBOL compilers pick and choose which parts of the standard they choose to implement based on customer demand, and there's not a lot of demand for most of the new stuff. To a first approximation, almost all COBOL code is legacy code in maintenance mode, and I would guess that a significant fraction of COBOL running today has barely been touched since the 90s, if not earlier.

There's no magic answer here. We track various metrics (page load time, responsiveness, gc pause time, and so on) in telemetry. (You can poke around at telemetry.mozilla.org.) We have some page load benchmarks that load a recorded copy of various pages and track how long it takes to load. Sometimes capturing a profile of a particular site can help. Of the big JS benchmarks, we've found Speedometer to be the least bad, because there was at least an effort to mimic actual websites.

Microbenchmarks are tricky because it's very easy to measure something other than what you intend: differences in inlining heuristics, say.

In the long run, I expect performance-critical websites to migrate to webassembly.

> Has there been any consideration to create AI to work around pathological cases in tracing?

None of which I'm aware.

> If compiling is done on a secondary thread, aren't you then reaping all the benefits of JIT optimization while still not paying the penalty on the actual JS execution thread?

Compilation can be done off-thread, but to decide what to compile you have to do some on-thread data collection, and until you're done compiling you have to run in a lower tier. Also, you usually have to do some work to make sure that your speculative optimizations are correct: for example, an inline cache needs to verify that you've got a cache hit.

> Why throw away optimized code? Why not add a type check on the parameters and then dispatch to the highly-optimized code version based on the parameter count and types?

Optimized code is optimized under a certain set of assumptions, which can be propagated through a function. For example, if the value that we read from an object's property is always an int, then we can do integer arithmetic with it afterwards. If we suddenly load a string or a double, then a bunch of downstream code will also be invalid. It's cheaper and less error prone to start over, rather than trying to track all the places that depend on a particular piece of information.

If you only specialize individual operations based on the type input, and don't propagate type information between operations, they you've basically got polymorphic inline caches. That's what we use in our baseline compiler.

SpiderMonkey caches pre-parsed bytecode for scripts, which can immediately start executing in the interpreter. (I believe roughly the same is true for other engines. See [1], for example.) The tricky part of caching compiled scripts is that they generally refer to various runtime state. For example, a compiled script might include a check to verify the shape / hidden class of an object. If you want to cache that code, you also have to cache the shape tree. Multiply that out by all the runtime state that is referenced from compiled code, and it's a hard problem. We've talked about simpler mechanisms like caching function hotness / the number of iterations before our profiling data stabilizes, to trigger compilation earlier, but we haven't gotten around to implementing it yet.

[1]: https://v8.dev/blog/code-caching-for-devs

Eh, with the benefit of 14 years of hindsight, I want to push back on some of the things in that talk. (Context: I work on SpiderMonkey.)

First, all the stuff about tracing and trace trees is kind of obsolete. SpiderMonkey abandoned TraceMonkey a long time ago. (To the best of my knowledge, V8 never implemented a tracing JIT at all.) The problem with tracing is that you can get really good performance when everything goes right, but it's brittle. There's a reference in the talk to how implementations of the Game of Life can have exponential blow-up, for example. You can usually fix any individual pathological case, but the inherently exponential number of possible paths through a program make it difficult to completely eliminate weird performance cliffs.

If your goal is to maximize performance on a known set of benchmarks, go wild. If you want a robust engine that can handle whatever weird code the web is throwing at you, tracing JITs are (as far as I can tell) a dead end.

(Counterpoint: LuaJIT seems to be doing alright with tracing, although it may just solve the problem by punting to the programmer: https://github.com/lukego/blog/issues/29. That's more feasible when you don't have multiple engines with performance cliffs in subtly different places.)

Second, the idea that JIT-compiled code can be faster than AOT-compiled code has been floating around for a long time, but I don't think it really holds in the general case. Doing work at runtime isn't free: not just time spent compiling, but also time spent profiling and validating that your speculative optimizations continue to be correct.

SpiderMonkey had a top-tier optimizing compiler, IonMonkey, that got pretty darn close to native code on hot benchmark loops. We tracked whole-program information to ensure that type checks could be elided in inner loops. (For example, if the `x` property of a certain set of objects only ever contained 32-bit integers, then we could unbox it without checking the type. If any code elsewhere ever stored a non-integer value in that property, we would notice and invalidate the optimized code.)

We threw IonMonkey away, because it was too brittle. In practice, real-world code falls off the happy path often enough that we got better performance by accepting that even your highly optimized JS code will include some runtime checks. Invalidation and recompilation are real costs. So is the upkeep of all the global data necessary to support Ion. There's an engineering tradeoff between pushing up the performance ceiling and bringing up the performance floor; we've been happy with our choice to shift focus more towards the latter. Our numbers are down on artificial benchmarks, but it seems to have paid off in real-world performance. (Also, bugs in the optimizing compiler are significantly less likely to be exploitable.)

A lot of really smart people have done some incredible work on the JVM. Nevertheless, I'm still not aware of any code written in Java instead of (say) C++ or Rust because Java was faster. I think it's more accurate to say that JIT compilation can be fast enough that the other advantages of the language can make it the right choice.

The correct answer was not provided as a multiple choice option, which means by definition that 0% of test takers selected it.

In fact, it already happened to this proposal, which started life as `Array.prototype.item` before it turned out that some libraries were using the presence of a `.item` property to duck-type DOM collections:

https://github.com/tc39/proposal-relative-indexing-method#we...

Roughly speaking, '?' means that the operation might throw an exception, which should be propagated if thrown, and '!' means that the operation should never fail. (In Rust terms, '? ToObject(this value)" -> "ToObject(thisValue)?", and "! ToObject(this value)" -> "ToObject(thisValue).unwrap()".)

See: https://tc39.es/ecma262/multipage/notational-conventions.htm...

As an educated guess: Sparkplug is very similar in design to SpiderMonkey's baseline compiler. I ran a quick test a few weeks ago to see what the performance delta was between SM's baseline compiler and baseline interpreter, and going from the interpreter to the compiler was about a 2x speedup. Based on that, I'd expect that you're looking at a ~2-4x speedup from the interpreter to Sparkplug.

Edited to add: Which is a very good result! Congrats to the V8 team for landing this!

In the document here[1], there's a picture of Gandalf the White at the bottom with the caption "Sparkplug: I am FullCodeGen. Or rather, FullCodeGen, as it should have been".

[1]:https://docs.google.com/document/d/13c-xXmFOMcpUQNqo66XWQt3u...

For the record, I suspect "Manis Heath" is in fact "ManishEarth", aka Manish Goregaokar.

The logo!

This isn't a major change; we're just polishing up our outwards-facing information a bit.

The announcement is here: https://support.mozilla.org/en-US/kb/what-happened-firefox-s...

Quick summary: it was being used for malware and phishing, aggravated by the trustworthy-seeming firefox.com URL.