IainIreland

One major benefit of compiling to bytecode first is that bytecode is a more convenient shared source of truth.

For example, SpiderMonkey has two interpreter and two compiler tiers. The output of the parser is bytecode. We can interpret it immediately, and it's a convenient input to the compilers. It also simplifies transitions between tiers: for example, when we bail out of Warp, we resume at a specific bytecode offset in the baseline interpreter.

I'm not sure how you would resume at a particular point in the execution of an AST-based interpreter without agreeing on a linearized traversal of the AST, which is 90% of the way to bytecode.

Assuming nothing's changed since I left, the front-end is taken from the old COBOL compiler, and is written in PL/X. It emits WCode for the backend, which is written in C++.

XL C is a separate codebase, with very little shared code IIRC.

We must have just missed each other. I started in January 2014.

I worked on the IBM COBOL compiler in a past life. It's not LLVM based; the optimizer and code generator are derived from J9, the IBM JVM. I would be a bit surprised if they open-sourced it; OpenJ9 is open-sourced, but the COBOL compiler for Z is one of those rare compilers that is still being sold to paying customers.

One of the things I've realized after working on SpiderMonkey for a few years is that simple solutions that accept a little bit of overhead are often faster than more complicated approaches that try to eke out every last cycle.

Math is fast. Stack space is rarely a key constraint. On the other hand, tracking typical argument counts for a function also requires overhead, and compiling multiple copies of a function is also not free. If a particular call site is hot enough that call overhead matters, it's probably hot enough that inlining is the correct answer.

As one additional data point: V8's new approach appears to be very similar to how SpiderMonkey's been implemented all along. Convergent evolution is generally a good sign.

(I work on SpiderMonkey.)

Cranelift was originally started as a project to make a new backend for wasm in SpiderMonkey. It took on a life of its own, and has been transferred by the Bytecode Alliance (which Mozilla is a part of). At the moment it's not mature enough for us to use in production (both in terms of performance and in terms of code churn). We're hopeful that will change over the next few years, but we need to ship wasm support now, so we're sticking with our existing backend.

(We intend to keep Cranelift working behind a compile-time flag.)

In a past life I worked on a COBOL compiler for IBM.

There is a COBOL specification, but AFAIK nobody actually implements it fully. Implementations pick and choose new features based on customer demand.

Also, COBOL is dominated by large legacy codebases, which means that if there's a discrepancy between an implementation and the specification, the users normally don't want it fixed, because they may have written code that depends on the "incorrect" behaviour and it's a lot of work to audit.

IBM built a new backend for its COBOL compiler using the optimizer from their JVM. IIRC by the time I left it generated code that was ~2x faster than the old compiler, but uptake was still slow because of migration concerns. In particular, we spent a lot of time working on features to help guarantee that code with some forms of undefined behaviour would have the same result as the old version.

I believe this is the best account: https://yoric.github.io/post/why-did-mozilla-remove-xul-addo...

(Mozilla employee, though not working directly on mobile)

Mozilla is invested in Firefox on Android for the long haul. We landed a complete re-architecture this year (Fenix). It hasn't been completely painless -- for example, as you seem to have experienced, we've had to split our development time between working on the new engine and maintaining the old -- but Fenix is already better than Fennec in most ways and is significantly easier to improve moving forward.

https://blog.mozilla.org/futurereleases/2020/01/17/a-brand-n...

You aren't allowed to goto a label in a different function.

From the C standard:

> The identifier in a goto statement shall name a label located somewhere in the enclosing function.

`Shall` is a term of art here:

> If a "shall" or "shall not" requirement that appears outside of a constraint is violated, the behavior is undefined.

(Sections 6.8.6.1.1 and 4.2, respectively: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf)

The economy gets worse. Ad spending goes down. Google's search revenue decreases. Mozilla's search deal with Google is less profitable.

The Mozilla Corporation is wholly owned by the Mozilla Foundation, a non-profit organization with the mission of making the internet a better place. Here's the manifesto: https://www.mozilla.org/en-US/about/manifesto/details/

> most of the recent improvements to Firefox were initially prototyped and developed in the experimental Servo engine which was to my knowledge intended to eventually replace all of gecko. Most of the team for that has been let go.

This is not the case. Servo worked on big experimental moonshot projects. They had some major successes (Stylo, WebRender), but the rest of the Firefox team wasn't sitting around twiddling their thumbs. I don't think Servo ever made up more than a small fraction of the overall Firefox workforce. They had to pick and choose their projects. Note, for example, that Servo embeds SpiderMonkey instead of writing its own JS engine from scratch: https://github.com/servo/rust-mozjs

Don't worry! SpiderMonkey still has additional cool projects coming down the pipeline, none of which are dependent on Servo.

An opposing point of view, from somebody at Mozilla who cared about the browser and wrote the code: https://yoric.github.io/post/why-did-mozilla-remove-xul-addo...

Mozilla employee here (albeit not on the mobile team):

Relax, everybody. Fenix is a major rewrite. It takes time to reimplement all the WebExtension APIs. Nevertheless, the intent is to eventually add support for all the APIs that make sense on mobile. (See here, for example: https://bugzilla.mozilla.org/show_bug.cgi?id=1632626#c1, or just scroll through the list of bugs here: https://bugzilla.mozilla.org/buglist.cgi?product=GeckoView&c...)

If you go back and read old HN threads from the first public release of Fenix, people were afraid that the lack of extension support was a sign that Mozilla had given up on adblockers. I hope it's clear by now that's not true. (From what I recall of lunchtime conversations, back when those were a thing, the plan for extensions was "well, we're definitely not shipping without uBlock Origin, and we'll see what else we can get working from there".) Similarly, the fact that parts of the WebExtension API are not currently available doesn't mean Mozilla has given up on it. It just means that things take time. (If it were easy, other mobile browsers might support add-ons!)

It takes a lot of resources to support two parallel codebases. At some point you have to rip off the band-aid. The sooner Fennec is cut loose, the more effort we can dedicate to Fenix improvements (like improving WebExtension support). The transition was always going to be a bit bumpy, but things should hopefully get better from here.

The TC39 minutes are all online [1]. You can see the initial introduction of lookbehind assertions [2], the meeting where they reached stage 3 [3], a status update while waiting for more implementations [4], and then the eventual agreement to move them to stage 4 (full standardization)[5].

They reached that point because: a) everybody agreed that they were a good feature, with precedent in other languages; b) the spec text had been scrutinized by the people who like to spend time scrutinizing spec text; and c) two implementations (Moddable and V8) had successfully implemented them. That's the process, and it was followed to the letter.

Does that put a burden on other engines to keep up? Sure. But that's what we signed up for. The webcompat bug you linked is from April 2020, more than two years after lookbehind assertions were standardized. That's nobody's fault but ours, and we did this project so that we don't end up in a similar situation in the future.

I'm not sure what your source is for JS regexps being impossible to implement from the spec, but if you have specific points of concern, you should open an issue [6]. Writing a spec is hard work, and things get missed, but the people working on this are genuinely trying their best.

(As far as I can tell, Canonicalize is what everybody thought they were implementing. V8 tried to get clever with ICU and ran facefirst into some of the dark corners of Unicode.)

1: https://github.com/tc39/notes/tree/master/meetings

2: https://github.com/tc39/notes/blob/master/meetings/2016-11/n...

3: https://github.com/tc39/notes/blob/master/meetings/2017-03/m...

4: https://github.com/tc39/notes/blob/master/meetings/2017-05/m...

5: https://github.com/tc39/notes/blob/master/meetings/2018-01/j...

6: https://github.com/tc39/ecma262/issues

tl;dr: The spec exists for the web; the web does not exists for the spec.

> I have not confirmed this but I have been told the regexp portion is tightly controlled by Google, more than most parts of the spec. When v8's regexp implementation is found to deviate from the spec, the spec gets changed. This makes sense (why risk breaking websites), but it illustrates how little weight the regexp spec actually has.

A few responses:

1. This was not at all my experience working with the V8 team. For example, when I noticed that Irregexp did not properly implement the Canonicalize algorithm for non-unicode case-insensitive comparisons [1], they were happy to accept my patches [2][3]. Nobody ever suggested changing the spec.

2. Nothing changes in the JS spec without consensus from all the major stakeholders. See, for example, the second sentence in the TC39 process document here [4]: "The committee operates by consensus and has discretion to alter the specification as it sees fit." Mozilla can block (and has blocked) proposals that it feels are bad for the web platform. Google has no special power to bend the committee to its will.

3. Another way of saying "when V8's regexp implementation is found to deviate from the spec, the spec gets changed" is "when the spec is found to deviate from the consensus of implementations, we correct the spec". The Chakra team's post notes: "In practice, all browsers accept regular expressions such as `/]/` and web developers write them." The purpose of the spec is to make the web work better by maximizing compatibility between implementations. In cases where all implementations agree, and real websites depend on that behaviour, the spec should change to match reality.

Don't get me wrong: Chromium monoculture is a real problem. If we thought that Google was going to add a bunch of new regexp features without going through the standards process, or if we had plans to prototype regexp features that we feared Google wouldn't accept upstream, we might have made a different decision. But regexps are not a plausible battleground, and the compatibility benefits of sharing code outweigh the harms.

PS: regress looks really sweet, and I am excited to see how it turns out.

[1] https://tc39.es/ecma262/#sec-runtime-semantics-canonicalize-... [2] https://github.com/v8/v8/commit/3fab9d05cf34a7f0bc0e9405729a... [3] https://github.com/v8/v8/commit/b65fcfe92566415c8aae373d5fb5... [4] https://tc39.es/process-document/

(Author here)

I love Rust. I've lost track of the number of times I've stared at a chunk of C++ code, considered how much nicer it would look in Rust, and sighed. At the beginning of this project, we did consider whether it was the right place to use Rust.

There are no existing JS-compatible regexp crates. BurntSushi's regex crate is great, but finite automata don't do backreferences, which is a dealbreaker for JS. After this code landed, somebody brought https://github.com/ridiculousfish/regress to my attention. It looks promising, but still has a long way to go before it's production-ready, and it didn't exist when we made our decision.

If we had written our own replacement, it would likely have been Rust. SpiderMonkey has a lot of cross-cutting issues (GC especially) that make it hard to replace individual C++ components with Rust, but the regexp engine has a pretty clean API boundary. It's the same reason that it was feasible to swap in Irregexp in the first place.

Ultimately we decided that writing a new engine wasn't the best use of time. A regexp engine is a complicated beast. Writing a new high-performance, JS-compatible engine in Rust could have been person-years of effort, with a long tail of corner cases and performance issues. We haven't had many memory safety bugs in regexp code. As sstangl points out in a sibling comment (hi Sean!), doing JIT compilation undermines some of Rust's safety guarantees.

When it comes down to it, the regexp engine is not a place where SpiderMonkey is looking to push the state of the art. We have to be reasonably fast and feature-complete, but beyond that nobody is going to notice marginal gains. There are higher-leverage opportunities elsewhere.

Last I heard, WebReplay was not on an official roadmap (in the sense that we do not yet have a target date for shipping it in release). That said, I'm pretty sure that the macOS-only restriction is just because that's what the primary developer is using, and it's a higher priority to get the system-independent parts working first. If/when WebReplay is ready to ship, I can't imagine a world where support isn't eventually added for Windows/Linux (although it could be a phased rollout, like WebRender).

Note that you can see exactly what is contained in the telemetry ping by going to about:telemetry.