How much Rust in Firefox? | Hacker News Reader

How much Rust in Firefox?

4e6.github.io

177 pointsushakov84 comments

This —nearly 10%— is a lot for a one of the oldest and biggest publicly available C++ codebases.

(First real public commit in gecko-dev is the historic open sourcing of the engine in 1998!)

https://github.com/mozilla/gecko-dev/commit/3b56a9af51519d2e...

Wow, that is some old farty code! "Free the lizard" indeed, many are we who are happy that it happened! So thank you Mozilla.

Has anyone tried to build that commit in modern days? Would be interesting to see how it runs and how little the existing web would work with it. Maybe HN would load, but most other sites wouldn't. Since TLS wasn't defined until 1999, I'm guessing you would have to run a HTTPS->HTTP proxy first, as HTTPS wouldn't work.

> Has anyone tried to build that commit in modern days?

I was almost tempted to try, but I think I'd rather sleep tonight instead.

This page actually crashes safari in iOS! Works all good on Firefox on same device!

> This page actually crashes safari in iOS!

FWIW, neither[1][2] crashes the current Safari for iOS (15.4).

[1] https://4e6.github.io/firefox-lang-stats/ [2] https://github.com/mozilla/gecko-dev/commit/3b56a9af51519d2e...

A large diff could crash a browser on an older or weaker device. I don't think Safari/WebKit has anything to do with it, though.

Firefox on iOS is a Safari reskin

It's interesting how the C++ files used .cp for the file extension instead of the usual .cpp nowadays. I did some brief research, and my understanding is that .cp was used by the CodeWarrior IDE, which has now been replaced by Eclipse.

For the longest time, ".cpp" was the only suffix MSVC would compile, so that forced anybody doing cross-platform to conform. Recently, MSVC got an option to accept other suffixes, but the damage is done.

The obviously best choice is ".cc": your finger is already on the "c". But IDEs have led people to care less about ergonomic code organization.

Can you double tap a letter quicker than hitting 2 letters on 2 different keys? This is hair splitting, but I don't think 'cc' is necessarily better or faster than 'cp'. Maybe better than 'cpp' but that's an additional letter, AND having to "double tap" one.

It must be said, most of the C++ is very bad C++ code. It would be an overwhelmingly more effective use of developer attention to upgrade or replace old C++ and C with modern C++ than to rewrite them in Rust. In many cases, the existing code would be upgraded in place using modern constructs, but much of the C++ is really lightly-rebranded C.

Another way to look at it, for the comparable language C, C++ and Rust contribute to ~50% of code in Firefox and 10% of it are Rust. So that is roughly 20% already. And that is a lot higher than I thought. I guess they will replace C, and C++ bit by bit in the code base.

Of all code in Firefox, only 50% of it is "systems language" code which includes C, C++, and Rust. Therefore "10% of all Firefox code is Rust" projects to "20% of systems language Firefox code is Rust".

It took me a minute to parse your post, so I restated it.

I agree that 20% seems quite a big larger, especially in a large multi-million LOC codebase like Firefox.

Note that this is across all platforms in the repository, so it is skewed relative to the build for a specific platform.

As an example, Java is in there for Android. There is also a lot of JavaScript that isn’t shared between desktop and Android.

Thanks, I was wondering about Java in the context of desktop Firefox. I think at least the latest Firefox on Android is mostly Kotlin and uses Gecko components which might use some glue Java code.

Firefox to my knowledge doesn't use much kotlin. Checking searchfox it seems the only uses of kotlin are in some geckoview tests, while the Android frontend including geckoview are still plain old .java.

The one other place I know of (other than android stuff) where java is used is the html parser (tokenizer really), which is written in java and then used to generate a C++ version.

I was thinking about https://github.com/mozilla-mobile/fenix - bunch of Kotlin there.

This is incorrect. The front-end for Fenix uses the Fenix and Android-components GitHub repos.

mozilla-central only contains GeckoView (the embedding API for Android) which uses Java for the API itself and Kotlin for all the unit tests.

You and blinkingled are both right of course. I totally blanked out that the mobile frontend code is now in it's own repository and indeed mostly kotlin.

Being in a different repository, it didn't show up in OP's stats, either.

Thanks! I was wondering about that, especially why it had so much Java in it. My theory was that it was still there from the applet days, although it still seemed like a lot more java code than I would expect since the local JVM did the executing.

Dang, this is making me a little nostalgic. I actually really enjoyed writing Java Applets. I'm sure if I went back to some of my old codebases though I'd be thanking my lucky stars for React and LiveView.

Also, this includes ASM by lines of code, and ASM has very short lines, so it's arguably over-represented (although LOC is a very vague metric anyways).

I wonder how much of that ASM is for JIT compilation, and how much is for stuff like fast media decoding.

I wonder what this looks like if you include all C, C++, and Rust dependencies. Is Mozilla more likely to spin out a crate than keep Rust code in tree?

Original goal of Rust, from what I've read, was to basically replace highly concurrent, critical modules in a C++ codebase.

I'm sure everyone who worked on Rust have their own "goals" with the language, that also changed over time. The language and ecosystem is now big enough that there isn't a single reason for Rust existence anymore, there are many.

But according to the OG author of Rust, the elevator firmware where he lived crashed often, so: "I'd been sad about software and thinking of doing a language for a while, but "elevator can't even go" just seemed so pathetic.". Hence Rust was born. See https://twitter.com/graydon_pub/status/1492634815748739077 and https://news.ycombinator.com/item?id=30321747

I'm curious what the 0.8% of assembly would be for, would that be maybe for things like video decoders.

JS engine?

I’m AFK but, almost certainly.

Most is in media codecs (like libvpx, dav1d and jpeglib-turbo), some is in nss (encryption), and some is in the JS engine (part of libffi, enabling privileged javascript code to load and call into non-js library code) and a tiny bit in nspr ("portable runtime", to implement things like platform-specific AtomicIncrement).

The actual JS JIT will produce platform specific low level code, but it's not really written in asm (other than some minor snippets maybe), so it doesn't really show up in such LoC stats. The engine itself is mostly C++, but if I remember correctly some Rust too in specific places (like a wasm compiler).

https://github.com/bolucat/Firefox/search?l=Assembly

Codecs, language bindings

I was curious what the lang stats would look like if charted over time. So I did it. This is a stacked area graph of all languages in the firefox repo, calculated by running tokei on one commit every week from the date of the initial commit in 1995 until today. This took about 12 hours running automatically overnight on my PC. Displayed using plain JS.

https://infogulch.github.io/firefox-lang-stats/

It's very rough but it's a good PoC. Note that you can zoom in by selecting a time span. It looks like there are some discontinuities, but I think they are caused by a combination of the somewhat arbitrary one-commit-per-week selection, combined with merging between multiple root commits (firefox and servo it seems), and this repo being derived from a hg. I suppose a different commit selection query would alleviate this.

I created a separate generic lang-stats repo template ( https://github.com/infogulch/lang-stats ) that I hope could be used to easily generate and maintain a chart like this for any repo, after it's been polished up a bit.

note that it is measured in SLOC.

You can do in 500 lines of rust what might take 5000 lines of C.

I would be interested to see this as a stack chart showing overall LoC.

What's the difference between LoC and SLoC?

Comments, I believe.

Something that would be cool to do, though I'm not familiar enough with build systems and profilers to do it myself would be mark which portions of the generated binary comes from Rust/C++/C and then instrumenting the executable to measure how much time is spent in each of those areas for a typical use of the browser

I bet you could do this with user space probes but it would be somewhat difficult so probably not worth the time.

Will Rust replace all C and C++ in FF codebase?

Edit: Sorry if this was stupid question.

No, the plan is to deploy it in targeted areas where doing so would have a clear benefit.

https://wiki.mozilla.org/Oxidation

Thank you.

Does anyone know if this project is still on going? I saw no plan for 2021 and no updates after version 73 or so.

I think the major integrations of pieces of servo have been done, but that new projects and rewrites of existing components will continue to evaluate Rust as an option going forwards.

Yes it is. WebRender shipped a few releases back.

Never.

There's no current plan to do that, but never is a long time.

Only 9.9%? That's kind of surprising. You frequently hear the refrain "why don't you rewrite it in Rust, like Firefox did" in response to security snafus, so I assumed that FF had more Rust in it.

I don't think they rewrote anything in Rust. Rather they write new code in it.

https://www.reddit.com/r/rust/comments/tioccm/language_detai...

"Only 9.9%" is 3 million lines of code. I'm not sure your expectations of a reasonable oxidation rate is in line with reality. Most big "rewrite everything at once!" projects fail; Rust is designed to enable rewriting incrementally which necessarily takes time.

Plus it's closer to 20% of all systems language Firefox code is Rust (see other discussion itt).

so around 1/5 of the "C/C++/Rust/Assembly" part is now Rust? That's more than I expected. Also, I imagine every line of rust is several lines of C or C++ removed.

You could imagine that, but would be foolish to. Rust is not denser than C++.

Do you have any numerical evidence for that? I would suspect it is slightly denser.

In my experiments from a few years ago, they were exactly even, in both size and speed. Those experiments were far from comprehensive, and both languages have changed since, but not really much. The differences between coding styles of different people are generally much larger than any difference I could imagine showing up in real programs.

What we can say is that both C++ and Rust can be much more concise, much more readable, and much less less prone to trivial mistakes than C or C-like C++.

While it's true that you can write very terse modern c++ my guess is that a lot of c++ code in the firefox codebase is quite old and it would surprise me if it's not closer to plain C in verbosity. Which definitely isn't as compact as Rust.

True, modern C++ is usually quite a lot more compact than older styles, and than C and might-as-well-be-C.

The language density isn't everything. The biggest benefit of Rust over C++ is the module/package system. if you can replace a 50 line utility with a crate (1 line of toml) that makes a big difference. You'd rarely add a library dependency in C++ to use a small utility function.

I wish Servo project would have continued to enable a real browser.

What’s Java? Android native UI?

And why Python? Does Firefox come with a Python interpreter?

Python is for build tooling I think

Correct, the build system is written in Python, with the definitions of rules for each module in files such as https://searchfox.org/mozilla-central/source/toolkit/library...

It’s entirely for build and test. No python actually ships with the browser.

It’s for GeckoView, the embedding API for Android. (I used to work on that team)

5 million lines of HTML?

Most likely due to the tests (Web Platform Tests and Mozilla specific ones).

Also desktop UI.

I would love to see how that pie chart changes over time.

It's not an animated pie chart, but you can look at the complete data here :

https://docs.google.com/spreadsheets/d/1flUGg6Ut4bjtyWdyH_9e...

Source: https://wiki.mozilla.org/Oxidation

They stopped updating it in July 2020, but this is that chart with regards to Rust vs C++ in Firefox: https://docs.google.com/spreadsheets/d/1flUGg6Ut4bjtyWdyH_9e...

Because this site is actually driven by github, of course the weekly raw data that drives it is in a git repository, so you can dredge that repository for the weekly changes and build your own charts.

But in essence things have not changed dramatically in some months, so far as I know there are no current Firefox pushes to land big pieces of new Rust, or to Oxidise large pieces of Firefox.

I'm assuming that Python code isn't shipped in python builds, as that must surely require bundling the Python interpreter with Python.

The python code is mostly really the build system and build support tooling and test systems, and some other "helpers", indeed. It's not shipped code.

I’d be curious to see this over time. Definitely doable via git

What's all that javascript for?

I believe most of the UI and extension stuff is done in JavaScript.

For me, the more important question: “How much Politics in Firefox?”

It was the politics that destroyed Firefox, it wasn’t the code base.

Even though there are some politics involved, "destroyed" is far from correct. I would say chasing corporate dollars had more of an affect, but someone has to pay the bills.

More than there should be, but it's their party.

Reply on news.ycombinator.com