You could frame it as a "failure of JITs in the 2010's". JavaScript isn't a good language to write the TypeScript compiler or a linter, because v8 isn't fast enough.
The semantics of JavaScript do not allow v8 to be fast enough.
IIRC this was precisely why the Dart project was started more than 10 years ago by the original v8 authors. They were spending a lot of time looking into why real world web page performance was falling off various cliffs in v8. They realized they needed to change the LANGUAGE in order to be able to write fast programs. A major use case was developing programs like Google Docs and GMail in the browser, which had to compete with native programs written in C++.
JITs are fast in common cases, but they not only have big costs in terms of memory (code storage) and startup time, but they're hard to ENGINEER with!
Similar story with Python tooling -- the linters, formatters, and type checkers are quite slow due to being written in Python. mypyc gives a bit of speedup, but it still uses the Python runtime.
Related story from yesterday: "Even the pylint codebase uses Ruff" (linter in Rust)
https://news.ycombinator.com/item?id=35035618
---
Emery Berger has memorable framing of this -- "Perl, Python, PHP, and JS are the irrational exuberance" languages.
https://www.sigarch.org/from-heavy-metal-to-irrational-exube...
That is, he says that in the 90's and 2000's, we thought that clock speeds would continually increase, and JITs would get better, and so we could design language semantics without regard to performance -- language that almost REQUIRE slow implementations.
(I think his take is about 50% true. The other 50% is that dynamic languages simply allowed people to produce popular and useful software at a greater rate, especially for the web, so we ended up with a lot of software written in dynamic languages! Doing web apps in Java vs. Ruby/Python/JS is a huge difference in productivity, and I'd say you often end up with a BETTER result, due to increased iteration / fast feedback.)
---
This also tracks with my experience with https://www.oilshell.org, where we reverse-engineered the shell in an experimental fashion with Python, and then evolved that implementation into a statically typed language that generates C++ (using MyPy, ASDL, and algebraic data types).
Oil Is Being Implemented "Middle Out" :https://www.oilshell.org/blog/2022/03/middle-out.html (and many other blog posts)
This core of the program is the elaborate and strongly typed "lossless syntax tree", which is basically what's used in linters and formatters.
Even though I've been using both C++ and Python for >20 years, I was a little shocked how much worse Python is for AST- and graph-based workloads.
I'm looking for references/measurements specifically on these types of workloads. I think a lot of papers about JITs are misleading with respect to them, or at least you have to read between the lines.
I'd say that Python and JS are 10x as slow as native code for "business" and "web app" workloads, and I've never had a problem with them in those settings. Quite the contrary, I've actually sped up poorly working code in static languages with Python. If you're within 10x of the hardware's performance, you're doing VERY WELL compared to "typical software", which has layers of gunk and can be 100x to 1000x too slow.
But bare Python and JS (no libraries) are closer to 100x too slow for ASTs and graphs. This is because of all the allocation and GC overhead -- in both time and space -- in addition to dynamic dispatch, etc.
(The funny thing is that Oil is now the most statically-typed shell implementation, even though it's nominally written in Python :) It uses fine-grained static types, where as shells written in C use a homogeneous "WORD*" representation in C, and strings with control codes embedded in them for "structure" and "types". I should probably write a blog post about that ...)
---
To shine some light on the other side, I'm still a bit skeptical of Rust specifically, because:
- memory management is littered all over the codebase.
- Borrow checking seems to work better for stateless/batch programs (it thinks about parameters and return values), but linters and type checkers for language servers are STATEFUL: https://news.ycombinator.com/item?id=34410187
- Many ASTs are actually graphs
- pattern matching can't see through boxing apparently ?
Also, the author of esbuild tried BOTH Rust and Go, and ended up with Go. IIRC it performed better because it didn't have deterministic destruction on the stack -- GC was more efficient?
It seems like a language with both garbage collection and algebraic data types would be nicer, but neither Go or Rust fit that description!
Rust makes a lot of sense for kernels and so forth, but for language processors -- especially stateful ones (which includes the Unix shell!) -- I think GC is still a big help. And we know already know how to make GC fast for that use case, i.e. you don't need a GC that scales to 1 TB of memory on 128 cores.