Comment by pizlonator | Hacker News Reader

Comment by pizlonator | Hacker News Reader

Parent Full thread

Those are some really bad benchmarks.

JavaScript is probably like 10x or 100x slower than C or similar languages when you write something bigger and any of the following happens:

- the ratio of code size to run time is too big. Then the jit can’t keep up.

- you use a lot of value types. Them be structs in C, no need for allocation. In JS them be objects. The JS VM will try to escape analyze them, and it will succeed some of the time, but fail enough of the time to cause massive punishment.

- you churn objects while having a large base heap size and the generational hypothesis holds only a bit. Then you’ll wait for the GC a lot.

- you have a class hierarchy with many descendants and you often access properties or call methods on the base type. Then vtables or whatever work great but JS inline caches blow up.

- probably lots of other conditions.

Write enough code and at least one of these will hold and you’ll be slow in JS, fast in C.

(Source: I work on JSC and I implemented a lot of its optimizations. Benchmarks like the ones in this post are the sort of thing my JITs eat for breakfast. It’s cool to see people throwing me softballs but I like to be honest about what the technology I work on is capable of.)

I've been thinking a lot about this over the years. Having been a dyed-in-the-wool Java VM person for years, and then also working on JS, I started to see the marketing speak I blabbed for years just kind of disappear in the face of huge applications.

JavaScript, Java, C#, generally all JITed languages will eventually end up exhausting the size of their JIT code caches. You just gotta hope that your hot paths are contained in there and that the JIT balances its hunger for deep inlining with the need to not overcompile. Because there's nothing worse for a JIT when the entire application is just a tepid soup and there are no hotspots. Then you aren't getting escape analysis, you end up with tons of polymorphism, and generally, performance suffers.

Despite 20 years working on JITs and dynamic optimization, I am more convinced than ever you just can't beat static compilation and programs designed to not over-allocate, to not overabstract with too much polymorphism, closures, and heavy allocation. Programs still need to be a bit miserly to get maximum performance.

With Java there's some relation to the native code, so if you reach that point you can use something like GraalVM to get better performance. In fact there's quite a lot of people who check that with large apps, for most cases JITs perform better in runtime and AoT performs better on startup. But for huge apps this might flip.

But the thing is that Java is pretty close to native. Primitive types, object allocation, threads etc. all correlate almost directly to how CPUs and OSs work. So you get a sense of performance and what's expensive/isn't.

With JavaScript there are so many complex abstractions and constructs it's really hard to get a sense of what the final ASM will look like in an optimal case. Unfortunately, some Java changes (e.g. Valhalla) aim to bring some of that lack of clarity into Java. Still, thanks to typing and a simpler syntax Java is MUCH better positioned here than JavaScript.

I agree that with Java you can get close to native performance if you are careful. (I wrote a MCU emulator that beat its C competitor back in 2005, and that's all low-level hackery). But in practice people write a ton of abstraction in Java and programs are quite trashy (i.e. allocate a lot of intermediate objects that are quickly garbage). The APIs and best practices encourage a lot of overabstraction, IMHO, and that leads to big, lumbering programs that struggle to get back to reasonable performance levels with the prodigious efforts of VMs.

> close to native performance if you are careful

I'd like to add that this argument exists for pretty much all "slow" languages and that "if you're careful" often can mean "throw away all upsides your language provides" [1].

Personally, I find these discussions tiring because the ones arguing such points are usually the kind of person that doggedly clings to the "wrong tool for the job".

For example: Although I see rust as the better alternative in many places, I personally find C++ to still be a better language than Java. That doesn't mean I'll insist that Java has no worth or that every program should be written in lower level languages. Horses for courses.

Why can't we just go the Python way and accept the shortcomings of the languages we hold dear?

[1]: Edit: For GCed languages, this often is "avoid the GC at all costs and essentially write less readable C".

What do you mean by exhausting jit caches? Do other VMs limit their size somehow? JSC basically doesn’t.

That doesn’t solve the problem of course. I have this metric that I use to philosophize about this: SIPS, or static instructions per second. If your program has low SIPS (I.e. it’s the benchmark of HotSpot’s wet dreams or OP’s post) it means that the total code is small and it runs a long time - so the JIT will go on a rampage early on and then you’re good. If the program has high SIPS, then the JIT will still be catching up when the program terminates. Most real things that aren’t web servers have low SIPS.

Reflecting more on this. AFAIR Hotspot does have a hard limit on the amount of machine code it will generate. V8 has a hard limit on the heap size, and code pages are part of that. But both of those are pretty big numbers, so if you are hitting that you are probably huge already. But generally there are counters in the metadata for functions or classes, depending on the system, that limit how many times a given unit might be recompiled, typically to limit the damage deopt loops can do. So programs end up gradually leveling off as they hit those limits.

I haven't really paid attention to V8's policies for years, but generally speaking, it still does this to some extent.

I think of SIPS as basically the resident set size for code. Depending on how the system ages your code, you might be stuck with unused JIT code until it can be recycled, if at all. V8 used to have code aging and the implementation was a huge bug farm. I am not sure if that survived the switch to TurboFan/Ignition. Of course, V8 GC's code about as aggressively as it compacts the old generation.

To what extent writing your code in WebAssembly (eg Rust) can help with those points (eg structs in C argument). It would still run in a JS VM so I'm guessing a bit, but not to the full extent?

WebAssembly helps a lot, but doesn’t solve the jit issue.

WebAssembly also introduces its own issues since it’s a BYORT system (bring your own runtime). So, there’s more to JIT (your language’s whole runtime) and more to hold in memory (your language’s whole runtime). You might say, “but pizlonator, every language has a runtime”, to which I’d say: yes but usually that shit gets shared by every running instance somehow. In WebAssembly every instance pays for its runtime’s memory footprint and it’s quite likely that every instance has a different runtime so the code isn’t shared either. Basically, WebAssembly is knee-capped on memory footprint by design. JS isn’t.

Your comment seems substantially misinformed or inapplicable.

WebAssembly has no truck with JIT. Just-In-Time compilation is all about not performing optimisation ahead of time, but only doing it when the code in question is used, and guiding the optimisation by how the code is being used. In WebAssembly, the conversion from the binary .wasm format to machine code is a comparatively lightweight process with no real optimisation of this form: rather, such optimisations must be done as part of producing that .wasm blob (in regular compilation possibly with profile-guided optimisation to go even further than JIT can).

So instead, when you’re using something like Rust (as distinct from, say, compiling CPython or V8 to WASM), what you’ve got is a fairly small amount of what you’d probably call runtime code (allocator, panic mechanism, str, Vec<_>, some other parts of the std crate), probably something like 25KB (or with a little care and compromise, more like 5KB) except when Unicode tables are required, compiled from WASM byte code to similarly-sized (though I don’t know the real ratio) machine code faster than it can be downloaded. That’s code memory usage; for the data memory usage, well, your Rust/WASM will normally blow your JavaScript out of the water there with much more efficient packing of data into memory, even if you’ve got a fair bit of overallocation.

The fact of the matter is that the runtime parts which can be shared for JavaScript are actually not all that large, and routinely dwarfed by included libraries (React, &c.). WebAssembly is by no means knee-capped on memory footprint; rather, so long as you’re using it in a sane way (Rust, standard WASM optimisation and code-shrinking techniques, that kind of thing), there’s a pebble in the way that you’ll notice if you’re a beetle, but if you’re even rabbit-sized you probably won’t even notice it.

As regards pbadenski’s comment: WebAssembly does not run in the JavaScript virtual machine, it’s a completely separate thing that is merely able to call and be called from JavaScript via a foreign function interface. The backing memory buffer may also be accessed from JavaScript, but that’s immaterial in the consideration.

The wasm blob has to undergo the hardest parts of optimization to get native code from it. You need to select instructions and allocate registers. Those things are time consuming; they are on the same order of magnitude of time consumption than most full compiler pipelines.

Compiling wasm to native is only faster than downloading if you compile without optimization. That’s common in wasm VMs but then there’s an optimizing JIT that runs adaptively later, just like a JS or Java VM would do.

The fact that JS VMs share the ICU implementation between instances is a huge deal. That’s not the only thing that gets shared. Also, it’s not about just sharing code; it’s about sharing memory for the runtime’s state and for allowing elastic reuse of space for objects. In wasm the sharing is page granularity at best.

Reply on news.ycombinator.com