Why Aren’t More Users More Happy with Our VMs? Part 2 | Hacker News Reader

Why Aren’t More Users More Happy with Our VMs? Part 2

tratt.net

21 pointsfniephaus7 comments

I find the discussion of warmup time here really interesting, the handwaving away of this something that I have observed when developers make wild claims about the performance of various JIT based languages vs statically compiled C or C++.

Tangentially related to this seems to be the type of optimizations that can be performed on the code as well; I've heard the argument that because JIT is done with the full state of the runtime environment clearly visible (ie. the system is up and running and can be observed during compilation), that better optimizations can be made at runtime.

The flipside of this argument seems to be that static compilation to machine code can allow better optimizations because more time can be spent searching for more expensive/aggressive optimizations since the startup time of the compiled program is not dependent on or constrained by the performance of the optimizer(s).

Muddying the waters even further are languages that are partially compiled (for example, Java to JVM bytecode) in a static compilation pass, and then dynamically JIT'd from bytecode to machine code at runtime.

Whew. I think I need to sit down and have another coffee.

Just to add a bit of color to what you're saying, Java has (historically at least) worked fairly hard to NOT do any optimization in the first pass. JVM bytecode is a fairly straightforward trivial translation of Java code.

JITs do have much more information to leverage for optimization. But they have 2 balancing acts:

1) All the tracking, profiling, and compilation machinery they need to operate concurrently need to cost less than the advantages they bring. There's amortization here in longer running programs.

2) Much of that optimisation goes into bringing languages UP to speed, let alone pull ahead. JITed languages don't just have more information at runtime but typically have much less at compile time.

Take Java and C++ for example. In Java, every call is a virtual call and worse, any call outside of your class can look completly different at runtime (besides the signature) than it did at compile time. Then add all the classloading/bytecode weaving that's actually common. C++ on the other hand knows what's virtual, what's not, what's linked from another lib up front.

Thanks for your informative reply. I'm not much of a Java programmer (my experience is mainly C and more recently C++, plus a lot of traditional scripting languages) - when you say C++ knows what's linked from another library, are you referring to the dynamic linking process, where ld at least looks at all the libraries and resolves the references at link time, is that correct? Whereas Java has to resolve all its references to stuff in .jar files at runtime?

Yep, that's exactly what I was referring to. Java will treat every other class as distant as C++ will with a .so ... But it can also in line into these java classes at runtime.

Yes, although quite interestingly C suffered from the same lack of steady state reaching as the Java code due to something in Malloc (see part 1). As well as suffering from optimizations in a compiler actually making existing code slower (of course in C/C++ new, valid, optimizations also have the side effect of breaking code :( UB values are called poison in LLVM IR for a reason...)

Problem with JIT based languages is that they tend to inhibit the optimizations that developers can make. Mostly because these days the largest benefit for performance is well organized memory access, which is inhibited in the widely used JITs. (e.g. pointer chasing happy java/python/js/ruby) code)

On the other hand JITs can really run with simple optimizations because they can assume (virtual calls assumed to have a single target are run as if they are static calls).

In the end neither of this matters for most developers as things are "fast enough" i.e. See Ruby on MRI, R, or CPython today.

Dunno ... because they ask for 2 cores and you give them 2 over -provisioned vcores instead (each core sliced into 17-24 vcores)?

Or because they ask for 16 GB RAM and you give them 8 GB?

So yeah, this is about programming language vms, like JVM, not cloud vpses....

Reply on news.ycombinator.com