Hacker News Reader

Comment by Spivak | Hacker News Reader

Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say dumping some data structure when it hits steady state to give hints to the JIT the next time it runs?

View on HN

Android does this with ART. They use JIT and profiling to generate AOT binaries dynamically. I believe this also lets Android update the runtime in a way that invalidates the AOT binaries. They are simply be regenerated as needed.

https://source.android.com/devices/tech/dalvik/jit-compiler

Azul's ReadyNow! for their commercial Zing JVM does this: https://www.azul.com/products/zing/readynow-technology-for-z...

If you look for "AOT" or "Ahead-of-time" you'll find examples in both .NET and Java, but as far as I know they're either largely experimental or limited to newer code (not backwards-compatible with all code). But I haven't looked too deeply into it. Dumping data structures to give hints for next time reminds me of something I read recently on the topic, but drat, I can't remember it right now.

Isn't that just compiling, then?

Sure, but something like Java already has to be compiled. Throw a few more minutes in there, maybe run the test suite a couple thousand times.

But many languages with VMs are intended to be portable across processor architectures, and a snapshot would be architecture-dependent.

Also, the charecteristics of your test suite may be very different than how it is run in production.

I would say those are reasonably easy to solve, you just need to offer platform specific trained binaries and add a realistic set of stress tests with which you can train the VM.

That would be great but it’s hard because the trick, at least in JS VMs, is to have the JIT specialize based on the heap.

So you’d need a heap snapshot or some way to link the generated code to a different heap.

Maybe not impossible, just hard enough that it’s not widespread.

It does seem like you could do JIT with a persistent cache which stores the JIT output along with a key that's a hash of all the relevant system parameters like CPU model and VM parameters like heap size. This would mean that the typical case of re-running a program in the same environment would be pre-warmed.

Kind of like memchached, https://en.wikipedia.org/wiki/Memcached

It’s much harder than that because the JIT is speculating on what lots of objects in the heap are doing, including watchpointing them to constant fold properties. It’s not clear what the key should be in that case.

Still not impossible but I want to be clear on what exactly makes this hard. CPU model for example is not what makes it hard.

> including watchpointing them

I assume this isn't literally using hardware watchpoints?

Yeah

Heap layout changes between runs due to OS obfuscation (ASLR).

I would bet that a VM isn’t going to be deterministic enough to produce the same heap twice even without aslr. Just building a deterministic JS engine seems like a super hard problem.

IBM OpenJ9 JVM does that with the AOT feature. It's good for startup time and the AOT code can be still further optimized by the JIT. https://www.eclipse.org/openj9/docs/aot/

IIRC V8 can now provide deterministic JIT+heap images in addition to preparsed bytecode images.

EDIT: worth noting that other JS VMs like JSC have these APIs for bytecode too, but i can’t remember off hand whether they can do generated code too.

The V8 snapshot feature is designed for this. https://v8.dev/blog/custom-startup-snapshots

We can and there are many implementations of it, but the relatively rare application of this approach suggests it's not a big enough win to justify the cost.

I believe this is what Deno tries to do.

https://deno.land/

dotnet does this, caches compiled assemblies then also tunes them. https://www.geeksforgeeks.org/what-is-just-in-time-jit-compi...

Emacs does this. See https://emacs.stackexchange.com/questions/2364/what-is-the-f...

Reply on news.ycombinator.com