Hacker News Reader

Comment by tombert | Hacker News Reader

I would be very curious to side by side performance benchmarks between this, GraalVM, and vanilla JDK. My gut tells me (with no data to back this us) that the vanilla JVM will inch ahead once it's paid the cost of starting up but I would be interested to see how wrong I am.

View on HN

Lead dev here. We've run a couple of benchmarks internally just for kicks. To create a fair comparison, you have to run the Hotspot JVM with the -Xint flag, which says interpret only. Right now our performance is anywhere from 15-25% of the speed of Hotspot with -Xint on small benchmarks. We figure that the use of go alone creates some important portion of that overhead when compared with the C++ of the Hotspot JVM. We're guessing that a well-optimized Jacobin interpreter will eventually get to 50-60% of the Hotspot's -Xint speed.

But we first want to get feature parity, before pivoting to performance. When we have feature parity, we'll run the Computer Language Benchmarks and post the results. That'll be fun to see!

Is that fair? The whole point of the tradeoffs made by HotSpot is that it's optimized for JITing.

Imho there is no way to do a fair comparison, as both implementations have completely different goals.

One thing that would be interesting is comparing the performance of some really simple command line program (e.g. `ls` or `cp` for small files) between Hotspot in interpreter mode, Jacobin and GraalVM native.

Do you (can you even have in go?) have a direct threaded interpreter? That alone may give a 2x performance difference last time I checked, or is that no longer the case?

I don't think you can even implement indirect threading in Go. Go supports neither computed goto nor mutually recursive tail-call optimization (only self-recursive).

Jump tables for switch statements was only implemented last year. If you squint that's close to indirect threading, but still with at least one unnecessary conditional per op.

For the curious, here's their giant switch: https://github.com/platypusguy/jacobin/blob/c508ec50f55ef381... In practice compilers have always been finicky when it comes to coaxing them to emit jump tables from switch statements, and I bet this is especially truly for Go.

Yeah, I think any bytecode interpreter ends up with a giant switch in the critical path at some point :)

Around the time that change was made to Go, Andrew and I were looking at this and wondering how big of a performance hit it was and if there were a better way to structure that. I had a hunch that the compiler should be smart enough to not compile that as a switch/giant if block, and a quick trip to a disassembler showed it using binary search. This commit: https://github.com/golang/go/commit/1ba96d8c0909eca59e28c048... added the jump table and has some nice analysis on where it makes sense to do binary search vs jump tables.

As far as I can tell, with certain restrictions (that are fine in this case), it is pretty reliable at optimizing giant if/else blocks and switches.

That's fair; I thought maybe the use of the Go GC might make the results a bit more interesting.

Go has a much more primitive GC, so I wouldn’t expect a positive result from that itself.

I'm not the author, but I have contributed a few things to the project over the past year. The performance isn't anywhere close to the vanilla JVM - several times slower at best. It is entirely interpreted and there aren't any of the optimizations that have made their way into the JVM over the decades it has been around.

It has been a fun project to play around with for someone like me who thinks this kind of stuff is fun and interesting but will probably never get a chance to work on it full time. Cool to see it noticed, though!

This JVM has no JIT, so OpenJDK and Graal will rather more than inch past it. It's not even a given that it will start up faster, since i don't believe it has been optimised nearly as hard.

You'd probably get better results writing a Go compiler in Java ...

Wrote a Go compiler in Java, used it to compile this JVM implementation written in Go

Great, now just use this JVM to run your compiler :)

Ouroboros

Lambda the ultimate

Well, you'd probably get pretty good perf with graalvm truffle, but certainly the building blocks of golang shouldn't be that problematic to implement in java given that the go language model is more straight-forward IMHO. I wouldn't assume it'd beat pure golang compilation but who knows.

The startup cost is dominated by class loading, which Jacobin will have to do more or less the same as OpenJDK [*]. GraalVM of course profits from AOT compilation.

[*] until the latter will implement condensers: https://openjdk.org/projects/leyden/notes/03-toward-condense...

Didn't they introduce some new format with Java 9 and jlink to store modules more efficiently? Why hasn't that fixed class loading performance?

The issue isn’t the format. Class loading includes bytecode verification (verifying the type soundness of whatever code the class files contain), and then executing the initialization code defined by each class (static code blocks etc.).

The optimizations in JDK 9 are for reducing the file size of the JDK (in terms of number of classes) when distributed with a standalone application, but that by itself doesn’t significantly affect startup time I believe, because classes are lazy-loaded only as required in any case.

The jimage format does optimize startup a bit because it builds a perfect hashtable to go from class name to classfile bytes. On the classical classpath that requires an O(N) scan of the JARs in the app.

Does a JVM really need to re-verify the standard library it ships with?

It’s actually the default to not do that (-Xverify:remote). So I was probably wrong about bytecode verification being a relevant part of startup time, unless you count the application itself.

The modules system is for better encapsulation and the ability to produce a JVM without features you don't need.

Faster start up time is in the works with projects like coordinated restore at checkpoint which will let you start a JVM up in an already "warmed" state.

https://github.com/CRaC/docs

If I understand the README correctly, CRaC in an abstract API that delegates to an implementation (CRIU) which is Linux-specific, described in its README as "the never-ending story, because we have to always keep up with the Linux kernel".

So this wouldn't help if I'm deploying a GUI app for Windows, for example.

Correct, it might, someday, become part of Java SE. If that ever happens I would expect it working on every platform Java supports would be a prerequisite for that.

Right now it's primarily useful for reducing start up times in AWS Lambdas and auto-scaling workloads.

It improves it a bit, and AppCDS improves it a lot (~30% startup win), but neither feature is widely used as it changes deployment workflows.

I see it more as an niche opportunity to integrate bits of Java code in Go applications. The start-up costs can be "paid" when the app starts. Performance will be bad, but it might save some rewrites (provided they can get things like db connections going).

Yeah that makes some sense; there's probably millions of lines of Java code out there where performance doesn't really matter, and so aren't worth rewriting. Being able to embed a JVM just to run those from Go projects isn't a bad idea.

Tangential, but a long time ago, I wanted to reuse a node.js slug library in .NET. I thought about trying to port it at first, but then I realized that this actual job didn't need to be terribly fast, so I instead embedded the Jurassic JS library into my code, and was able to load in the slug library that way directly). It wasn't especially fast (but actually faster than I thought it would be!), but it was certainly fast enough, and I didn't have to worry about not having feature parity.

Another good example was IKVM, which basically transpiled Java bytecode into .NET IL. Saxon XSLT used to build on top of that to provide .NET libraries from the same codebase as the original Java version.

(Saxon had since moved to transpiling the source code instead - the languages are close enough that this is fairly straightforward.)

Why not just do traditional IPC, or even a batch process starting up a JVM from time to time when needed?

There are always other solutions. This one might just work in some cases. As I wrote: it's (probably) niche.

Exactly.

The cool thing is that since this uses the Go garbage collector, you can create a very nice Go-native interface to that JVM code.

Clearly this is not going to be a high performance thing. I'd be surprised if it is even close.

It seems the goals for this are purely academic and about figuring out how Java works. They'll probably just do a simple interpreter and not a JIT compiler. That would be good enough for a POC. Additionally, they already indicated that they'll use Go's garbage collector, which won't be setting speed records with this either. And Java's typical usage of memory might actually stress it out a bit. Then there is the standard library which is going to need plenty of support for things like Threads, IO, various synchronization primitives and locks, etc. Doing that in Go is going to be a bit interesting but probably doable. Alternatively, they might just interface with native code directly and bypass the Go ecosystem. They might even reuse some things from openjdk for that. Speaking of which, native code and JNI would need to be implemented anyway.

It looks like a cool research project. But realistically I would be surprised if it could beat OpenJDK in any benchmark (though I don't think that's the purpose of the project).

Writing something which is compatible with a specification is one thing, making it performant is another thing. For example it's not that hard to create a webserver from scratch but making it performant takes quite some effort.

Reply on news.ycombinator.com