Hacker News Reader

Comment by langsoul-com | Hacker News Reader

Interesting how Java is so fast. Among the fastest of c, cpp, asm, go and rust.

How?

Because the JVM has had 27 years of R&D work done to it. If you have a really hot path a JIT will typically continually recompile it in the background with more and more optimization work and start putting the loop through that progressively more optimized code. When you're doing gigabytes of FizzBuzz the JIT will quickly have something pretty comparable to the best native code.

oddly enough fizzbuzz stuff likely needs an on the stack code replacement, i.e. compiling and replacing the currently executed code, which tends to be subpar.

The submitted code snippets for Java do not need OSR, though... yet they can be improved further, e.g. should drop the use of String entirely (which would not feel very Java). The other attempt to convert int -> String (byte) uses the naive way to divide by 10 on each iteration, Java.'s Integer.toString does it way better.

Edit: on a 2nd thought, having a dedicated direct buffer [same allocation, different slices] per all the 8 out of 15 ares for numbers and NOT converting int->String each operation but adding 15 would be a pretty boon as most of the time the change would be only the last 2 bytes, and there won't be any 'div' to be had. Div is generally slow (compared to L1/L2 cache misses, and L3 hit), there is not algorithm to parallelize it, and there is one (few) unit that can perform div, unlike 'add')

It doesn't matter for FizzBuzz, but in general JIT should really out-perform "native" statically compiled compilers on most real-world workloads. While there's a small penalty for the profiling JIT needs, there are massive gains for data and access dependent optimizations that static compilation can't achieve without explicit hints. For example, a JIT can observe the most commonly accessed data and re-order it to optimize for the pre-fetching pipeline. Or it can re-order operations to optimize for cache-line hotness.

> but in general JIT should really out-perform "native" statically compiled compilers on most real-world workloads.

In theory, yes, but in practice I've never seen it happen. Best I ever saw was matching C speed at toy benchmarks. Even in this benchmark here, Java is decent, but does not beat even the naive implementations in C/Rust.

Also, AOT compilers can do PGO as well, so they can use the same techniques. But they also have way more time and resources, so they can do things like whole program optimization, which is something JITs cannot do because they have much smaller computation and memory budget.

It happened already many times to me that the first naive version of a C/C++/Rust program/function I wrote was already faster than a carefully tuned Java equivalent. AOT compilers for "fast languages" got really good these days. The design of language also influences how well it can be optimized by the compiler. E.g. it might look impressive JVM can devirtualize dynamic calls at runtime, but C++/Rust often don't have to do this at all as programs in those languages tend to have very few virtual calls if any at all.

If you didn't need more evidence, the fact that the Java community has been slowly moving towards AoT with GraalVM should cement it. Java's JITs are damn impressive pieces of engineering, but the structural advantages of traditional compilation still win out surprisingly often.

The move towards AOT is mostly for lower startup times and lower memory usage, not to make the application faster in general.

>Java is decent, but does not beat even the naive implementations in C/Rust.

Java doesn't beat C in this benchmark but beats Rust with ease.

There is not enough data to conclude that. There are 3 Java implementations slower than the faster Rust one, and there are 2 Java implementations faster. But there are only 2 data points for Rust in total.

Also, technically there is nothing C can do that Rust can't.

Will V8 be able to do the same some day?

Java is a lot better source (and target) to optimize for than JS. It's quite bit more work to have all numbers being floating point by default.

My understanding is that a statically typed language is inherently faster than a dynamically typed one, no matter how much you optimise the latter. The JS interpreter/VM simply has a lot more work to do than the JVM.

No. Well…it’s complicated.

I think WASM will be the mainstream for frontend before that happens... But you never can't predict the future!

Java is only slow to start the VM. If your process is long-lived it's exceedingly quick because you only pay the startup penalty once.

"GC pauses" are greatly exaggerated in terms of impact and frankly for the vast majority of uses cases GC simply doesn't become an issue.

The JVM is really really good because at one point or another they had basically every luminary in the field working on it.

> Java is only slow to start the VM

And it's not even that slow anymore compared to, say, starting the JVM in 2010.

Some numbers for those curious...

On modern hardware launching the JVM to run a program immediately exiting takes less than 100 ms (so does starting Emacs complete with its GUI and running some elisp code exiting Emacs: 80 ms on my Ryzen 7000 series including reading from the M.2 NVMe PCIe 4.0 x4 SSD).

It's once you start loading lots of classes that JVM startup time can be slow.

One example would be a Clojure program doing nothing besides exiting: thousands of Java classes being used and you get into 1.2 seconds territory to do nothing. 12x slower than a Java program doing nothing.

As a sidenote for both Java and Clojure there are now ways to reduce startup time, like using GraalVM (which, for example, Babashka, a natively compiled Clojure interpreter, making Clojure startup so fast it can be used for scripts).

> The JVM is really really good because at one point or another they had basically every luminary in the field working on it.

I agree. The JVM is an impressive piece of machinery and it'll even give you, say, an AIOOBE (ArrayIndexOutOfBoungsException) instead of an exploit if you fuck up.

>so does starting Emacs complete with its GUI

I'm with you all the way and am a daily emacs user but I'm not sure I'd point to emacs as a good performance comparison, haha.

Emacs has always been a bit of a dog in my experience.

The fear of GC pauses is mostly outdated. It was right in the past, but it is becoming less and less of an issue with the new GC work coming online. And for small heaps in web application servers, the pauses were a blip on the high percentiles of response times and you usually had a dozen other issues before that.

It's similar to SWAP in Linux. It was implemented somewhat meh in older kernels, and usually if swap got hit, the system died anyhow. So there was no real difference between random processes being OOM-killed or the system grinding to a halt swapping. Modern kernels in the 4+ line have received quite a bit of work on the swap handling and swap is used a lot and very cleverly to eek out just a bit more available memory more quickly.

Old habits die hard though and it takes time for old knowledge to change.

The problem of GC pauses is solved only if you can afford to waste 5x-20x more memory than the app is really using. Otherwise those low-pause modern collectors can burn a significant number of CPU cycles or even fall into emergency STW path (when they can't keep up with garbage production rate).

If the allocations die trivially (the extreme/vast majority do) in the young gen, STW doesn't occur. Indeed being aware, how the GC operates is important to write a decent Java application. So is understanding how the hardware works in general.

> If the allocations die trivially (the extreme/vast majority do) in the young gen

Low pause GCs (ZGC, Shenadoah) for Java are not generational yet.

Also, even if they were, a very high temporary object allocation rate increases young gen GC frequency and thus increases the number of objects pushed to old gen eventually.

And it is not like those young gen GCs are free either. They burn quite some CPU time and they cause micro-pauses - the GC has to scan parts of the heap to learn which objects are reachable and then it has to copy the survivors. In practice, high allocation rate requires a decent amount of overhead RAM to make that process efficient. It doesn't matter if pauses are only 10 ms short if you do 50 of them per second. ;)

In non-GCed languages those trivial short-term objects are typically allocated on stack and their allocation/deallocation is trivial and doesn't pause at all.

>Low pause GCs (ZGC, Shenadoah) for Java are not generational yet.

They are. GenZGC is merged into mainline and got into the latest JDK, GenShenandoah didn't manage but will be in next release.

>The problem of GC pauses is solved only if you can afford to waste 5x-20x more memory than the app is really using

I don't think that is true. ZGC uses multi-mapping in order to dereference its colored pointers, this causes some tools to report excessive memory usage but nothing actually hits RAM.

>the GC has to scan parts of the heap

Hmm, for generation GCs, it should use Card marking, the pointers in the tenured gen should have means to be trivially determined where they belong to. With 64bit pointers, there is space for quite a lot of metadata, incl. the Class (or most commonly allocated/used classes).

Card marking does not identify pointers directly, but only identifies parts of the heap that may contain pointers to the new gen. So those parts still need to be scanned. Surely, it is way less work than scanning the whole heap, but still non-zero.

> it is becoming less and less of an issue with the new GC work coming online

I feel like I've been hearing this same line for 20 years now

Java has a ridiculously fast new operator. Much faster than the one in C++.

It works by allocating all or most needed memory at program start, instead of asking the operating system for it every time. But, as soon as you don't use heap memory, and use the stack, C++ is again much, much faster than Java.

It all depends on memory management.

C++'s new operator can also do this using "placement new" semantics[1], which usually means making (and using) a custom allocator. You're still using the normal new operator, just passing extra parameters to do the placement you want.

To your second point about raw speed, one of the extraordinary things we found in long-running processes performing computation using real-time marketdata at Goldman was that the HotSpot JVM was able to optimize java programs through the day due to their usage, so if you started them each day they would actually end up faster than the C++ versions at the end of the day even though they would start off slower. That's not due to memory allocation it's due to things like inlining of functions.

The implication of that is that if you very carefully inlined all the functions appropriately in the C++ version based on profiling actual usage you would be able to achieve the same result, but for the JVM it just happens automatically without you doing anything.

[1] Search https://en.cppreference.com/w/cpp/language/new for "Placement New"

> Java has a ridiculously fast new operator. Much faster than the one in C++.

Can you point us to some benchmarks?

I don't doubt it is faster, but I doubt the "ridiculously" part. Last time I measured it was only a tad faster than jemalloc (~20%) if you allowed the benchmark to run long enough for GC to start cleaning up. Unfortunately I haven't saved it (it was a very informal benchmark). I'm curios to see that esp. on modern low-pause GCs.

It's just pointer bumping (still). If you're continously using new memory then soft page faults and finding physical backing via kernel will dominate. mallocsen want to avoid this scenario, so you often see malloc using more memory than necessary.

Anyway, don't use new in C++ is my opinion.

It's pointer bumping in optimistic case. Eventually it runs out of current slab and... then there is some more work to do. And also those page faults are actually quite likely as it gives you memory that hasn't been used recently. Malloc tends to return often whatever was released last that matches the requested allocation size. It is indeed a tad more work to find it, but when it's found it's usually hot in the cache.

Imo it's one of Javas strengths. If you need to, you can write ugly but ridiculously fast code. It does end up looking more like writing C than java though, but obviously your whole codebase doesn't need to look like that and it's portable.

The only part Java lacks - headless objects (aka project Valhalla). It has been mimicked with direct buffers and offset for ~20y now (since Merlin release). Other than that java is similar to a guided "gcc -O2"

Sure, you can write Java like C and get performance typically within 0.3x to 1x of C. But Java written in this style is even less ergonomic than C is, so if you have to do do it, there is not much point in using Java.

It is quite common that you only need to optimize very small parts of a program to this level. The rest of the program can be written in more conventional styles.

You could of course FFI into e.g. C for those parts, but that is usually harder to maintain than a few well optimized java classes.

> It is quite common that you only need to optimize very small parts of a program to this level.

It's a quite common myth developers believe about performance. Hotspots do happen sometimes, but once they have been optimized you quicky end up with a flat profile and an "everything is slow" problem. And in some types of apps, the majority of code is performance critical.

It depends - even when you run into an "everything is slow" problem, it might be that it's like 1 endpoint out of 2000 that causes performance issues. In this case, you might need to focus very much on performance for that endpoint, but maybe not for other endpoints. Profilers can help you figure out what code to focus on.

If the majority of code is performance critical, the tradeoffs are of course different.

Nah, "everything is slow" is vanishingly rare in practice.

Depends on the application. In area of compilers, databases, CAD, game engines, simulation, distributed analytics, machine learning the only code that isn't on the critical path is some configuration / control plane / UI - which is minority of the code.

Certainly not in the entirety of those areas. I've worked in a couple of them and functionality was a higher priority than performance and we picked our tools accordingly.

The cases you’re talking about are the rare ones. Source: I did this professionally and now recreationally.

> You could of course FFI into e.g. C for those parts, but that is usually harder to maintain than a few well optimized java classes.

Hopefully Foreign Function & Memory API [0] makes FFI so much easier that we get to drop down to C without much fuss.

[0] https://openjdk.org/jeps/442

Reply on news.ycombinator.com